Interpreting survey data on 반말 in Korean workplaces
A student investigating changing address norms in Korean workplaces ran an online survey through KakaoTalk among family and family friends in Seoul and Sydney. Recruitment began with the student's own relatives. n = 40. The single question was: "직장에서 나이가 많은 동료에게 반말을 사용한 적이 있습니까? (예/아니요)" Results: respondents in their 20s-30s (n = 22): 68% 예; respondents in their 40s and over (n = 18): 22% 예. Fifteen of the 예 respondents added free-text comments such as "정말 친한 경우에만", "같은 팀이고 자리가 편할 때", and "원래는 존댓말인데 그분이 먼저 편하게 하자고 하셨어요". The draft conclusion reads: "한국 직장에서 반말 문화가 빠르게 확산되고 있다."
1. Identify the problems in the draft conclusion. Separate problems with the evidence from problems with the wording.
Read the worked answer
Problems with the evidence:
- Cross-sectional, not longitudinal. A single snapshot cannot show 확산 (spread) or 빠르게 (rapidly). A trend claim needs at least two time points or a retrospective measure ("3년 전과 비교하면").
- The question confuses age with rank. "나이가 많은 동료" is not the same as 상사. Korean workplaces run on two overlapping axes - age and 직급 - and a 반말 decision may be driven by either. The data cannot tell you which.
- The qualifiers contradict the headline. "정말 친한 경우에만", "같은 팀이고 자리가 편할 때", "그분이 먼저 편하게 하자고 하셨어요" describe narrow, licensed use, often initiated by the senior person. That is the opposite of a general norm shift. 15 of the 예 responses are effectively conditional yeses and were coded as plain yes.
- Sampling. Convenience sample recruited through the student's relatives: correlated respondents, one occupational network, two cities. Self-report on a politeness norm is also subject to social desirability in both directions.
- Coding. Age was binarised at 30/40 with no data on company size, industry, or tenure. Age and cohort are confounded - a 50-year-old in 2024 is not a 30-year-old in 2044.
Problems with the wording:
- "문화" totalises: it turns 40 reports into a national characteristic.
- "빠르게" is an intensity claim with no measured rate.
- The conclusion restates the hypothesis as a finding - no hedging ("~경향이 있다", "~로 보인다") is present.
2. Rewrite the research question so that the data collected could actually answer it.
Read the worked answer
Rewrite: "서울과 시드니에 거주하는 한국계 직장인 40명을 대상으로, 같은 팀의 나이가 많은 동료에게 반말을 사용하는 빈도와 그 사용이 허용되는 조건(친밀도, 직급 차이, 회사 규모, 먼저 제안한 사람)은 어떻게 나타나는가?"
Why this is answerable:
- "빈도" replaces the binary 예/아니요, so a respondent who uses 반말 only with one close colleague is no longer identical to one who uses it team-wide. A 4-point frequency scale (거의 매일 / 가끔 / 아주 드물게 / 전혀) plus the existing free-text field would have captured the qualifiers that were already in the data.
- "허용되는 조건" turns the anomaly into the object of study. The 15 conditional comments are the most interesting result; the rewritten question makes them the finding rather than noise.
- "먼저 제안한 사람" distinguishes a junior person claiming a new norm from a senior person granting permission - two completely different social facts that the draft conclusion merges into one.
- Scope is stated (40 people, two cities, same team). The conclusion can then only be about those respondents.
What to drop: "빠르게 확산" and "문화". If a trend is the real interest, the question must become longitudinal and the design must change.
Standard reframe: a study of this size can describe conditions of use; it cannot describe cultural change.
3. You are allowed to add three more data points before submitting. Which three, and what would each rule out?
Read the worked answer
1. 회사 규모 (employee count). Korean workplace speech norms vary sharply between a 5-person startup and a 500-person conglomerate, where honorifics are reinforced by formal reporting lines. Without this, a difference you attribute to age may be entirely a difference between two employers. Rules out: the industry/scale confound.
2. 관계 기간 and 직급 차이 (months known; rank of the person addressed). This separates the two axes the original question merged. If 반말 use clusters with short 관계 기간 and same-rank colleagues, the finding is about peer intimacy, not about hierarchy loosening. Rules out: the age-vs-rank confound.
3. A retrospective point: "3년 전에는 어땠습니까?" or, better, one observational data point - recorded or reported use in a single meeting. Retrospective recall is imperfect but it is the cheapest way to get a second time point, and it converts a snapshot into a weak trend measure. Rules out: the "확산" claim having no temporal evidence at all.
Ideal but not available within the limit: a matched sample of colleagues from the same company, so that company culture is held constant.
Overarching judgement: three variables do not rescue n = 40. The correct move is to narrow the claim to "이 40명의 응답에서는" rather than to add variables and keep the grand conclusion. In an in-depth study, a modest, well-bounded finding scores higher than an unsupported large one.