PRF common mistakes
PRF · Understand the traps and how to avoid them.
When asked which design best answers a causal question, the candidate picks a cross-sectional or case-control design because it is described as 'efficient' or 'feasible', instead of matching the design to the temporal and confounding demands of the research question.
Candidates memorize design labels and their textbook strengths in isolation, then pattern-match on buzzwords like 'efficient' or 'low cost' rather than reasoning from the question's causal structure (temporality, comparison group, control of confounding).
What to do instead
Before selecting a design, write three lines: (1) the exposure and outcome, (2) whether the question is descriptive, associative, or causal, and (3) whether exposure must be measured before outcome. Then eliminate any design that cannot establish temporality or lacks a concurrent comparison group. Practice by justifying rejection of each distractor, not just endorsing the key.
The candidate treats 'obtaining IRB approval' as equivalent to 'the study is ethical', and answers ethics items by citing committee review alone, ignoring consent process quality, vulnerability, coercion, or ongoing risk monitoring.
Training often reduces research ethics to a compliance checkpoint, so candidates conflate regulatory permission with ethical justification and miss items probing respect for persons, justice, or beneficence.
What to do instead
For every ethics item, test the scenario against the three Belmont principles explicitly (respect for persons, beneficence, justice) and ask whether consent is voluntary, informed, and ongoing. Distinguish procedural compliance (approval, forms) from substantive ethics (coercion, undue inducement, confidentiality limits). Build a one-page checklist you apply to each vignette.
When appraising a systematic review or meta-analysis, the candidate judges quality by journal prestige or by the number of included studies, and ignores heterogeneity, publication bias assessment, risk-of-bias appraisal, and whether pooling was appropriate.
Candidates default to superficial credibility cues because they have not internalized the specific appraisal domains (search completeness, risk of bias, heterogeneity, selective reporting) and so cannot evaluate the methods critically.
What to do instead
Appraise in a fixed order: search strategy and reproducibility, inclusion criteria, risk-of-bias tool used, heterogeneity statistic and its interpretation, funnel plot or Egger test for small-study effects, and sensitivity analyses. Ask 'would pooling these studies produce a meaningful summary?' before accepting a pooled estimate. Practice on abstracts by listing what appraisal information is missing.
The candidate treats a Likert scale as interval data and computes means and parametric tests without checking whether the instrument's response format, scoring, and validation support that level of measurement.
Statistical software accepts any numeric code, and candidates learn 'if it's a number, use a t-test' without linking measurement properties to permissible analyses. They also confuse a Likert item (ordinal) with a summed Likert scale (often treated as approximately interval after validation).
What to do instead
Classify each variable's scale of measurement before choosing a test, and state the assumption explicitly. For single ordinal items, use medians and nonparametric tests; for validated multi-item summated scales, justify interval treatment by citing the instrument's validation evidence. Write the measurement level next to each variable in your analysis plan.
The candidate selects a statistical test by matching keywords in the question stem ('compare groups' → t-test) rather than checking the number of groups, pairing, distributional assumptions, and whether the outcome is continuous, categorical, or time-to-event.
Keyword-to-test mapping is fast and feels reliable under time pressure, so candidates skip the assumption-checking step that determines whether a t-test, ANOVA, chi-square, or nonparametric alternative is correct.
What to do instead
Use a decision tree keyed to four questions: how many groups, paired or independent, what is the outcome type, and are parametric assumptions met. Practice by writing the decision path aloud for each item before looking at options. When two tests seem plausible, identify which assumption distinguishes them.
The candidate interprets a p-value as the probability that the null hypothesis is true or the probability that results are due to chance, and treats 'not significant' as evidence of no effect.
This is the most commonly taught misconception; candidates absorb the shorthand 'p < .05 means significant' without learning what the p-value actually quantifies, so they endorse intuitive but incorrect interpretations.
What to do instead
Memorize the correct definition: the probability of observing data at least as extreme as obtained, assuming the null hypothesis and model are true. Distinguish it from effect size, confidence intervals, and the probability of replication. For each interpretation item, ask 'does this statement describe the data given the null, or the null given the data?' Only the former is correct.
In qualitative items, the candidate judges rigor by sample size or generalizability, dismissing a small purposive sample as weak, and fails to evaluate trustworthiness criteria such as credibility, transferability, dependability, and confirmability.
Candidates apply quantitative quality heuristics (large n, random sampling, statistical generalization) to qualitative work, missing that qualitative rigor is established through triangulation, member checking, audit trails, and reflexive documentation.
What to do instead
Learn the trustworthiness framework and map each criterion to its technique (credibility → triangulation/member checking; transferability → thick description; dependability → audit trail; confirmability → reflexivity). When evaluating a qualitative design, ask whether sampling was purposeful and information-rich, and whether analytic saturation or an equivalent stopping rule was justified.
The candidate stores or shares research data without de-identification, version control, or a documented data management plan, and answers data-governance items by focusing only on password protection.
Data management is often treated as an administrative afterthought, so candidates underestimate risks from indirect identifiers, uncontrolled file versions, and undocumented transformations that undermine reproducibility.
What to do instead
Build a data management plan checklist: data types and identifiers, de-identification method, storage location and access controls, versioning conventions, retention period, and sharing agreement. For governance items, distinguish technical security (encryption, access control) from procedural reproducibility (documented cleaning, codebooks, versioned datasets) and address both.
When asked how findings should be reported or disseminated, the candidate selects the option that maximizes reach or impact (press release, social media, conference) rather than matching the channel to the audience and ensuring accurate, complete, pre-specified reporting.
Candidates equate dissemination with publicity and overlook reporting standards (complete methods, pre-registered outcomes, limitations, negative results) and audience-appropriate formats for policymakers, clinicians, or participants.
What to do instead
Separate two questions: 'is the report complete and unbiased?' and 'is the channel right for this audience?'. Check reporting against relevant guidelines (CONSORT, STROBE, PRISMA, COREQ as appropriate), flag outcome switching or selective reporting, and choose dissemination by audience need rather than visibility. Practice identifying the missing reporting element in a short methods summary.
The candidate confuses internal and external validity, labeling a study with tight inclusion criteria as 'invalid' because results do not generalize, or praising a convenience sample as strong because it is large.
The two validity types are taught together and their trade-off is often memorized as a slogan rather than understood, so candidates misattribute generalizability problems to internal validity and vice versa.
What to do instead
Define each explicitly before answering: internal validity concerns whether the observed effect is causal within the study (bias, confounding, measurement error); external validity concerns whether findings extend to other populations, settings, and times. For each threat listed in an item, label it internal or external and state the direction of the trade-off. Size alone affects precision, not either validity type.