BCS Written Statistics common mistakes
BCS Written Statistics · Understand the traps and how to avoid them.
When asked to find the probability of a union or intersection of events, the candidate directly multiplies marginal probabilities without checking independence, or adds probabilities without subtracting the intersection.
The candidate conflates the multiplication rule for independent events with the general multiplication rule and forgets the inclusion-exclusion principle. This often stems from memorizing formulas without understanding the underlying set logic and conditions.
What to do instead
Always write down the general formulas: P(A∪B)=P(A)+P(B)-P(A∩B) and P(A∩B)=P(A)P(B|A). Before computing, explicitly state whether events are independent, mutually exclusive, or neither. Use a Venn diagram or contingency table to visualize the relationships and verify that the conditions for any shortcut apply.
In hypothesis testing, the candidate states conclusions as 'accept H0' or interprets the p-value as the probability that H0 is true, rather than making a decision about the evidence against H0.
This reflects a fundamental misunderstanding of frequentist inference: the p-value is the probability of observing data as extreme or more extreme given H0, not the probability of H0. The candidate may also be influenced by everyday language that treats 'accept' as synonymous with 'prove true'.
What to do instead
Practice writing conclusions in the form: 'At the α=0.05 significance level, there is sufficient/insufficient evidence to conclude that [alternative in context].' Explicitly define the p-value in every practice problem and contrast it with the probability of H0. Use simulation or examples to reinforce that H0 is never accepted, only not rejected.
When conducting a one-way ANOVA, the candidate checks only the overall F-test and, if significant, concludes that all group means differ without performing any post-hoc multiple comparison procedure.
The candidate overlooks that a significant F-test only indicates that at least one mean differs, not which ones. This often happens because the candidate focuses on the mechanics of computing the F-statistic and forgets the exploratory follow-up required for a complete analysis.
What to do instead
Always pair the ANOVA F-test with a planned or post-hoc multiple comparison method (e.g., Tukey's HSD, Bonferroni). In practice, write a complete conclusion that first states the overall test result and then identifies which specific pairs differ, adjusting for multiple comparisons. Emphasize that the ANOVA is an omnibus test.
When asked to select a sampling method for a survey, the candidate labels a convenience sample as 'simple random sampling' or 'quota sampling' without recognizing the lack of randomization and the resulting selection bias.
The candidate may not fully grasp the defining feature of probability sampling: every unit has a known, nonzero chance of selection. They might focus on surface features (e.g., using a list) rather than the random mechanism, or they may confuse quota sampling (non-probability) with stratified sampling (probability).
What to do instead
Memorize the definitions and key distinctions: probability methods (SRS, systematic, stratified, cluster) require random selection at some stage; non-probability methods (convenience, quota, purposive) do not. For each scenario, explicitly state whether randomization is used and whether inclusion probabilities are known. Practice classifying examples and explaining the bias implications.
In multiple linear regression, the candidate interprets a regression coefficient as the marginal effect of that predictor on the response, ignoring that it is the effect holding all other predictors constant, and fails to check multicollinearity.
The candidate may treat regression as a simple extension of correlation and not appreciate the conditional nature of partial coefficients. Additionally, they may not realize that high correlation among predictors inflates standard errors and makes individual coefficients unstable and hard to interpret.
What to do instead
Always phrase interpretations as: 'For a one-unit increase in Xj, holding all other predictors fixed, the expected response changes by βj units.' Before interpreting coefficients, compute variance inflation factors (VIF) or condition indices; if multicollinearity is high (e.g., VIF>10), consider removing or combining predictors, or use ridge regression. Emphasize that regression coefficients are partial effects.
When forecasting with an ARIMA model, the candidate selects the model solely by minimizing AIC or by eyeballing the autocorrelation function, without checking stationarity or performing residual diagnostics.
The candidate may treat model selection as a purely mechanical exercise and overlook the assumptions required for valid inference and forecasting. They might also confuse the roles of ACF and PACF in identifying p and q, or forget that differencing is needed to achieve stationarity.
What to do instead
Follow a systematic Box-Jenkins approach: (1) plot the series and check stationarity (e.g., ADF test); difference if needed; (2) examine ACF/PACF to propose candidate orders; (3) estimate models and compare using information criteria; (4) conduct residual diagnostics (ACF of residuals, Ljung-Box test) to ensure white noise; (5) validate forecasts with out-of-sample performance. Never skip diagnostics.
When calculating a crude death rate or a standardized mortality ratio, the candidate uses the total population as the denominator without adjusting for age structure, leading to misleading comparisons between populations with different age distributions.
The candidate may not understand that crude rates are influenced by population composition and that age-standardization is necessary for fair comparisons. This often arises from a focus on formula plugging without considering the demographic context.
What to do instead
Always distinguish between crude, specific, and standardized rates. When comparing mortality across populations, compute age-specific rates and then direct or indirect standardization. Explain in words why standardization removes the confounding effect of age. Practice with datasets having different age structures to see the impact.
When writing R or Python code for a statistical analysis, the candidate fails to set a random seed before simulation or cross-validation, and does not check for missing values or factor variable coding, leading to irreproducible or incorrect results.
The candidate may treat coding as an afterthought and not appreciate the importance of reproducibility and data preprocessing. They might also assume that software automatically handles missing data or encodes categorical variables correctly, which is often not the case.
What to do instead
Adopt a reproducible workflow: always set a seed (e.g., set.seed(123) in R or np.random.seed(123) in Python) before any random operation. Before modeling, inspect the data with summary() and str() to identify missing values, variable types, and factor levels. Explicitly handle missing data (e.g., complete-case analysis or imputation) and ensure categorical variables are coded as factors. Document all steps in a script.
In experimental design, the candidate analyzes a randomized complete block design (RCBD) as if it were a completely randomized design (CRD), omitting the block effect from the model and inflating the error mean square.
The candidate may not recognize the role of blocking in reducing variability due to nuisance factors, or may not know how to set up the ANOVA table with block and treatment sources. This often stems from a lack of understanding of the design structure and the corresponding linear model.
What to do instead
For every experiment, first identify the design (CRD, RCBD, factorial, etc.) and write the appropriate linear model, including all blocking factors. In RCBD, the model is Yij = μ + τi + βj + εij, and the ANOVA table must include a block term. Practice computing and interpreting the relative efficiency of blocking. Always check that the analysis matches the design.
When constructing a confidence interval for a population mean, the candidate uses the critical value z* instead of t* when the population standard deviation is unknown and the sample size is small, or uses the wrong degrees of freedom.
The candidate may have memorized the z-interval formula and applies it indiscriminately, not recognizing that the t-distribution is required when σ is estimated from the sample. They may also forget that degrees of freedom for a one-sample t-test is n-1.
What to do instead
Always check whether σ is known. If unknown, use the t-distribution with n-1 degrees of freedom. For small samples, verify the normality assumption or use nonparametric alternatives. Write down the formula with t* explicitly and state the degrees of freedom. Practice with examples where n is small (e.g., n<30) to reinforce the distinction.