Scientific Research Methodology and Data Analysis
This domain tests experimental design (controls, randomisation, replication, blinding), measurement quality, the choice of statistical test, and the correct reading of p-values, confidence intervals and effect sizes.
Core rules and concepts
- State hypotheses before seeing data; the null hypothesis asserts no difference or no association.
- Randomise allocation to treatments, include a concurrent control, and replicate at the level of independent experimental units to avoid pseudo-replication.
- Precision is agreement among repeats; accuracy is agreement with the true value. Both matter and neither guarantees the other.
- A p-value is the probability of data at least as extreme as observed if the null hypothesis were true - it is not the probability that the null is true.
- A 95% confidence interval gives the range of plausible values for the parameter; an interval excluding zero corresponds to significance at α = 0.05 in a two-sided test.
- Match the test to the design: two independent means → unpaired t-test; three or more group means → ANOVA; counts in categories → chi-square; two continuous variables → correlation or regression.
Worked example
A trial compares a new fertiliser with current practice on 12 plots each. The mean yield difference is 320 kg/ha, with a 95% confidence interval of 90 to 550 kg/ha and p = 0.02. The difference is statistically significant: p < 0.05 and the interval excludes zero. The plausible true gain lies between 90 and 550 kg/ha. If the extra cost of the fertiliser is recovered only above a 200 kg/ha gain, the evidence is promising but not conclusive, because part of the interval falls below that break-even point.
Traps that cost marks
- Writing that p = 0.02 means a 2% chance the null hypothesis is true.
- Calling a non-significant result proof of no effect; the interval may simply be too wide.
- Running many comparisons without correction, then reporting the smallest p.
- Inferring causation from observational correlation.
- Measuring several sub-samples from one plot and treating them as independent replicates.