Scientific Reasoning and Evidence Evaluation
This domain tests whether you can tell a supported scientific claim from a persuasive one. Scenarios present a short news item, an advertisement or two conflicting studies, and ask you to judge the quality of the evidence, name the reasoning used, and state what conclusion is actually licensed.
What counts as evidence
- Controlled, repeated measurement by an independent group ranks above a single uncontrolled observation, which ranks above a testimonial from a satisfied user.
- Anecdotes are data points without controls: they cannot separate the treatment from expectation, natural recovery, or other simultaneous changes.
- Peer review and replication are quality filters, not guarantees. Funding source and conflict of interest still matter.
Reasoning patterns to check
- Correlation versus causation. Ice-cream sales and drowning deaths rise together; summer heat is the confounder. Always ask whether a third variable explains both.
- Confounding variables are differences between groups other than the one being tested - age, income, health status - that could produce the result.
- Generalisation. Twelve university students cannot support a claim about "people".
- Selection bias. Volunteers who answer a survey are not a random sample.
Hypothesis, theory and law
A hypothesis is a testable, falsifiable prediction; a law describes a regular relationship, often mathematically; a theory is a well-tested explanatory framework. In science, "theory" never means "guess".
Worked example
A manufacturer reports that 12 volunteers aged 18–25 scored 8 % higher on a memory test after four weeks of a herbal capsule, with no placebo group. Verdict: the design cannot support the claim. Without a comparison group, practice effect, expectancy and normal variation all remain live explanations; the sample is small and unrepresentative; and the seller profits from a positive result. A stronger design adds a placebo, randomises assignment, blinds participants and testers, and uses a larger, broader sample. The study is not "proven false" - the correct conclusion is "insufficient evidence".
Traps
- Accepting a claim because it appeared in a newspaper or on a familiar website.
- Writing "proves" where the data only suggest or are consistent with.
- Dismissing a study for small sample size without naming what that limits: generalisability.
- Reversing the direction of a causal claim.