Three Soils, One Vintage: A Terroir Dissertation Under Budget
A Master of Wine candidate has 12 weeks and a £4,000 budget to test whether soil type shapes the sensory profile of Riesling. A single commercial estate offers three adjacent parcels on slate, loam and limestone, each about 0.4 ha. The winemaker will provide four 225-litre barrels per parcel, all fermented with the same commercial yeast. Fruit from the slate parcel is machine-harvested; the loam and limestone parcels are hand-harvested. The candidate plans to convene nine tasters - six trained, three novices - and ask them to rate liking on a nine-point scale, then run a one-way ANOVA comparing the three soil types. The dissertation question is: 'Does soil type produce a measurable sensory signature in Riesling?' The candidate wants a statistically significant result to support a terroir claim.
1. Identify the most serious threat to internal validity in the proposed comparison and explain how it could create a false soil effect. Then state the minimum design change needed before any causal soil claim is defensible.
Read the worked answer
The most serious threat is pseudoreplication at the parcel level: soil type is confounded with parcel identity. There is one slate parcel, one loam parcel and one limestone parcel, so any difference among the three wines could be due to aspect, drainage, vine age, rootstock, pruning or winemaking logistics rather than soil. A significant ANOVA would only show that these three parcels differ; it cannot attribute the difference to soil. The harvest-method difference - machine on slate, hand on loam and limestone - is a second confounder because it changes fruit integrity, extraction and possibly reductive characters. The panel composition is a third issue, but it is secondary to the lack of replication.
Minimum change: replicate soil types across at least two independent parcels or, better, two vineyards, and standardise harvest method across all parcels. If that is impossible within 12 weeks, the study must be reframed as an exploratory single-vineyard case study and the dissertation must not use causal soil language. A defensible claim would be: 'In this estate and vintage, wines from these three parcels differed in X, but soil type cannot be isolated from parcel effects.'
2. Redesign the sensory component within the same budget and 12 weeks to produce the strongest defensible evidence, and specify what inference you would and would not make.
Read the worked answer
Use a trained descriptive panel, not a liking panel, because the question is about sensory signature, not preference. Recruit 10–12 assessors, all trained on a defined lexicon for Riesling (for example citrus, stone fruit, mineral, petrol, acidity, body). Calibrate with reference standards before the trial. Present wines blind, monadically, in randomised balanced order across three replicates, coded with three-digit numbers. Include a warm-up and forced rest; use expectoration and water. Collect intensity on 15-cm line scales, not nine-point liking. Analyse with a mixed model: parcel or soil as fixed, assessor as random, replicate as random; report effect sizes and 95% CIs, not only p-values. Keep novice data separate or exclude by a pre-specified protocol; do not exclude after seeing results. With £4,000, pay modest honoraria and use the estate's barrels; do not add a second site unless free. Inference permitted: 'trained assessors perceived differences among these three parcel wines in this vintage under blind conditions'. Inference not permitted: 'soil type causes the difference' or 'the difference generalises to Riesling'. State that parcel and harvest method remain confounded.
3. The candidate proposes to drop the three novice tasters because their responses are noisier and significance improves. Evaluate this decision.
Read the worked answer
Dropping assessors after inspecting the data is a form of data-driven exclusion. It inflates the false-positive rate and makes the reported p-value uninterpretable unless the exclusion rule was pre-specified and applied blind. It also changes the target of inference: the trained panel is an analytical instrument, while novices represent a consumer-like response; mixing them was a design error, not a data-cleaning problem. The correct response is to pre-specify the panel in the protocol, report both full-panel and trained-only results as a sensitivity analysis, and explain why. If the trained-only result is the primary analysis, say so before seeing outcomes. With only nine tasters, removing three also weakens power and may leave insufficient replication; the candidate should not present the improved significance as evidence of a soil effect. The honest conclusion may be that the study is underpowered and that panel composition should be fixed in future work.