A Supplement Claim and a Pharmacy Notice
A news item reports that a 'new supplement boosts memory by 23%'. The underlying study recruited 42 adults from a supplement company's customer mailing list. Participants rated their own memory on a 1–10 scale before and after eight weeks of taking the supplement. There was no placebo group and no blinding. Nine participants dropped out; the report analyses the 33 who completed. Mean self-rated memory rose from 5.1 to 6.3. The company funded the study, which was not published in a peer-reviewed journal. A local pharmacy asks you to write a short customer notice about the product. The pharmacy manager says, 'Just say it improved memory by 23% - that's what the study found.'
1. Identify the main design features of the study and explain whether each one would tend to exaggerate or understate the reported improvement.
Read the worked answer
No control or placebo group. Participants knew they were taking a supplement and may have expected improvement. Expectancy and regression to the mean can both raise post-test scores. Someone who rates memory 5.1 at baseline may be having a bad week; a later rating of 6.3 may partly reflect ordinary fluctuation. This exaggerates the apparent effect.
Self-rating, unblinded. Self-reported memory is subjective and susceptible to demand characteristics: participants may want to justify their effort or please the company. This exaggerates the effect.
Recruitment from a customer mailing list. People already interested in supplements are not representative, and they may be more motivated to report benefits. This limits generalisability and may exaggerate the effect.
Dropouts. Nine of 42 left (21%). If dropouts were people who found no benefit, analysing only completers exaggerates the improvement. The report does not say why they left.
Funding and no peer review. Commercial funding does not automatically invalidate a study, but it removes an independent check on analysis choices. It does not by itself bias the number, but it lowers confidence.
Overall, the design is capable of showing a difference even if the supplement does nothing. A control group would not necessarily understate a real effect, but its absence means the observed change cannot be attributed to the supplement.
2. Calculate the absolute and relative change in self-rated memory, and explain why the '23%' figure alone is misleading.
Read the worked answer
Absolute change = 6.3 − 5.1 = 1.2 points on a 10-point self-rating scale. Relative change = 1.2 ÷ 5.1 × 100 = 23.5%, which rounds to 23% or 24%. The arithmetic is consistent with the news item.
The percentage alone is misleading because it has no denominator context and no comparison group. A 23% change on a subjective 1–10 scale is a 1.2-point shift; the reader cannot tell whether that is large, small, or within normal week-to-week variation. If the scale were 1–100, the same 23% would be a different absolute change. Also, the 23% is calculated only for the 33 completers; the 9 dropouts are excluded. If they had unchanged or lower scores, the group average change would be smaller.
The item also gives no standard deviation, confidence interval or test result, so it is impossible to judge whether 1.2 points is more than noise. The honest summary is: 'Self-rated memory rose by 1.2 points in 33 volunteers; without a control group, the cause cannot be determined.'
3. Draft the key paragraph of the pharmacy's customer notice in 90–110 words. It must be accurate, understandable and not claim official approval.
Read the worked answer
Draft: 'A small unpublished study reported that volunteers who took this supplement rated their memory 1.2 points higher on a 1–10 scale after eight weeks. The study had no placebo group, and participants knew what they were taking, so the improvement may not be due to the supplement. Thirty-three of 42 volunteers completed the study, and the research was funded by the company. These results do not prove that the product improves memory. If you are considering it, check with your pharmacist, especially if you take other medicines or have a medical condition.'
Why this wording works: it gives the absolute change, not just the percentage; it states the main design limitation; it reports the dropouts and funding; it avoids the words 'proven' and 'approved'; and it directs the customer to a pharmacist. It does not reproduce promotional language or imply that the pharmacy endorses the claim.