A Non-Significant Correlation, Six Weeks to Submission, and Three Suggested Fixes
A doctoral candidate in Education has completed data collection for a study on the relationship between classroom questioning behaviour and achievement motivation. The sample is 240 higher-secondary students drawn from six schools that volunteered after a circular. The hypothesised correlation comes out at r = 0.09 with p = 0.21 on a two-tailed test. Eighteen response sheets show straight-line marking on the 20-item scale. With the thesis submission deadline six weeks away, the supervisor suggests three moves: drop the 18 'careless' cases, re-run the test as one-tailed because 'we predicted a positive direction anyway', and remove two items that lowered the reliability. The candidate also finds a 2019 paper in a low-visibility journal that used a nearly identical 20-item scale, and wants to reproduce it without citation because 'a questionnaire is not authorship'. The candidate has already signed the university's plagiarism declaration.
1. Assess each of the supervisor's three suggestions separately. For each, state whether it is defensible and what a defensible alternative would be.
Read the worked answer
Treat the three separately, because they carry different degrees of risk.
Dropping 18 cases. Removing careless responders is defensible only if the criterion is set in advance, applied blind to the outcome, and both analyses are reported. Straight-lining is a recognised careless-response indicator, so the criterion itself is reasonable; the problem is the timing. If the decision is taken after seeing that r = 0.09, the cleaned sample is selected on the dependent variable. Defensible version: report the full-sample r = 0.09, then report the sensitivity analysis excluding the 18 cases, and let the reader see whether the conclusion changes.
One-tailed test. A directional test is legitimate when the direction is specified before data collection and the study is designed around it. Re-running as one-tailed after seeing a positive but non-significant r halves the p-value without new evidence; here p = 0.21 becomes roughly 0.105, still not significant at .05, so it does not even rescue the result - but the principle matters more than the arithmetic.
Removing two items. Post hoc deletion to raise reliability capitalises on sample noise. It is acceptable only with a stated psychometric rationale, disclosure that the scale was modified, and re-estimation of reliability on the reduced scale; ideally the decision comes from pilot data.
Overall, r = 0.09 in a sample of 240 is a genuine finding. The 95% interval around it runs roughly from −0.04 to 0.21, so the study rules out a strong association but not a weak one. That is the sentence the thesis should contain.
2. The candidate proposes to reuse the 2019 scale without citation. Analyse this on the dimensions of plagiarism, instrument validity and research integrity, and state what the candidate should do.
Read the worked answer
Plagiarism. A scale's items, response format and instructions are authored text. Reproducing them verbatim in an appendix or instrument section without attribution is plagiarism under the declaration the candidate has signed, even though the candidate is not claiming the idea. The fact that the 2019 journal has low visibility is irrelevant; the obligation does not depend on the source's prestige.
Validity. Copying a scale without its development evidence tells you nothing about whether it measures questioning behaviour as this sample experiences it. The 2019 paper's reliability coefficient belongs to that sample. The candidate needs reliability estimated on the present 240 responses, and ideally cognitive interviews with a few students to check that the items are read as intended.
Integrity. The 2019 authors may also attach conditions to reuse, and the candidate's examiner may know the paper.
What to do. Cite the original scale-development study in full; state whether the instrument is reproduced or adapted; contact the corresponding author for permission and the scoring key; describe any wording changes; report Cronbach's alpha or omega for the present sample; and place the instrument in the appendix with its citation. If substantial adaptation is made, that adaptation is itself a reportable methodological step. Correcting this now is a two-day task; correcting it after the viva is a disciplinary matter.
3. What should the thesis report, and what change at the design stage would have made the study capable of supporting a stronger claim?
Read the worked answer
Report. The pre-specified hypothesis and analysis; the full-sample result with effect size and 95% confidence interval; the handling of the 18 straight-lined cases as a declared cleaning rule together with the sensitivity result; the reliability of the scale in this sample; the volunteer nature of the six schools and what that constrains; and the non-significant result stated as a finding, with a short discussion of why a weak association is plausible - restriction of range if the schools are homogeneous, single-informant measurement, or low variability in achievement motivation.
The missing design element is identification. A cross-sectional correlation among volunteers cannot support a causal or even a strong associational claim. What would have strengthened it:
- Probability sampling from a frame of higher-secondary schools in the district, stratified by type and locality, instead of schools that volunteered.
- Better measurement of the behaviour: classroom observation rather than student report, or at least a second informant, to reduce common-method variance.
- A comparison or repeated-measures structure: the same students observed before and after a teacher-training intervention.
- Pre-registration of the directional hypothesis and the analysis plan, which would also have made a one-tailed test legitimate.
If a redesign is not possible in six weeks, the honest move is to reframe the thesis as exploratory and say so in the abstract.