The Dropout-Risk Dashboard and the Journalist's Request
A district education office in West Bengal plans a pilot in 40 government secondary schools covering about 18,000 students. A vendor's model gives each student a 0-100 'dropout risk' score. Inputs are attendance, term marks, household income band, parents' occupation, social category, distance from school, and free-text remarks teachers write in the register, such as 'family problem' or 'works at night'. Head teachers see the scores; the 500 highest-risk students are automatically enrolled in a remedial scheme; the list is also uploaded to the vendor's cloud server outside India for 'model improvement'. No consent form is used, on the grounds that the data is already collected by schools for routine state reporting. The model was validated only on last year's district data, where it scored 0.81 AUC overall. A local journalist files a request for the list of the 500 students. Parents have not been told the score exists.
1. Identify the data-protection and ethical problems in this pilot, and say how the journalist's request should be handled.
Read the worked answer
Several problems compound each other.
Purpose limitation and consent. Data collected for routine school reporting is being reused for a predictive risk score - a materially different purpose. Under India's DPDP Act 2023 framework, processing needs a lawful basis, notice and consent, and for children (under 18) verifiable parental consent is required. 'The school already has the data' is not a lawful basis for a new use.
Data minimisation. Social category, household income band, parents' occupation and free-text teacher remarks are far more than is needed to identify academic risk, and the free-text remarks are unverified hearsay ('family problem') that a student can never see or contest.
Cross-border transfer and security. Uploading identified student records to a foreign cloud for 'model improvement' is a separate purpose with no notice, no transfer assessment and no retention limit. Under the Act, cross-border transfer is restricted to countries not on the government's negative list, and it must still satisfy the other obligations.
Transparency and contestability. Parents do not know the score exists, so there is no way to correct a wrong input - the model is being used on people who cannot see it.
The journalist's request should be declined as a request for third-party personal data. A lawful response offers aggregate or anonymised statistics (counts by school, block, class) and the methodology, while publishing no identifiable list.
2. Analyse the bias and feedback-loop risks in the model, and explain why an AUC of 0.81 overall is not sufficient evidence of fairness.
Read the worked answer
Proxy variables. Distance from school, income band, parents' occupation and social category are all correlated with caste, class and geography. The model can achieve good overall discrimination by learning to flag poor, remote and marginalised students - which is not the same as identifying students who need help, and risks converting disadvantage into a label.
Label bias. 'Dropout' is read from school records. Students who migrate for work or transfer schools are often recorded as dropouts; students who drop out silently may not be recorded at all. The model learns the record-keeping artefact, not the social process.
Teacher remarks. Free-text notes carry individual teacher judgement and can encode personal or caste bias. Feeding them into a score launders that bias into a number that looks objective.
Self-fulfilling prophecy and feedback loop. Automatically enrolling the top 500 changes their outcomes, and the next training round then learns from a world the model itself altered. Meanwhile a student wrongly not flagged receives nothing, which reinforces the pattern.
Why AUC 0.81 is not enough. AUC is a single pooled number; it can be 0.81 while recall for, say, first-generation learners or a particular block is far lower, or while the score is systematically shifted upward for one group. Fairness requires disaggregated evaluation: per-group recall, precision, calibration and error rates, plus a check of which features are driving the score. Also note the validation used last year's district data - the same population and the same biases the model will reproduce.
3. Redesign the pilot so it could be defended to parents, and state what must change before any student list is produced.
Read the worked answer
Before any score is produced:
- Notice and consent. A short notice in Bengali explaining what is collected, why, who sees it, and for how long - with verifiable parental consent for minors and an opt-out that carries no penalty for the student.
- Minimise the inputs. Drop social category, income band and parents' occupation unless a specific, documented justification survives review. Replace free-text remarks with a structured, consented checklist (for example 'repeated absence', 'after-school work') that a guardian can see and correct.
- Keep data in the district. Host on district-controlled infrastructure; if a vendor needs access for model improvement, use de-identified extracts under a contract with a retention limit, or on-premises training.
Changing how the score is used:
- Support, not classification. A high score should trigger an outreach conversation - a call to the guardian, a check on transport or textbook access - never an automatic scheme enrolment or a permanent record.
- Explain and allow appeal. Show the guardian the main factors behind the score and give a named person who can review it.
- Human decision with documented override, so teachers can disagree with the model and that disagreement is recorded.
- Governance: publish a model card (inputs, validation population, per-group performance, known limits), run an independent audit before go-live, and review disaggregated recall every term with a published summary.
Only aggregate, non-identifiable statistics should ever leave the district, including to the journalist.