42,000 Duplicates Eleven Weeks Before the SIS Go-Live
A state university is replacing three legacy student systems with a single SaaS student information system. In the second mock conversion, 1.8 million person records produced 42,000 suspected duplicates, 3,100 students ended up with two active IDs, and 780 financial-aid awards attached to the wrong person. The vendor's matching rule uses last name, date of birth and postal code. No data governance council exists; the registrar, HR and finance each maintain their own identifiers and reject any change to them. Go-live is 11 weeks away and the legacy mainframe hosting contract expires 60 days after that, with no renewal option at the current price. The integration lead proposes assigning three temporary staff to hand-key the exceptions during cutover weekend. Financial aid is subject to a federal audit in nine months.
1. Evaluate the hand-keying proposal and set out the matching strategy you would use instead.
Read the worked answer
Do not hand-key 42,000 records. Three temporaries over a weekend will make inconsistent choices with no audit trail and create a new generation of duplicates - converting a data-quality problem into a permanent one at the worst possible moment.
Treat it as entity resolution with business rules:
- Deterministic matching first on strong identifiers (national ID, prior institution ID, verified email) for automatic merge.
- Probabilistic matching on name, DOB and postcode for the remainder, with a score threshold. Auto-merge only above a high threshold; route the middle band to human review; reject below it.
- Survivorship rules written before any merge: which system wins for legal name, address, aid status. The registrar, HR and finance each own their own attributes - that resolves the 'nobody gives up their ID' deadlock without a political fight.
- A permanent crosswalk mapping every legacy ID to the new one.
- The 780 aid awards reviewed by financial aid staff, not temporaries, because they are the audit exposure.
Aim the manual queue at a few thousand high-risk records, and measure match precision on a sample before cutover.
2. What governance can realistically be stood up in 11 weeks, and why does sequencing matter?
Read the worked answer
A thin but real layer, not a quarterly council.
- Named data stewards now for person, student and financial data, each with authority to sign off survivorship rules for their attributes. One meeting, one decision, escalation to the CIO if they refuse.
- A weekly data-quality forum for the 11 weeks with authority to change matching rules and thresholds, chaired by the programme, attended by the three stewards, the vendor and the integration lead.
- A two-page written policy: identifiers, thresholds, merge approval, crosswalk retention. Written now, refined later.
- Weekly metrics: duplicate rate, review queue depth, sample match precision, aid mismatches.
Sequencing is the point. Ownership and survivorship decisions must land before any bulk merge, or the merge bakes in one department's version of the truth and unwinding it later costs more than the project. The federal audit in nine months raises the bar further: every merge needs a reconstructable audit trail, which is only possible if the rules existed before the merge ran.
3. Would you recommend going live in 11 weeks? What would you put to the sponsor?
Read the worked answer
Not on the current evidence: 3,100 dual IDs and 780 aid errors on student aid records subject to federal audit is not a 'fix it in hypercare' item.
Ask for decision gates rather than a date. Go-live proceeds only if: duplicate rate in the next mock is at or below an agreed threshold; aid mismatches reconcile to zero on a full run; the crosswalk is complete and tested; the review queue is empty or has an owner and a clearance rate the team can sustain in a week.
The mainframe expiry is a commercial problem, not a licence to skip data quality. Options for the sponsor: a short-term or per-month extension of the legacy contract even at a premium, running legacy read-only in parallel, or a phased go-live by cohort (new students first, continuing students later). Price each against the cost of a failed aid cycle and an audit finding.
If the organisation still chooses to proceed, get the residual risk accepted in writing by the business owner, with the aid reconciliation plan attached and a named owner.