Master Data Match Run Produces 22% False Positives Before Pricing Initiative
A global manufacturer has 2.4 million customer records across SAP ERP, Salesforce CRM, and a legacy order-entry system. A match-merging project found 310,000 suspected duplicate parties. Sales operations uses CRM as the source of truth for contacts; Finance uses ERP for billing; Customer Service uses the legacy system for order history. The MDM program proposed a central hub with survivorship rules: CRM wins for contact name and email, ERP wins for legal name and tax ID, legacy wins for order history. The first match run produced 68,000 clusters, but 22% were false positives: separate ship-to locations of the same parent company, and franchisees sharing a corporate email domain. The steering committee wants a single customer ID in 5 months for a global pricing initiative. Data stewards disagree on whether ship-to locations should be separate parties. The vendor match tool supports probabilistic and deterministic matching but requires tuning. The program has no documented party hierarchy and no stewardship decision on franchisees.
1. Why did the first match run produce 22% false positives, and what should be done before scaling?
Read the worked answer
The false positives are a modeling problem before they are a matching-tool problem. The program has not defined what a party is. Ship-to locations are locations under a legal entity, not separate customers; franchisees may be separate legal parties linked to a parent. Matching on shared corporate email domains conflates parents, franchises, and locations.
Before scaling:
- Define entity types: legal party, location, contact, and franchise relationship. Document whether ship-to is a party, a location, or both with different identifiers.
- Build a party hierarchy: parent, subsidiary, franchisee, ship-to. Match within and across levels using different rules.
- Tune matching by entity type: deterministic rules for tax ID and legal name; probabilistic only for contact-level matching; remove email domain as a strong key when it is shared.
- Measure precision and recall on a labeled sample before accepting clusters. A 22% false-positive rate is unacceptable for pricing, because it can merge unrelated buying entities and distort contracts.
- Add manual review for high-value and high-ambiguity clusters and a stewardship decision log.
Scaling the current run would propagate bad merges into pricing and billing, which are hard to unwind. Fix the entity model and matching thresholds first.
2. Recommend survivorship and stewardship rules for the golden customer record, including ship-to locations and franchisees.
Read the worked answer
Separate the golden identifier from the best record per attribute. A single customer ID should identify the legal party; ship-to locations should have their own location IDs linked to that party unless they are separate legal entities. Franchisees should be modeled as parties with a parent-franchise relationship, not merged into the parent.
Survivorship rules:
- Legal name, tax ID, billing address: ERP wins, because Finance owns statutory reporting.
- Contact name, email, sales contact: CRM wins, because Sales owns the relationship.
- Order history and legacy account references: legacy system supplies history but not master attributes.
- Ship-to address: location source system wins, with postal validation; do not let CRM overwrite a validated ship-to address.
- Franchisee legal name and tax ID: franchisee's own ERP record wins; the parent link is a relationship attribute, not a merge.
Stewardship rules should require a decision record for entity-type exceptions and a review queue for clusters above a risk threshold. Survivorship must be attribute-level and auditable; a single 'winning system' for the whole record would corrupt statutory or sales data. Communicate that the golden record is not a physical single row but a governed, linked view.
3. What five-month plan would deliver a single customer ID for the pricing initiative?
Read the worked answer
Use a phased, confidence-tiered plan rather than a big-bang merge:
- Weeks 1–4: Entity model and stewardship decisions. Define party, location, franchisee, and ship-to. Get written sign-off from Sales, Finance, and Customer Service. Without this, matching will repeat the false-positive problem.
- Weeks 3–8: High-confidence matching. Run deterministic matches on tax ID plus legal name and country, and tune probabilistic matching for contact records. Measure precision on a labeled sample; require at least 98% precision before clusters are used for pricing.
- Weeks 6–12: Manual review for high-value and ambiguous clusters. Prioritize top accounts by revenue for pricing. Issue provisional IDs for reviewed clusters; keep unreviewed clusters out of pricing.
- Weeks 10–16: Crosswalk and ID service. Build a crosswalk from source keys to the new party ID, expose an API, and backfill downstream systems. Maintain lineage to source records.
- Weeks 14–20: Pilot with pricing and controls. Use the ID for a limited pricing segment, monitor disputes, and define a correction process.
Five months is tight but feasible for a usable ID if scope is limited to pricing-relevant parties. Promising a perfect 2.4-million-record merge in that window would create technical debt and business distrust.