First Unified Profile Rollout for a Regional Retailer
Contoso Home Goods has 1.8 million customer records spread across an e-commerce platform, a 240-store POS system, and a five-year-old loyalty database. The loyalty database stores emails in mixed case and has roughly 300,000 rows where the email field contains a placeholder such as 'unknown' or a store code. The POS feed delivers phone numbers as 10-digit strings without country codes, while the e-commerce feed includes country codes. The data team wants to see value within three weeks and has one engineer allocated part time. Leadership has asked whether they should start with all three sources or phase the rollout, and whether the loyalty source should be included at all given its quality problems.
1. Should the team onboard all three sources at once to maximize match rates, or phase the rollout? Justify the sequencing and describe what the first milestone should be.
Read the worked answer
Phase the rollout. Onboarding three sources simultaneously multiplies the number of data-quality defects that must be diagnosed at once, and with one part-time engineer the feedback loop becomes too slow to hit a three-week milestone.
Start with the two cleanest, highest-signal sources: e-commerce and POS. These give a defensible first milestone - a unified customer profile built on known identity keys (email for e-commerce, normalized phone for POS) - that demonstrates value and establishes the ingestion and unification pipeline patterns the loyalty source will reuse.
Before adding loyalty, fix it at the source or stage it: replace placeholder emails with nulls so they do not create false matches, normalize casing, and standardize phone numbers to E.164 with a country code. Placeholder values are worse than missing values because they can unify unrelated customers into one profile.
Include loyalty eventually - it likely carries tenure and historical spend that enrich segments - but add it as a second wave after the identity normalization rules are proven on the first two sources. Document the normalization rules in the unification configuration so the same logic applies consistently.
2. The team worries that 300,000 placeholder emails will silently merge distinct customers. What configuration and validation steps address this before unification runs?
Read the worked answer
Treat placeholders as missing data, not as identity evidence. Concretely:
- Cleanse at ingestion. In the data source transformation, map known placeholder tokens ('unknown', store codes, 'n/a') to null in the email field. A null email cannot participate in a match, which is the desired behavior.
- Normalize identity fields. Lowercase and trim emails; convert phones to E.164 with a default country code applied only where the number lacks one. Mixed formats are a leading cause of both false negatives and false positives.
- Choose match rules deliberately. Use exact match on normalized email and phone as high-confidence rules, then add fuzzy rules (name plus address) at a threshold that requires corroboration. Never let a single fuzzy attribute create a match on its own.
- Validate before publishing. Run unification and inspect the match counts and a sample of merged profiles, especially any cluster larger than a plausible household size. A cluster of hundreds of customers sharing one email is a placeholder leak, not a real household.
- Iterate. Adjust rules or thresholds, re-run, and only then expose the unified profiles downstream.
3. Leadership asks for a single number: 'how many unique customers do we have?' What should the team present, and what caveats belong with it?
Read the worked answer
Present the unified profile count with an explicit confidence framing rather than a bare number.
The count is an estimate whose accuracy depends on match-rule coverage and data quality. Present it alongside: the number of source records ingested, the percentage that resolved to a unified profile versus remained unmerged, and the count of profiles built from a single source only. That last figure matters because single-source profiles are the least verified identities.
State the known limitations: loyalty records with placeholder emails were excluded from email-based matching, so some of those customers may appear as duplicates; phone normalization applied a default country code that may be wrong for a small international segment; and fuzzy matches above the threshold carry a small false-merge rate.
Recommend tracking the number over successive refreshes. A count that moves sharply between runs signals rule instability, which is more actionable than debating the absolute figure. Frame the number as 'approximately X unified customers, with Y still unresolved' so leadership can make decisions without treating an estimate as an audited fact.