Double-fired purchases and a nightly POS feed at Northwind Outfitters

Two different problems are being conflated: duplicate events and missing events. Fixing one badly breaks the other.

Do not try to dedupe inside mParticle. Events from different inputs are processed independently, and connection transformations run per event with no memory of earlier events. Any rule that says 'drop this if the transaction_id was already seen' will fail as soon as a second instance or a retry handles the event.

What to change:

1. Separate the event names. Server sends purchase (authoritative, carries transaction_id). Client sends checkout_completed_client with the same transaction_id. Downstream can union on the key without ambiguity, and the warehouse dedupe becomes trivial.
2. Move the data plan from review to enforcement. Reject purchase without transaction_id at the input, so a malformed order is caught at ingest rather than discovered in a revenue reconciliation three weeks later.
3. Use data filtering on the client purchase connection only if you accept losing the fallback. If you keep both paths, dedupe downstream on the order ID - most ad platforms dedupe on an order ID natively if you pass it.
4. Resilience is server queueing plus nightly reconciliation against the OMS, not the client event. Treat the client event as a UX signal.

Guarantee: exactly one authoritative purchase per order, with duplicates visible and filterable rather than silently additive.

Start with a decision, because mParticle merges profiles by any identity value you present: which key is canonical per person?

• If loyalty_id is genuinely per-person, send it as customer_id in the S3 feed. Send the SHA-256 email as an email identity only if the online side uses the identical hashing convention and normalisation (lower-case, trim). A different convention produces two profiles for the same human and your suppression audience silently under-delivers.
• If loyalty IDs are shared across a household - common in retail - do not use them as identity. As an identity they merge every family member into one profile and personas bleed across people. Attach them as a user attribute on a profile keyed by hashed email.
• Timing matters. The nightly file creates or updates profiles, but events already tied to a different MPID are not rewritten. Backfill the identity key on the online side first, then start the file feed, otherwise the first weeks of suppression are wrong.
• Validate the destination. Several ad platforms match on raw lower-cased email or a device ID; a hashed-email-keyed audience will look healthy inside mParticle and arrive empty at the partner.

Proof point: run a match-rate test on a 5,000-user sample before switching the audience on.

They are not lost, but they are not decisionable in real time either.

• The device keeps an anonymous MPID. When identify() eventually fires with a device ID and a customer ID, mParticle merges the anonymous profile into the known profile, so the earlier browsing history does come along.
• What is lost is timing. A real-time audience that fires on 'viewed three products' evaluates against an anonymous MPID at that moment. If the destination needs a CRM ID or email, the forward lands unmatched; if it only needs a push token or device ID, the same audience works fine.
• Consequence for segmentation: any audience whose criteria depend on post-login attributes systematically under-counts anonymous-heavy segments. That is a data-quality artefact, not a behaviour signal, and it will be read as one if nobody documents it.
• Practical split: maintain anonymous-eligible audiences (push, in-app, device-matched ads) separately from known-only audiences (CRM, email, offline). Add an explicit identity criterion to the known-only set rather than letting the destination fail quietly.
• Instrument the anonymous-to-known merge rate. A sudden drop usually means an SDK release stopped calling identify(), not that users changed behaviour.