Reconciling Journey Conversion With Campaign Reporting

The two reports are answering different questions. The journey report attributes any upgrade_purchased within 21 days of journey entry to the journey. The campaign report attributes any purchase within 7 days of the campaign send to the campaign, even if the user was already in the journey. Because the campaign was sent on day 5 to journey members who had not purchased, the same purchase can be counted by both reports.

The campaign's 9.8% is also inflated relative to the journey's 6.2% for three reasons: the campaign audience is conditioned on not having purchased by day 5, the campaign window is shorter and may capture a period of higher intent, and the campaign has no holdout. The journey has a 5% holdout, so its 6.2% can be compared with a no-journey baseline; the campaign's 9.8% cannot.

The risk is double counting and overstating the campaign. A purchase may be credited to the campaign when the journey's earlier touches or natural demand drove it. Comparing the two headline rates directly would lead to the wrong budget decision.

I would run a campaign-specific holdout within the journey population. On day 5, randomly split eligible journey members who have not purchased into two groups: campaign send and no campaign send. Keep both groups in the journey so journey exposure is held constant. Measure upgrade_purchased at 7 days and at 21 days from the campaign send.

The primary comparison is campaign group vs. campaign holdout, not campaign report vs. journey report. This isolates the campaign's incremental lift on top of the journey. I would also report the journey-only holdout separately to confirm the journey itself is beating no journey.

If a prospective holdout is not possible, I would use a matched control from the journey population and compare purchase rates over the same 7-day window, but that is weaker than randomization. I would not use the existing journey 5% holdout as the campaign control because those users were not eligible for the campaign in the same way.

The decision rule: keep the campaign only if the campaign group shows a statistically significant lift over the campaign holdout and the lift is large enough to justify the send cost and any discount.

I would standardize four things:

1. Attribution window by channel: document that journey reports use a 21-day entry window and campaigns use a 7-day send window. When comparing, use a common window and state it.
2. Overlap rules: exclude active journey members from standalone campaign sends unless the campaign is deliberately layered. If layering is intentional, tag the campaign and report it as an overlay, not as a standalone channel.
3. Holdouts: require a holdout for any campaign that will be compared with a journey. Without a holdout, campaign conversion is correlation, not lift.
4. Goal-event consistency: use the same upgrade_purchased definition in both reports and confirm that refunds, cancellations, and duplicate events are handled the same way.

I would also create a custom report that shows journey and campaign exposure side by side at the user level, with a flag for users who were in both. That makes double counting visible before it reaches an executive dashboard.