Overnight Conversion Drop After a Tracking Update
At 08:00 Monday, your e-commerce dashboard shows conversion rate fell from 3.2% to 2.6% overnight. Revenue is down 18% week over week. The drop appears in all channels but is largest in paid search (-34%). At 07:30 the analytics team deployed a tracking-script update. The site had no code release. Customer-service tickets about checkout errors rose from 12 to 19 per day, within normal weekly variation. Last year the same week had a 2.9% conversion rate. Paid search spend is unchanged, but impressions are down 11%. Your CMO asks whether to pause paid search spend today.
1. List the first three checks you would run, in order, and what result would make you treat this as an instrumentation problem versus a real demand problem.
Read the worked answer
- Verify the tracking deployment. Compare the dashboard's session count with server-side logs and order records for the same hours. If sessions rose but orders did not, the conversion denominator is inflated-instrumentation problem.
- Reconcile revenue independently. Pull payment-processor totals and compare with dashboard revenue. If processor revenue is flat while dashboard revenue is down 18%, the dashboard is wrong.
- Check the paid-search impression drop. Confirm whether impressions fell in the ad platform or only in your analytics. A platform-side drop is a real demand signal; an analytics-only drop is tracking.
Rule: If independent order/payment data are stable, treat the conversion drop as instrumentation until proven otherwise. If independent revenue is genuinely down and checkout tickets are rising, treat it as a real funnel problem. The 07:30 tracking update makes instrumentation the leading hypothesis, but the 11% impression decline is a separate signal that needs its own check.
2. Should the CMO pause paid search today? Give a conditional recommendation.
Read the worked answer
Do not pause all paid search today on the basis of the dashboard alone.
Pausing spend is a real intervention with real cost: you lose impressions, Quality Score momentum, and possibly market share, and you may be reacting to a measurement artifact. The tracking update at 07:30 is a strong reason to wait for reconciliation.
Conditional recommendation:
- If server-side orders and payment data confirm revenue is down and the paid-search landing page or checkout is failing, pause or reduce paid search immediately and redirect budget to working channels.
- If only the analytics dashboard is down while orders are stable, do not pause; fix tracking and annotate the dashboard.
- If impressions are down 11% in the ad platform but conversion is unconfirmed, hold spend at current level for 24 hours, check search-term and landing-page health, and set a trigger: pause if confirmed revenue per click falls below the pre-update baseline by more than 15%.
This separates the measurement question from the spending question.
3. The tracking update is confirmed to have double-counted sessions but not orders. How does that change your interpretation of conversion rate and revenue?
Read the worked answer
Conversion rate: If sessions are double-counted and orders are correct, the denominator is roughly doubled (or inflated by some factor). The reported 2.6% is therefore artificially low. Recompute conversion using deduplicated sessions or, better, use orders per unique visitor. The true rate may be close to the prior 3.2%.
Revenue: Revenue should be unaffected if orders and payments are counted correctly. The reported 18% week-over-week revenue drop is then suspect and should be reconciled to the payment processor. Do not treat it as real until confirmed.
What remains real: The paid-search impression decline (-11%) is from the ad platform, not necessarily the tracking script. That is a separate signal of demand or competition and should be investigated on its own. The checkout-error ticket rise from 12 to 19 is within normal variation and is weak evidence. Decision: fix tracking, restate the metrics, and re-evaluate paid search with clean data before making budget changes.