Scaling a Program That Worked at Small Volume
A nurture program was built for 5,000 leads and performs well. It now needs to handle 400,000 leads. The program uses a single smart campaign that triggers on 'Lead is Created' and sends a 6-email sequence via a nested series of Wait steps. The instance has 2.3M person records and runs ~120 active smart campaigns. Users report that campaign membership processing lags by hours, the activity log is slow, and API integrations are hitting rate limits during business hours. The team proposes simply raising the batch size and adding more wait steps.
1. Why won't 'raise batch size and add wait steps' scale, and what would you change?
Read the worked answer
The proposal treats a throughput problem as a knob-turning problem. Symptoms (hours-long membership lag, slow activity log, rate-limit collisions) point to contention across smart campaigns, not to one campaign's batch size.
Problems with the plan:
- Wait-step chains don't scale. Long sequences of Wait steps in a single campaign hold people in a long-running flow; at 400K that is a huge in-flight population and hard to observe or throttle.
- Trigger-on-create is the wrong entry point at this volume. It fires for every new person and floods processing. Entry should be a scheduled batch that qualifies people on a defined cadence, so volume is shaped.
- Raising batch size increases contention with the other 120 campaigns and with API traffic, worsening lag and rate limits.
What to change:
- Break the monolith. Split into smaller, purpose-built campaigns (e.g. by segment or stage) with bounded membership, rather than one giant flow.
- Use scheduled batch entry with a controlled qualifying cadence instead of create-triggers at scale.
- Stagger execution. Schedule heavy batches off-peak and separate them from API-heavy windows to relieve rate-limit collisions.
- Reduce in-flight population by exiting people promptly on conversion or disqualification.
- Instrument it. Measure membership processing time and API usage before and after; scale is managed with observability.
The judgement: at scale you shape and segment the workload and schedule it, rather than turning up a batch size.
2. API rate limits are colliding with business-hours traffic. How do you architect around that?
Read the worked answer
Rate limits are a shared budget across all integrations; the fix is governance and scheduling, not just retries.
- Inventory consumers. List every integration and its call volume and peak window. Often a few chatty jobs dominate.
- Shift batch work off-peak. Move bulk syncs and enrichment to overnight windows; keep real-time, user-facing calls for business hours.
- Batch and cache. Consolidate many small calls into fewer batched requests where the API supports it; cache reference data instead of re-fetching.
- Backoff and queue. Implement exponential backoff and a queue so transient 429s retry gracefully instead of failing loudly or hammering the API.
- Prioritize. Reserve capacity for latency-sensitive integrations; deprioritize bulk jobs.
- Monitor. Track usage against limits and alert before saturation.
The judgement: rate limits are a capacity-planning and scheduling problem. Retries alone just move the collision.