Sizing a Second Worker Node for a Burst-Prone Gallery
A company runs Alteryx Server with a single Worker node (16 vCPU, 64 GB RAM) and a separate Controller. Overnight batches of 40 workflows run between 01:00 and 05:00; daytime users run ad hoc workflows that finish in under 2 minutes. Recently, overnight jobs began queuing past 07:00, and daytime users complain that their short jobs wait behind long batch jobs. Average batch workflow runtime is 6 minutes with high variance (some 45+ minutes). Peak RAM utilization on the Worker reaches 58 GB. The company wants to add a second Worker node but is unsure whether to match the existing specs or scale differently, and how to prevent long batch jobs from starving interactive users.
1. Should the second Worker node be identical to the first, and what is the strongest argument for or against matching specs?
Read the worked answer
Matching specs is the safer default because Workers must run the same Alteryx engine version and the Controller distributes work without regard to per-node capacity differences; heterogeneous nodes make queue behavior unpredictable and complicate capacity planning. But the real constraint here is RAM: peak 58 GB on a 64 GB node leaves little headroom, and a 45-minute batch job that spikes memory could OOM. The second node should at minimum match CPU and RAM so that any workflow can run on either node. The stronger argument is not about matching per se but about whether adding one node solves the problem: if 40 jobs average 6 minutes, total demand is roughly 240 engine-minutes; a single Worker at, say, 4 concurrent engines can process that in about 60 minutes of wall time, so the 01:00-05:00 overrun suggests concurrency is capped low or a few long jobs dominate. Adding a node doubles throughput but does not address starvation. So: match specs, but pair the addition with scheduling controls rather than treating the node as the whole fix.
2. How should the team prevent long batch jobs from blocking short interactive workflows?
Read the worked answer
Use the Server's queue and scheduling features rather than relying on node count. Options: (1) Assign the batch workflows to a dedicated Worker (or Worker pool) via the workflow's Worker assignment setting, so interactive jobs land on the general-purpose node. (2) Stagger the batch schedule so the 45-minute workflows start first and short ones fill gaps. (3) Use workflow priorities if available in the deployed version to let interactive jobs jump the queue. (4) Cap concurrent engines per Worker so a single node isn't oversubscribed, which would otherwise inflate every job's runtime. The cleanest answer combines dedicated Worker assignment for batch with priority or scheduling for interactive jobs. Pure horizontal scaling without assignment just moves the starvation problem to a bigger queue.