Batch Queue Starvation After Adding a Second CPU

The generic queue is almost certainly not configured with /GENERIC=(BATCH_A1, BATCH_A2), or it is configured with an allocation class / job limit that concentrates work. The decisive clue is that all four jobs landed on one execution queue while the other sat empty.

Key reasoning:

• A generic queue distributes jobs to its target execution queues based on availability (execution queue job limit and, if set, the generic queue's own job limit and allocation class).
• If the generic queue has /JOB_LIMIT=4 and only one target has free slots at submission time, all pending jobs go to whichever target the scheduler picks first - typically the first available target in the generic's target list.
• Adding a CPU to ALPHA1 does not change queue selection; it only changes throughput per job. The imbalance predates the CPU change but became visible when total runtime dropped enough that the window's critical path shifted.

So the fix is to verify SHOW QUEUE/FULL BATCH_NIGHT and confirm the target list and job limit, then either raise the generic job limit, set per-target job limits that reflect capacity, or attach jobs with explicit /NODE or use a resource-based allocation class (/ALLOCATION_CLASS) so ALPHA1's extra CPU attracts more work proportionally.

Raising BATCH_A1 to 8 is risky on a 1 GB node:

• Each batch process starts with a 512-page default working set. Eight concurrent compilations can drive the working set total past physical memory, causing excessive paging and page-file I/O, which slows both batch and interactive users.
• The interactive sluggishness is a symptom of memory pressure, not CPU starvation. Adding concurrency on the same node worsens it.
• The cluster has a second node with free capacity; the correct move is to use it, not to over-subscribe ALPHA1.

Safer alternatives, in order:

1. Rebalance the generic queue: set /GENERIC=(BATCH_A1, BATCH_A2) with a generic /JOB_LIMIT that reflects total capacity, and set execution queue job limits to 3 (A1) and 2 (A2).
2. Constrain batch memory: set /WSDEFAULT=256, /WSEXTENT=1024, and /PGFLQUOTA on the execution queues so batch cannot crowd out interactive working sets.
3. Use scheduling priority: give interactive users a higher base priority via their UIC group or via /BASE_PRIORITY on the batch queues, so batch yields under contention.
4. Only after memory is bounded should you consider modestly raising A1's limit.

Measure with MONITOR PAGE, MONITOR PROCESSES/TOPCPU, and SHOW MEMORY during the window before and after.

Collect evidence from three angles:

CPU

• MONITOR SYSTEM / MONITOR CPU during the window: if CPU is near 100% but not accompanied by paging, it is genuine CPU saturation.
• MONITOR PROCESSES/TOPCPU to see whether batch or interactive processes dominate.

Memory / paging

• MONITOR PAGE - look at page fault rate, free list size, and modified list. A shrinking free list plus rising page faults is the signature of memory pressure.
• SHOW MEMORY - check Free, Modified, and Paging file free space. Low free pages during the window confirms pressure.
• SHOW PROCESS/ACCOUNTING on a running batch job to see its peak working set and page fault count.

Interactive symptom correlation

• MONITOR MODES and MONITOR IO - if interactive response degrades while CPU has headroom, the bottleneck is I/O from paging, not CPU.

Interpretation: if CPU is high and free memory is low with elevated page faults, the root cause is over-committed memory; if CPU is high with healthy free memory and low page faults, the batch window is simply CPU-bound and the fix is queue rebalancing. The two fixes are different, so the measurement must distinguish them before changing queue limits.