The End-of-Day Batch That Will Not Fit

Do not buy capacity until the wait/run split is known. The proposal assumes a compute bottleneck; the symptoms do not yet support that.

1. Interface arrival delays (highest probability). The 14 Monday 'file not found' failures point to upstream file delivery, not CPU. Evidence: per-interface arrival timestamps for the last 90 days compared with the job's scheduled start; count of jobs that start on time but wait on a dependency.
2. Critical-path serialization. A batch can run long with idle CPUs if job B cannot start until job A finishes. Evidence: a Gantt chart of job start and end times; compute the longest dependency chain and compare it with total elapsed time.
3. Data growth without archiving. 2.9 billion ledger rows and 41% volume growth will slow full-table scans and index rebuilds. Evidence: top 20 SQL statements by elapsed time, rows examined per execution, index fragmentation, and the time-per-1,000-transactions trend over 12 months.
4. Inefficient new interfaces. Three interfaces added without scheduling review may run unindexed queries or contend in parallel. Evidence: execution plans, lock waits, deadlock graphs.
5. Infrastructure contention. Evidence: CPU, IOPS and IO latency sampled every minute across the window. If CPU sits below 50% while elapsed time grows, nodes will not help.

A two-week instrumented baseline costs far less than ₹2.4 crore a year and prevents a multi-year lock-in to the wrong fix.

The 22:00 start is a buffer, not a fix. Batch duration is 6h50m; moving the start earlier by 2h30m restores the 09:00 output only by consuming the online window and colliding with branch close, vendor file windows and the 22:00-00:30 maintenance slot. It also hides the defect, so the next volume increase re-breaks the SLA. It is acceptable only as a dated interim measure with a written expiry.

The node purchase is premature and poorly framed. ₹2.4 crore a year, presumably on a multi-year term, buys capacity the evidence may not justify. Ask the vendor for a benchmark showing the batch is CPU-bound, a per-node throughput commitment, and a termination right if the SLA is not met.

Recommendation:

• Approve a 60-day remediation: archive or purge closed accounts, add the missing indexes, re-sequence the critical path, move the ATM cash file to a 05:00 cut-off, pre-stage interface files.
• Set a decision gate: if the batch is not under 4h30m at day 60, approve nodes.
• Use the change freeze exception only for risk-reducing batch changes, with CAB approval and rollback.
• Escalate the vendor's refusal to provide execution statistics as a contract-performance issue.

The COO's option and the CTO's option are not the only two; a measured remediation with a gate is cheaper and produces evidence either way.

To the IT steering committee, report: batch elapsed time against the 3h30m design and the 09:00 commitment for each of the last 90 days; root-cause status; customer-impacting incidents (two salary-credit misses, ATM replenishment); remediation plan with owners and gate dates; vendor SLA performance; and the ₹2.4 crore proposal with the evidence gap stated.

To the regulator, report: any breach of the end-of-day balance commitment and the customer-impacting salary-credit delay, under the incident-reporting requirement, with cause, containment and remediation. Do not wait for the fix; late reporting is a separate violation.

What I would refuse to sign:

• A statement that 'the capacity upgrade resolves the batch overrun' before the wait/run analysis exists.
• A certification that there is no customer impact, because salary credits were late twice.
• An approval that treats the 22:00 start as permanent without an expiry date.

The test is whether a reader six months later can see what was known, what was decided and what remained open.