Flash-sale web tier with a licence-constrained Oracle database
An online retailer runs a Java monolith on 40 ECS instances (8 vCPU / 16 GB) behind an ELB. HTTP sessions are held in local Tomcat memory, and the 6 TB Oracle database sits on a single 32-core server. The annual promotion produces roughly ten times normal traffic for six hours, with a hard requirement that no shopper sees "session expired" after login. The DBA reports database CPU already reaches 70% at today's peak, and the Oracle licence is charged per physical core, so finance refuses to add cores. The event is ten weeks away and the application cannot be re-architected before then.
1. Design the compute scaling approach for the promotion. What must be changed before the scaling group is created, and what will the scaling group fail to protect?
Read the worked answer
Make the web tier disposable first, then scale it.
- Move sessions from Tomcat memory to DCS for Redis. Without this, every scale-in event logs shoppers out - precisely the failure that is forbidden - and Auto Scaling cannot safely remove an instance. This is a prerequisite, not an optimisation.
- Build a golden image containing the application and the Redis session client, and create a scaling group of min 40 / max 200. Booting a cold image adds several minutes plus JVM warm-up per instance, so image hygiene and a load test matter more than the max count.
- Add a scheduled policy that raises capacity about 45 minutes before the event (boot + warm-up + ELB slow start), with an alarm policy on ELB concurrency and average CPU as a safety net, plus a cooldown long enough to stop add/remove thrash.
- Explicitly exclude the database from the scaling path. At 70% CPU today, ten times the web traffic against uncached queries will saturate it. Catalogue and stock reads must be served from cache, and the DBA must load-test the peak SQL mix - otherwise Auto Scaling will succeed and the database will take the site down.
2. The DBA proposes moving the Oracle database to a Bare Metal Server. Is that the right call, and what do you have to verify before committing?
Read the worked answer
Yes, conditionally - the reasoning is about what a vCPU is, not about performance.
Oracle's core-based licence counts physical cores, normally with a core-factor adjustment for the processor type. On an ECS, the vCPU you buy is a hyperthread presented by a shared host; you cannot map a vCPU to a guaranteed physical core, so a licence audit is likely to count the entire physical host rather than your flavour. A BMS gives you a dedicated physical server whose populated cores are known and can be licensed exactly - which is the entire economic argument here, since finance has capped core count rather than spend.
Before committing, verify: (1) the specific Oracle licensing rules for the chosen processor and for virtualised versus physical deployment, in writing; (2) that the BMS flavour's memory and local storage meet the 6 TB working set, or pair it with EVS; (3) the availability model - a BMS is a single server and does not enjoy the host-level resilience an ECS does, so plan a standby plus database-native replication and accept a manual or scripted failover; and (4) that the application vendor still supports the platform.
If the licence audit risk can be resolved on ECS by licensing all host cores, the BMS business case collapses and ECS is the simpler answer.
3. The load test succeeds but a mid-event incident occurs. Which failure modes would you pre-emptively design out?
Read the worked answer
The scaled-out fleet creates failure modes that a static 40-instance fleet never had:
- In-flight work on scale-in. An instance removed at 21:00 may be mid-checkout. Enable connection draining on the ELB and make checkout steps idempotent; otherwise scale-in becomes a source of duplicate or lost orders.
- Connection-pool storms. 200 instances each opening a default pool can exhaust the database's session limit. Cap pool size per instance and total connections before the event - this is the most common way a successful scale-out kills a database.
- Health-check mismatch. If the ELB health check only tests TCP, instances that cannot reach Redis stay in service and serve errors. Health-check the dependency path.
- Warm capacity lag. An alarm policy reacting to CPU adds capacity after the queue has already formed. Pre-scale on schedule and treat alarms as a backstop.
- Cache cold start. Scaling out does not warm caches; after a scale-in/out churn the cache hit ratio drops and the database load rises exactly when it is most fragile.
Pre-empt these with a staged load test that includes a deliberate scale-in, not just a scale-out.