Right-Sizing a Bursty Video Transcoding Pipeline

The workload is batch, CPU-bound, idempotent, and bursty with a predictable weekly peak - a textbook fit for Serverless / Elastic Container Instance (ECI) or Auto Scaling ECS driven by a queue depth metric, not a fixed fleet.

Reasoning:

• A fixed 40-instance fleet is sized for the average, so it is simultaneously over-provisioned 90% of the week and under-provisioned at peak. That is the worst of both cost and reliability.
• Because jobs are idempotent and retryable, the team can safely decouple ingestion from processing with a message queue (e.g., MNS/RocketMQ). Producers drop job messages; workers pull and process. This makes scaling a function of queue depth, which is a direct, reliable signal.
• During the Friday peak, scale-out adds workers within minutes; overnight, scale-in returns to a small baseline. Pay only for what runs.
• Keep a small always-on baseline to absorb weekday traffic and avoid cold-start latency on the steady 60/hour load.

The key judgement: decouple first, then autoscale on the decoupled signal. Autoscaling a tightly-coupled web tier would be fragile; autoscaling a queue consumer is not.

Both are viable; the decision hinges on job duration, runtime control, and operational preference:

• Job duration: transcoding takes 4-12 minutes. Function-as-a-service platforms typically impose execution time limits that may be shorter than the longest jobs. If a single job can exceed the function timeout, you must either chunk the job or use container/ECS workers. This is often the deciding constraint.
• Runtime and dependencies: transcoding needs specific codecs, binaries, and possibly GPU or high-memory profiles. Container images on ECI/ECS give full control; a function runtime may fight you on native binaries and image size.
• Cold start vs. steady throughput: at 900 jobs in two hours, you have sustained load, so cold-start penalties matter less than raw parallelism and per-job cost. Serverless shines for spiky, short, low-volume invocations; a long-running batch favors containers where you amortize startup over a 4-12 minute job.
• Ops capacity: the team is small. Managed autoscaling groups with container images keep ops low while preserving control.

Recommended: containerized workers on ECI/ECS with queue-depth autoscaling, and reserve serverless functions for lightweight orchestration (e.g., validating uploads, enqueuing jobs).

Validate with a load test that reproduces the peak shape and measures end-to-end, not just worker throughput:

1. Replay the peak: inject ~900 jobs over two hours (450/hour sustained) plus a burst to test headroom, using realistic file sizes and durations.
2. Measure the real SLO: time from upload to all renditions complete (p95/p99), not just CPU utilization. A pipeline can look healthy while the queue backs up.
3. Watch the scaling signal lag: confirm that queue-depth scaling reacts fast enough. If scale-out takes longer than the queue can absorb, pre-warm a higher baseline before the known Friday window (scheduled scaling) - the peak is predictable, so use that.
4. Test failure paths: kill workers mid-job and confirm idempotent retries produce correct output without duplication.
5. Cost check: compare projected spend at peak and at baseline against the old fleet to confirm the savings are real, including any minimum-billing granularity.

The judgement: a predictable peak should be handled with scheduled pre-scaling plus reactive autoscaling, and success is measured by end-to-end latency under a realistic replay, not by infrastructure metrics alone.