Canary Deployment That Never Canaries

Traffic shifting happens on the alias, not on the function. CodeDeploy shifts weighted traffic between the two versions referenced by an alias. If API Gateway invokes the function by unqualified ARN, the request goes to $LATEST (or the pinned version) and the alias's weight is irrelevant - the alias is updated, but nothing invokes it.

Required chain:

1. API Gateway integration URI must be the alias-qualified ARN: arn:aws:lambda:<region>:<account>:function:<name>:live.
2. Each deployment publishes a new immutable version and points the alias at it.
3. CodeDeploy (via the SAM DeploymentPreference / AutoPublishAlias wiring) shifts the alias weight in steps - 10% for 10 minutes, then 100%.
4. All callers - including internal services, EventBridge targets, and scheduled invocations - must also use the alias, otherwise you get split behavior that looks like flaky data.

Verification: watch the alias's ProvisionedConcurrencyUtilization/invocation metrics per Resource dimension, and inspect the deployment in CodeDeploy for the traffic-shift steps. A successful deployment that shows a single step from start to finish is the signature of an alias that nobody is calling. For a 300 rps service, confirm the 10% slice (~30 rps) is actually visible before trusting the canary.

Alarms must watch the alias-qualified metrics. An alarm on the unqualified function aggregates every version and will never cross the threshold during a 10% canary. Create alarms whose Resource dimension is function-name:live (or that use the alias's own metrics), and use two of them: one on Errors/5XXError rate and one on latency (p99) - a canary that is merely twice as slow should also roll back before it reaches 100%.

Tune the alarm to the window. A 10-minute canary with a 5-minute alarm period and 3 datapoints out of 5 needs the failure to persist to trigger. Set the period short enough (60 seconds) and treat missing data as notBreaching only if you have separate synthetic checks - otherwise a function that fails to start can stall silently.

Add the hooks:

• A pre-traffic hook Lambda that invokes the new version directly with representative test events (including a read of the new schema) and writes success/failure to CodeDeploy. This catches contract breaks before 1 request is served.
• A post-traffic hook that validates real requests and business metrics (orders created, not just HTTP 200s).

Auto-rollback must also be enabled in the deployment group's rollback configuration for deployment failure, not just alarm failure, or a hung canary will sit at 10% indefinitely. Keep a manual-stop runbook; automation will not catch a subtle business-logic regression quickly.

The canary is not the only version running - and rollback resurrects the old one. During the 10-minute window both versions serve live traffic, and if the alarm fires the previous version returns for everyone. A destructive schema change means the old code reads a map where it expects a string, throws, and the rollback itself becomes an outage. Canary deploys protect you against bad code, not against incompatible data.

Sequence with expand/contract:

1. Expand (deploy N): write both attribute (old string) and attributeV2 (new map). Reads prefer V2 and fall back to V1. This deployment is fully backward compatible, so it is safe to canary and safe to roll back.
2. Backfill existing items with a one-off job or Streams consumer, throttled so it does not crowd out production writes.
3. Migrate (deploy N+1): switch reads to V2 only, and verify with the pre-traffic hook that no code path touches V1.
4. Contract (deploy N+2, at least one release later): stop writing V1 and remove it.

Each step must be independently deployable and revertible. Also add a version or schemaVersion attribute on the item so any consumer can tell what shape it is holding - that is what makes step 4 safe to do a week later when everyone has forgotten the details.