Continuous Delivery Architecture cheat sheet

Continuous delivery keeps software in a deployable state at all times; continuous deployment additionally releases every passing change to production automatically.

Core metrics (DORA): lead time for changes, deployment frequency, change failure rate, and failed deployment recovery time.

Lead time for changes is measured from commit to running in production, not from ticket creation; queue and wait time usually dominate.

Value stream mapping records process time versus wait time per step; flow efficiency = process time / total lead time, typically under 25% before improvement.

Deployment is the technical act of installing a version in an environment; release is the business decision to expose it to users. Decoupling them enables dark launches and canary exposure.

A change is deployable only if build, automated tests, and environment-independent configuration validation all pass on the same artifact.

Trunk-based development with short-lived branches (under one day) reduces merge cost; long-lived branches increase integration risk and lead time.

The three ways of DevOps: flow (left to right), feedback (right to left), and continual learning and experimentation.

Build once, deploy many: produce one immutable artifact per commit and promote that same binary through environments; rebuilding per environment invalidates test evidence.

Artifact identity should be content-addressed (SHA-256) or tied to a commit SHA plus build number; tags alone are mutable and unreliable.

Repositories: use a private proxy or virtual repository in front of public registries to cache, scan, and survive upstream outages.

Dependency resolution must be deterministic: commit lockfiles, pin versions, and forbid dynamic ranges such as latest or 1.x in release builds.

Reproducible builds require pinned toolchain versions, pinned base images by digest, and normalized timestamps; verify with a rebuild-and-compare check.

SBOM (for example SPDX or CycloneDX) is generated at build time and attached to the artifact for vulnerability and license traceability.

Incremental and cached builds are safe only when cache keys include all inputs: source, dependencies, toolchain, and build flags.

Semantic versioning: MAJOR for incompatible changes, MINOR for backward-compatible features, PATCH for backward-compatible fixes; pre-release suffixes sort before the release.

A deployment pipeline is a set of automated stages from commit to release; each stage is a quality gate, and failure stops promotion.

Stage ordering rule: cheapest and fastest checks first (lint, unit), then integration, then acceptance and performance; this minimizes feedback time.

Commit stage target is under 10 minutes; if exceeded, parallelize test shards or move slow suites to later stages.

Pipeline as code stored with the application version gives reviewable, auditable, and reproducible pipeline definitions.

Parallelism and fan-out/fan-in: shard tests across agents, then aggregate results; use fail-fast only where later stages cannot add diagnostic value.

Environment promotion should be a single trigger per stage with no manual rebuild; manual approval gates belong between stages, not inside build steps.

Idempotent pipeline steps allow safe retries; steps that mutate external state need compensating cleanup or explicit non-retryable marking.

Artifact and test-result retention: keep the deployed artifact and its test evidence for at least the audit window required by governance.

Test pyramid: many fast unit tests, fewer service or contract tests, very few end-to-end UI tests; the inverse (ice-cream cone) produces slow, flaky pipelines.

Test doubles: stubs return canned data, mocks verify interactions, fakes are working lightweight implementations; prefer fakes for service boundaries.

Consumer-driven contract testing (for example Pact) verifies provider compatibility without deploying both services together.

Flaky test policy: quarantine on first detection, track flake rate, and block merge on new flakes; a test failing intermittently erodes gate trust.

Test data must be created per run and isolated (unique namespaces or tenants) to allow parallel execution and avoid cross-run interference.

Coverage thresholds are a floor, not a goal; measure changed-line coverage in pull requests to avoid rewarding low-value tests.

Non-functional gates: performance budgets, load tests on representative data, and security scans (SAST, dependency, container, DAST) run as pipeline stages.

Acceptance tests run against the deployed artifact in a production-like environment; they validate deployment, not just code.

Infrastructure as code treats environments as versioned, reviewable definitions; manual console changes create drift and are the leading cause of environment divergence.

Declarative tools (Terraform, CloudFormation, Bicep) converge to a desired state; imperative scripts encode steps and are harder to make idempotent.

State files map declared resources to real ones; store them remotely with locking and encryption, never in the repository.

Plan-then-apply: review the diff before applying; any unexpected destroy or replace on stateful resources is a stop signal.

Immutable infrastructure replaces servers instead of patching them; this makes rollback a redeploy of the previous image.

Ephemeral environments per branch or pull request enable realistic integration testing; they must be auto-destroyed on merge or timeout to control cost.

Environment parity: keep the same OS, runtime, and configuration mechanism across stages; differences should be data (sizes, endpoints), not structure.

Drift detection runs on a schedule and alerts when live infrastructure diverges from declared state.

Blue-green: two identical environments, switch traffic at the router; rollback is switching back, but both stacks must stay schema-compatible during the switch.

Canary: route a small percentage of traffic to the new version, compare error rate and latency against baseline, then ramp or abort automatically.

Rolling deployment updates instances in batches; capacity must absorb the loss of one batch, and mixed versions must coexist during the rollout.

Feature flags decouple deploy from release; flags need an owner and expiry date, and stale flags are a defect.

Shadow or dark launch mirrors production traffic to the new version without serving its responses, validating behavior under real load.

Rollback vs roll-forward: rollback restores the previous artifact; roll-forward ships a fix. Data migrations often make roll-forward the only safe option.

Expand-and-contract for schema changes: add new columns and dual-write, backfill, switch reads, then remove old fields in a later release.

Deployment strategies should be chosen per risk: stateless services suit canary, stateful stores need careful batch sizing and version compatibility checks.

Configuration belongs in the environment, not the artifact; the same binary must run in every stage with different injected configuration.

Twelve-factor rule: store config in environment variables or a config service, never as constants compiled into the build.

Secrets must never be committed; use a vault or managed secret store with short-lived credentials and automatic rotation.

Secret scanning in pre-commit and CI catches accidental leaks; on detection, rotate first, then purge history.

Least privilege for pipeline identities: each stage gets only the permissions it needs, scoped to its target environment.

Database migrations must be backward compatible for at least one release so old and new application versions can run concurrently.

Migration ordering: run schema changes as a separate, idempotent, versioned step before application rollout; never rely on application startup to mutate schema.

Backfill and destructive steps are separated from deploy and gated by verification queries and backups.

Three pillars: metrics (aggregatable time series), logs (discrete events), traces (request path across services); correlate with a shared trace ID.

Golden signals: latency, traffic, errors, and saturation; alert on symptoms users feel, not on every internal cause.

Service level indicators measure actual behavior; SLOs set a target; error budget = 1 - SLO and governs release velocity.

Deployment markers on dashboards let responders correlate a change with a metric shift; automated rollback triggers on error-budget burn.

Governance as code: policy checks (for example Open Policy Agent) in the pipeline enforce tagging, approved base images, and required approvals.

Change audit trails must link commit, artifact digest, approver, and deployment timestamp; this is what auditors actually request.

Post-incident review is blameless and produces tracked actions; repeat incidents indicate a missing automated control, not a people problem.

Feedback loops should be fast and visible: build status on the pull request, deployment notifications in team channels, and dashboards for lead time and failure rate.