CCOA (IJCAHPO) cheat sheet

Availability math: serial components multiply (0.99 x 0.99 = 0.9801); redundant parallel paths give 1 - (1-A)^n, so two independent 99% paths yield 99.99%.

Single-AZ deployment caps effective availability at roughly the AZ SLA; multi-AZ active-active removes single-facility failure as a correlated fault domain.

RPO = maximum tolerable data loss (time); RTO = maximum tolerable downtime. Backup frequency drives RPO; failover automation drives RTO.

Pilot light keeps data replicated plus a minimal core running; warm standby keeps a scaled-down full stack; multi-site active-active runs full capacity in 2+ regions.

Loose coupling primitives: queue (point-to-point, one consumer), topic/pub-sub (fan-out), and idempotent consumers keyed on a deduplication ID.

Well-architected trade-offs are explicit: strong consistency raises latency and lowers partition tolerance under CAP; choose per data class, not per system.

Stateless tiers scale horizontally behind a load balancer; state belongs in managed data services so instances are disposable.

Shared-responsibility split: provider secures physical/infrastructure layers, customer secures identity, data, configuration, and network policy.

Idempotency test: running the same automation twice must leave identical end state and emit no duplicate side effects.

Declarative orchestration converges on a desired state; imperative scripts encode steps and break on partial failure unless they checkpoint.

Retry with exponential backoff plus jitter avoids thundering herds; cap attempts and route exhausted retries to a dead-letter queue.

Saga pattern replaces distributed transactions: each step has a compensating action; failure triggers reverse-order compensation.

Event-driven choreography (no central controller) vs orchestration (central controller owns flow) - pick choreography for loose coupling, orchestration for visibility.

Cron/scheduled triggers must be timezone-explicit (UTC) and guarded by a lock (e.g., lease or DB advisory lock) to prevent overlapping runs.

Runbooks should be executable: parameterized, version-controlled, logged with run ID, and safe to re-run after interruption.

Blast-radius control: canary or one-batch-at-a-time rollout with automatic halt on error-rate threshold breach.

State file is the source of truth mapping declared resources to real IDs; remote backend with locking (e.g., object storage + lock table) prevents concurrent-apply corruption.

plan/diff before apply: review destroy vs update-in-place vs replace; replacement of stateful resources (databases, disks) is the highest-risk change.

Modules/packages should be versioned and pinned; floating 'latest' references make builds non-reproducible.

Immutable infrastructure: bake images, replace rather than patch in place; configuration drift becomes impossible because instances are disposable.

Desired-state config management (e.g., declarative resource blocks) is idempotent; procedural shell provisioning is not.

Secrets never live in code or state in plaintext - reference a secret manager and mark values sensitive; state itself is often unencrypted by default.

Drift detection runs plan/diff on a schedule; any non-empty diff on unchanged code signals out-of-band manual change.

Environments (dev/stage/prod) share modules but differ only by variable files and separate state backends.

Pod is the scheduling unit; containers in one pod share network namespace (localhost) and can share volumes. Pods are ephemeral and not restarted in place.

Requests drive scheduling; limits drive cgroup throttling/OOM kill. Setting request > limit is invalid; request far below limit causes noisy-neighbor risk.

Deployment = stateless rolling updates with ReplicaSet history and rollback; StatefulSet = stable identity, ordered rollout, per-pod persistent volumes.

Service types: ClusterIP (internal), NodePort (host port), LoadBalancer (cloud LB). Ingress routes HTTP by host/path to Services.

Liveness probe failure restarts the container; readiness failure removes it from Service endpoints; startup probe protects slow-booting apps from liveness kills.

QoS classes: Guaranteed (requests=limits on all), Burstable (some set), BestEffort (none) - eviction order is BestEffort first.

ConfigMap/Secret mount as files or env vars; env-var secrets are visible in process listings, file mounts are safer.

HorizontalPodAutoscaler scales replicas on metrics; Cluster Autoscaler adds nodes when pods are Pending due to insufficient resources.

Pipeline stages: build -> test -> package/artifact -> deploy; artifacts are immutable and content-addressed (digest, not tag) for reproducibility.

Trunk-based development with short-lived branches and feature flags reduces merge risk vs long-lived branches.

Blue/green swaps traffic between two full environments for instant rollback; canary shifts a small percentage first and gates on metrics.

Deployment != release: deploy dark, then flip a flag; rollback is a flag change, not a redeploy.

Every pipeline run should be reproducible from a commit SHA; pin tool and base-image versions, avoid 'latest'.

Gate merges on required checks (unit, integration, security scan); a red pipeline must block promotion to the next environment.

Database migrations must be backward-compatible (expand/contract) so old and new app versions can run during rollout.

Deployment frequency, lead time, change-failure rate, MTTR are the four DORA delivery metrics.

Three pillars: metrics (aggregated numbers, cheap, cardinality-limited), logs (discrete events, high volume), traces (request path across services with span IDs).

SLI = measured indicator (e.g., success rate); SLO = target (99.9% over 30 days); error budget = 1 - SLO, i.e., 43.2 min/month at 99.9%.

Four golden signals: latency, traffic, errors, saturation. USE (utilization, saturation, errors) for resources; RED (rate, errors, duration) for services.

Alert on symptoms (SLO burn rate) rather than causes; page only on user-impacting conditions, ticket the rest.

Metric cardinality explosion from high-cardinality labels (user ID, request ID) inflates cost and query latency; keep those in logs/traces.

Structured JSON logs with correlation/trace ID enable cross-service joins; sampling (head or tail) controls trace cost.

Distributed tracing propagates context via headers (e.g., W3C traceparent); a span is one unit of work with start/end and parent link.

Log retention tiers: hot searchable days, warm, then cold archive - set per compliance need, not per convenience.

Least privilege: grant the minimum action on the minimum resource; prefer roles/assume-role over long-lived access keys.

Defense in depth: identity, network (segmentation/security groups), encryption at rest and in transit, and application-level authz.

Encryption: symmetric for bulk data (AES-256), asymmetric for key exchange/signatures; KMS-managed keys enable rotation and audit.

Secrets lifecycle: generate, store in manager, rotate on schedule and on compromise; never log or commit them.

Compliance frameworks (SOC 2, ISO 27001, HIPAA, PCI DSS) require evidence of controls - logging, access reviews, change records.

Network controls: default-deny security groups, private subnets for data stores, egress restriction, and WAF for public HTTP.

Authentication vs authorization: who you are vs what you may do; MFA and short-lived tokens reduce credential-theft impact.

Audit trail must be append-only and tamper-evident; separate the logging account/role from the workloads it monitors.

Right-sizing: match instance/container request to observed p95 utilization; over-provisioned requests waste reserved capacity.

Commitment discounts (reserved/savings plans) suit steady baseline load; spot/preemptible suits fault-tolerant batch - never stateful single-instance workloads.

Autoscaling on the right metric (queue depth or CPU) with cooldown prevents flapping; scale-in slower than scale-out.

Storage tiering: hot for active data, infrequent/cold for backups, archive for compliance; lifecycle rules automate transitions.

Egress and cross-region data transfer are common hidden costs; co-locate chatty services in one region/AZ where durability allows.

Idle resources (unattached disks, unused load balancers, stopped-but-billed instances) are the fastest cost win; tag for attribution.

Unit economics: cost per request or per tenant reveals scaling inefficiency better than total monthly spend.

Performance: cache read-heavy paths, use CDN for static assets, and set connection pooling to avoid per-request handshake cost.