Ensuring Solution and Operations Excellence
Operations questions test observability design and how you detect, diagnose, and recover from failure.
Observability stack
Cloud Monitoring collects metrics, uptime checks, dashboards, and alerting policies. Cloud Logging ingests logs into buckets with configurable retention (the _Default bucket defaults to 30 days), routed through sinks to BigQuery, Cloud Storage, or Pub/Sub; exclusion filters control cost. Error Reporting aggregates exceptions, Cloud Trace shows latency across services, and Cloud Profiler finds CPU/memory hot spots. Log-based metrics turn a log pattern into an alertable signal.
SLO-driven alerting
Define SLIs (availability, latency, error rate), set SLOs, and alert on symptoms and error-budget burn rather than on every underlying cause metric. A paging alert should mean a user is affected; everything else belongs on a dashboard or a ticket.
Audit logging
Admin Activity and System Event logs are always enabled and free. Data Access logs are opt-in per service and are the ones you enable when investigating who read specific data.
Incident and change management
On-call rotation, severity definitions, runbooks, clear escalation, and blameless postmortems with tracked action items. Progressive delivery (canary, approval gates in Cloud Deploy) limits the blast radius of bad releases; rollback must be a rehearsed operation.
Capacity and resilience
Watch quotas before they become outages, load test to validate autoscaling limits, and run DR exercises that genuinely fail traffic over.
Traps
Believing Data Access logs are on by default; alerting on CPU instead of user-facing symptoms; assuming log retention is unlimited; ignoring quota exhaustion as a failure mode.
Rule of thumb: page on SLO burn (symptoms), investigate with logs and traces (causes), and keep every alert tied to an actionable runbook.