Troubleshooting and Operational Procedures
Troubleshooting and Operational Procedures
This domain tests method under pressure: structured diagnosis, safe change, and clean handover.
Core rules
- Use a repeatable method: identify the problem, establish a theory (most likely cause first), test one variable, plan the fix with rollback, implement, verify, document.
- Work the OSI model bottom-up for connectivity: link (
ip link), IP (ip addr,ping), transport (ss), application (curl -v, logs). Top-down is fine when the app is clearly broken. - Change one thing at a time and record the result. Correlation is not causation.
- Escalation criteria and contact paths should be written before an incident. Severity drives communication cadence.
- Post-incident: blameless review, timeline, contributing factors, action items with owners and dates.
Worked example
Users report a slow application after a patch. Reasoning: confirm scope (one user, one site, all users), check the timeline against the change record, and compare metrics before and after. If CPU is normal and latency rose, inspect downstream dependencies and database waits. Roll back the patch on one canary node and compare. If latency returns to baseline, you have a causal signal; if not, keep the patch and continue. Document the canary result either way.
Traps
- Changing multiple settings at once, then being unable to attribute the fix.
- Restarting before collecting logs or a memory dump.
- Skipping the rollback plan because "it's a small change".
- Closing a ticket without a root cause or a prevention action.