Documentation, Change Management, and Operational Best Practices
The documentation set
A usable package includes logical and physical topology diagrams, an IP address management record (subnets, VLANs, VRFs, allocations), interface and patch-panel port maps, rack elevations, a device inventory with serial numbers, EoL/EOS dates and contract IDs, circuit records with provider references, and runbooks for routine tasks. Diagrams decay fastest - attach them to a change record so updates are mandatory, not optional.
Change management
An RFC states the change, business justification, affected services, risk, test plan, implementation steps, verification, and backout. CAB review focuses on risk and rollback, not on whether the engineer is competent. A method of procedure lists exact commands, expected output at each step, a maintenance window, an out-of-band access path, and explicit backout triggers with a time limit. Peer review catches copy-paste errors, which are the most common cause of self-inflicted outages.
Operational hygiene
Back up configurations before and after changes; confirm the backup is retrievable, not merely scheduled. Run consistent NTP so logs from different devices correlate, centralise syslog with retention that satisfies policy, and label interfaces and cables physically and logically. Track configuration drift against a golden template. Review accounts and privileges, disable unused services, and keep spare hardware matched to critical parts.
Worked example
Replacing an uplink on a core switch stack. Pre-checks: capture the running config and routing table, confirm console access, verify a second path exists. Steps: shut the uplink in the window, verify convergence and that traffic shifted, swap the optics, re-enable, confirm neighbours re-form and error counters stay flat. Backout trigger: neighbours not up within ten minutes - restore the original port and revert the config. Post-verification: diff configs against the saved baseline.
Traps that cost marks
Running a change with no working rollback, relying on a backup nobody has restored, ignoring NTP so log timelines disagree, and leaving diagrams unupdated after the change closes.