Network Troubleshooting
Troubleshooting asks you to move from symptom to cause with a repeatable method, in a system where you may not own every hop. Expect scenarios with intermittent failures, one-directional traffic, breakage immediately after a change, and performance that degrades only under load.
A method that scores
Fix the scope first: one user or all, one zone or all, one protocol or every protocol, one direction or both. Then separate control plane from data plane. Control plane means the route exists, is associated with the correct route table, is advertised and accepted over BGP, and DNS resolves to the address you assume. Only then inspect the data plane: security groups, NACLs, firewall policy, NAT, and the return path.
High-yield failure patterns
- Asymmetric routing through a stateful firewall, which drops the return direction.
- MTU black holes: handshakes succeed, bulk transfer stalls, PMTUD is blocked by an ICMP deny rule.
- Missing return route, or a NAT gateway available in only one subnet.
- Overlapping CIDRs, which appear as silent drops rather than errors.
- Split-horizon DNS returning a public address to an internal client, causing a hairpin.
- BGP session established but prefixes filtered by a route map.
Worked example
Users report that SFTP to a partner over an IPsec tunnel authenticates and lists directories, but file transfers hang after about thirty seconds. Small packets work and large ones do not - the signature of an MTU problem, not authentication or routing. Test with a large don't-fragment ping through the tunnel; if it fails while a small one succeeds, PMTUD is being blocked. Fix by permitting ICMP type 3 code 4 toward the endpoints and clamping TCP MSS to 1360 on the tunnel interface, then re-test a full transfer. Reasoning: the symptom shape isolated the layer, a targeted probe confirmed it, and the fix addressed the mechanism rather than the symptom.
Traps
Concluding the network is fine because ping succeeds - ICMP may be permitted while the application port is filtered. Changing several variables at once. Testing from a single source that shares a subnet with the destination. Ignoring timing, where failures at a fixed hour follow a scheduled job or credential rotation.