Troubleshooting, monitoring, and operations
Domain 5 tests operational diagnosis: choosing the right tool, reading the right log, and separating a BIG-IP fault from an application or network fault.
Where to look first
- Statistics: tmsh show ltm virtual, show ltm pool and show ltm pool members for connection counts, resets and members marked down.
- Logs: /var/log/ltm for daemon and monitor messages, /var/log/asm for security events, /var/log/restjavad files for API problems, plus remote syslog and SNMP traps for monitoring platforms.
- Packet level: tcpdump on the data interface or VLAN (not the management interface), ssldump for handshake failures, and curl --resolve to test through a VIP from a client host.
Health, sync and recovery
- tmsh show cm sync-status and show cm device confirm device-group state; configuration sync and traffic-group failover are separate mechanisms.
- qkview collects a diagnostic snapshot for iHealth analysis; treat the file as sensitive because it contains configuration.
- UCS archives save and restore configuration; AVR supplies historical analytics when a problem is intermittent.
Worked example
Users report intermittent timeouts to an HTTPS VIP while some requests succeed. Reasoning: check virtual server and pool member statistics. If members are flapping, the monitor is the suspect; if members stay up, filter tcpdump on the server VLAN by the pool member address and port. A SYN with no SYN-ACK points to routing, a firewall or a dead service; an immediate RST points to a closed port or application rejection; many connections from a single SNAT address that fail point to SNAT port exhaustion. Each branch leads to a different fix, which is why the capture comes before any configuration change.
Traps that cost marks
- Capturing on the management interface while traffic flows on a data VLAN.
- Sending a qkview externally without reviewing its contents.
- Assuming a successful config sync means traffic groups failed over cleanly.
- Reading only /var/log/ltm when the fault is in the WAF or API layer.