Node Operations, Monitoring, and Troubleshooting
This domain tests day-two operations: getting a node to peer, observing it, and diagnosing why it stopped producing or accepting blocks.
Core rules
- Logs go to stdout. Raise verbosity with
--loggingor supply log4j configuration via--logging-config-file. - Metrics are off by default. Enable with
--metrics-enabled, then scrape--metrics-hostand--metrics-port(default 9545) using Prometheus. - Health endpoints
/health,/readinessand/livenessare served on the metrics port and are the correct targets for container probes. --ethstats=<name>:<secret>@hostpublishes node status to an ethstats dashboard.- P2P uses
--p2p-port(default 30303).--p2p-hostis the address advertised to peers, so it must be the externally reachable address, not loopback. - Bootnodes are supplied with
--bootnodes;--discovery-enabled=falserestricts connections to configured peers. - Useful diagnostics:
admin_peers,net_peerCount,eth_syncing, anddebug_traceTransactionwhen the DEBUG API is enabled.
Worked example
A cloud node reports zero peers. You check that the enode URL uses the public IP, that the security group allows TCP and UDP 30303, that the bootnode enode string matches the bootnode's actual key, and that both nodes share the same genesis and chain ID. The log line about a failed handshake usually points at a genesis or fork mismatch rather than networking.
Traps
- Leaving
--p2p-hostat the default inside a NAT or cloud environment. - Assuming ADMIN or DEBUG are enabled by default; they are not.
- Ignoring clock skew, which causes rejected blocks in BFT networks.
- Debugging peering before confirming identical genesis and chain ID.