Verification, Monitoring, and Troubleshooting
What this domain covers
Final domain: proving a deployment is healthy, wiring up monitoring so failures are seen before users report them, and diagnosing the common failure classes with the right command instead of a reboot.
Post-deployment verification checklist
- Hardware and state:
system show hardware,alerts show current,disk show statefor failed or missing drives. - File system:
filesys statusandfilesys show space- confirm enabled, running, and free space above the reserve. - Network:
net show config,net show hardware,net show stats. - Time and name resolution:
ntp statusand a forward and reverse lookup test. - Services:
ddboost status,nfs show exportsorcifs show,license show. - Replication:
replication statusshows the state and lag for each context. - End-to-end proof: a real backup from each client type, a restore of one file, a test alert email and a successful Autosupport transmission.
Monitoring surfaces
- System Manager dashboard for live capacity, throughput and alerts.
- Autosupport daily and event-driven, the main escalation artifact; confirm SMTP or transport gateway works.
- SNMP v2c/v3 traps to the monitoring platform, syslog to the central collector, and DDMC for fleet-wide reporting when several systems exist.
- CloudIQ for cloud-side health analytics, which needs outbound 443.
Troubleshooting by symptom
- File system read-only - usually capacity exhaustion, a failed disk without a usable spare, or an unclean shutdown needing attention. Check
alerts show current,filesys show space,disk show state, then free space by deleting snapshots or running cleaning. - Backups failing but pings working - suspect the DD Boost port 2051, a missing DD Boost role, or a full MTree quota.
- Replication lag growing - check
replication show stats, throttle settings and link errors before touching configuration. - CIFS or retention lock misbehaving - check time skew and DNS first; most such tickets are NTP issues.
Worked example
A system starts rejecting writes at 02:00. alerts show current reports the file system is read-only; filesys show space shows 99% used after a large snapshot tree was retained. Reasoning: the free-space reserve was consumed, so the file system was made read-only for protection. The fix is to expire and delete snapshots, run cleaning off-peak, then confirm the state returns to read-write - not to restart the appliance, which simply returns to the same full state.
Traps that cost marks
- Assuming alert email works without sending a test message.
- Restarting the system to clear a replication or capacity problem - it returns to the same condition and lengthens the outage.
- Confusing benign log noise with active alerts (
log showversusalerts show current). - Ignoring a persistent cleaning-needed state until backups fail.