240 Switches, no Failed Hosts, and 50 Unchanged Devices
An Ansible playbook pushes syslog and NTP settings to 240 IOS-XE switches with serial: 10. The task uses ios_command with a show run fragment, has ignore_errors: true, and the final recap printed ok=240 failed=0, yet post-change verification found about 50 switches still running the old syslog server. Separately, 30 switches became unreachable from the automation host after a management VLAN change; those same 30 still answer pings from hosts on a neighbouring segment. Their management SVI is configured with a /25 mask, while the automation host sits in the upper half of the same /24 range and uses a /24 mask.
1. Explain why the recap said ok=240 failed=0 while roughly 50 devices were not changed, and what the playbook should do instead.
Read the worked answer
Three separate reasons stack up:
ios_commandexecutes show/exec commands; it does not push configuration. Runningshow runsucceeds on all 240 devices whether or not the config is right, so 'ok' measures command execution, not desired state. Changing config requiresios_config(or a resource module) withparents/lines.ignore_errors: truesuppresses the failures you need. Any device that did error is counted as ok and the play proceeds. The recap becomes a fiction.- No verification task. Nothing compared the running configuration to the intended one, so drift had no way to surface.
Instead: use a declarative config module, register the result per host, drop ignore_errors in favour of explicit block/rescue handling that records the failure and keeps going deliberately, and add an assert task after the change that re-reads the device and fails the host if the settings are absent. Then the recap means something. Preserve the old settings from the pre-change show run so the rescue path has a rollback target - without that, retrying is the only recovery.
2. Diagnose the 30 unreachable switches. How does the mask mismatch explain reachable-from-neighbours but unreachable-from-automation, and how would you confirm it?
Read the worked answer
The masks describe different networks. With a /25, the switch's management address belongs to a block covering only the lower half of the /24. The automation host, in the upper half with a /24, believes the switch is on-link and ARPs for it directly. The switch believes the automation host is off-link and sends its replies to its default gateway instead of answering on-link. If the gateway has no route back to the upper half, or routes it asymmetrically, connectivity fails in one direction - and it fails specifically for hosts in the other half of the range, which matches the symptom. Hosts on the neighbouring segment succeed because their traffic is routed and both ends agree on the topology.
Confirm rather than assume:
show ip interface <mgmt>andshow run interfaceon the switch, to read the actual mask.show ip route- is there a valid default gateway, and is it inside the /25?- From the switch:
ping <automation-host> source <mgmt-interface>. From the automation host:tracerouteand an ARP table check. - Inspect the automation host's own mask and route; a wrong mask on the controller produces the same asymmetry inverted.
Fix the mask on whichever side changed, then re-run the playbook - do not add static routes to paper over a mask error.
3. The team wants to detect configuration drift weekly and re-apply automatically. What makes a task idempotent here, and what would you refuse to automate?
Read the worked answer
A task is idempotent when a second run against an already-correct device changes nothing and reports changed=false. For IOS-XE that means declarative modules: ios_config with parents and lines, or resource modules like ios_system/ios_ntp_global, where the module compares against the running config and only issues what is missing. ios_command can never be idempotent - it is a remote shell, so it reports success regardless of device state.
Run drift detection with check_mode plus --diff first: it reports what would change without touching the device, which is both a safety net and your evidence that the play is idempotent. Only escalate to applying when the diff is non-empty and the device is in scope.
What I would refuse to automate end-to-end:
- Anything that could sever management reachability - VLAN, SVI, ACL, or routing changes on the management path. Those get a human, a console fallback, and a rollback plan, because an automated mistake locks you out of the device that would fix it.
- Applying to devices that failed verification last cycle without a human clearing them; otherwise an unreachable-but-recovering device gets rewritten.
- Silent auto-remediation of drift that is not understood. If a switch is deliberately off-standard, the automation will fight the engineer forever. Report first, remediate by policy.