Ansible VLAN Rollout Drifts on Six Switches
An enterprise manages 60 Dell OS10 switches from Ansible: 40 S5248F-ON and 20 S5232F-ON. The playbook uses the Dell OS10 collection over the REST API to enforce VLANs 100-199 and VLT port-channels. A recent run reports 'ok' on 54 switches and 'changed' on 6. The six have VLAN 150 with description 'old-campus' instead of 'campus-users', and one VLT pair has VLAN 150 on leaf-a only. The switch management network is stable; REST API credentials are shared in a vault. The team has no configuration backups older than the last run, and the next change window is in four hours. A separate sFlow collector is dropping samples, but that is not blocking this change.
1. How would you determine whether the six 'changed' switches are a playbook bug or manual drift, without guessing?
Read the worked answer
Use the playbook in check mode and capture a diff against the running configuration on all 60 switches. If the six differ from the intended VLAN 150 definition, compare their show running-configuration and REST API responses to the other 54; repeated differences in description and VLT membership point to manual drift, while a consistent failure on a platform subset points to a module or template bug. Check Ansible logs for which task reported changed and whether it was the VLAN description or the VLT port-channel. Since the shared credential works elsewhere, authentication is unlikely. Before changing anything, export running configs from all six and store them; then fix the source of truth. Do not simply re-run with force, because forcing VLAN 150 onto only one VLT peer can create a partial service state.
2. What is the safe remediation sequence for the six switches, especially the VLT pair missing VLAN 150 on one peer?
Read the worked answer
Treat the VLT pair as one unit. First, confirm which VLANs are allowed on the VLT port-channel and whether VLAN 150 is required on both peers; VLT requires consistent VLAN membership across the peer-link. On the peer missing VLAN 150, add the VLAN with the approved description and ensure it is allowed on the VLT port-channel; if the platform uses VLT configuration synchronization, verify the sync result rather than assuming. Do one pair at a time, starting with a low-risk pair, and use a maintenance window or non-disruptive commit if supported. After each pair, verify show vlt, show vlan, and show running-configuration interface vlan 150 on both peers, plus a test host in VLAN 150. Then re-run the playbook in check mode to prove idempotency before a normal run.
3. What automation and monitoring controls would you add so the next drift is caught before a change window?
Read the worked answer
Make the repository the source of truth and add a scheduled compliance job that runs the playbook in check mode every 15-30 minutes and alerts on any diff. Store per-run configuration backups and retain more than one generation so a bad run can be rolled back. Add a pre-change gate that checks VLT peer consistency and VLAN allow-lists before applying a VLAN change, and require a canary switch or pair before the full fleet. Use Ansible inventories grouped by platform and role so S5248F and S5232F templates cannot silently diverge. For monitoring, alert on syslog config-change events, SNMP traps for VLAN creation/deletion, and REST API audit logs. Treat sFlow drops as a separate capacity issue: check collector ingestion and sampling rate, but do not let it obscure configuration drift.