Ansible override wipes branch-specific SRX configuration
A platform team manages 120 SRX branch firewalls with Ansible. The repository contains a single file, baseline.conf, captured from a headquarters SRX, and the nightly play uses junos_config with src: baseline.conf and update: override. Branch-specific IKE gateways, DHCP pools, and interface addresses were added over time by hand. Last Tuesday's run pushed the baseline to all branches; 14 sites lost IPsec because their peer addresses were overwritten. A rollback play exists but depends on a backup file stored on the Ansible controller from before the run. The team wants to move to per-device configuration in Git, run in check mode first, and detect drift. There are 120 devices, 3 regions, and 6 engineers. The Junos versions range from 18.4R3 to 21.2R3.
1. What is the root cause, and why is update: override with a headquarters baseline dangerous here? What immediate recovery steps would you take?
Read the worked answer
update: override replaces the candidate configuration with the supplied configuration. Anything not present in baseline.conf is removed, including each branch's unique IKE peer, DHCP pool, and interface addressing. The baseline is not a source of truth for the branches; it is one device's snapshot. The nightly play therefore converts every branch into a headquarters clone and deletes intentional local state.
Immediate recovery: stop the scheduled play and disable the job. On the affected devices, inspect show system commit and compare the current configuration with the previous rollback using show configuration | compare rollback 1. If the pre-change configuration is in the rollback history, restore with rollback 1 and commit during a controlled window, then verify IPsec and management reachability. Where rollback history is insufficient, use the controller backup, but check its timestamp and device identity first. Do not rerun the baseline play while restoring. After recovery, compare all 120 devices against the last known good per-device configuration, not against headquarters.
2. Propose a target configuration-management structure and the Ansible mechanics you would use to prevent recurrence.
Read the worked answer
Use per-device intent in Git: group_vars/all.yml for truly global settings, group_vars/region_*.yml for regional differences, and host_vars/<device>.yml for branch-specific IKE peers, DHCP pools, and addresses. Render configuration from Jinja2 templates and pass only the rendered result for that device to junos_config. Use update: merge for owned stanzas, or a narrowly scoped replace pattern if the device supports it safely, rather than a global override. Include a commit comment with the Git SHA and merge request ID.
Run ansible-playbook --check --diff first. Before committing on a risky device, use commit confirmed with a short rollback timer and require an explicit confirm after verification. For drift, render the intended configuration and compare it with show configuration | compare or retrieve the running configuration and diff it against the intended state; report differences without automatically overwriting unmanaged local state. Keep per-device backups on the controller but treat them as recovery artifacts, not the source of truth.
3. How would you handle the Junos version range from 18.4R3 to 21.2R3 and validate the rollout?
Read the worked answer
Put version-dependent syntax behind variables and conditionals in templates rather than branching the whole play. Test each template against representative versions in the lab; four vSRX devices can be built at the oldest and newest supported versions, plus one per region if the feature set differs. Use commit check on the target before committing and fail the play if the check fails. Roll out by region and by device role, starting with a canary device in each version group. Capture the rendered candidate and the commit result for each device. If a version cannot support a feature, fail closed with a clear message instead of silently rendering a partial configuration.