Git Workflow and CI Gate for a Multi-Device Config Change
A network team of 6 engineers manages 300 devices with Ansible and Jinja2 templates in a Git repo. Today they edit templates on a shared jump host and run playbooks manually. They want CI: on pull request, render templates and run a syntax check; on merge to main, deploy. Two incidents drive this: (1) a typo in a Jinja2 loop deployed a misconfigured description to 40 devices, and (2) an engineer's local branch was 3 weeks behind main and reverted a security ACL on merge. The team debates trunk-based vs GitFlow and whether to use commit confirmed in the playbook.
1. Design the CI stages and the merge gate that would have caught both incidents.
Read the worked answer
Stages:
- Lint/render - run
ansible-lint, and render every Jinja2 template for every host in inventory (ansible-playbook --check --diffor a render-only task). A loop typo surfaces as a render error or an obviously wrongdescriptionin the diff. - Diff review - post the rendered per-device diff as a PR comment (or artifact). Reviewers approve rendered output, not just template source. Incident (1) is caught here.
- Unit test - assert rendered output against expected fixtures for representative hosts.
- Merge gate - require PR review, passing CI, and branch up to date with
main(or rebase required). Incident (2) is caught because a stale branch cannot merge without rebasing, and the resulting diff is re-reviewed. - Deploy on merge - gated, with
commit confirmedand a verification step.
The lesson: review the effect (rendered config), not the cause (template text), and never allow a stale branch to merge silently.
2. Should the deploy use `commit confirmed`? Give the conditions under which it helps and when it hurts.
Read the worked answer
commit confirmed helps when a change could sever the management path (ACLs, routing, interface addressing): the device auto-rolls back unless a confirming commit arrives within the window. It converts a remote-bricking mistake into a 10-minute outage.
It hurts when:
- The change is not management-path-affecting and the confirm step adds failure modes (a lost confirmation rolls back a good change).
- The playbook cannot reliably reach the device to confirm (flaky path), causing spurious rollbacks.
- Changes are non-idempotent or ordered across devices (e.g., a coordinated routing change) where partial rollback leaves inconsistency.
Recommended: use commit confirmed with a short window (e.g., 5 min) for management-path changes, and have the playbook verify reachability (ping/SSH/NETCONF) before confirming. For low-risk changes, a plain commit with post-change validation is fine. Make it a per-change decision, not a blanket policy.
3. Trunk-based or GitFlow for this team? Justify with the incidents in mind.
Read the worked answer
Trunk-based with short-lived feature branches fits better. GitFlow's long-lived develop and release branches encourage exactly the drift that caused incident (2): a branch diverges for weeks and merges stale. Trunk-based with branches measured in hours/days, mandatory rebase-or-up-to-date before merge, and small PRs keeps divergence small and reviewable.
GitFlow's ceremony (release branches, hotfix branches) adds merge complexity without buying much for a 6-person team managing config. The real controls are: small changes, rendered-diff review, required CI, and up-to-date branch enforcement - all of which work in trunk-based and are harder to enforce in GitFlow.
Caveat: if the team needs staged rollouts across environments, use environment-specific branches or, better, the same code with environment-specific inventory/vars - not long-lived code branches.