RESTCONF Rollout Hits 401s, 403s, and Stale Writes
You operate 38 Catalyst 9200 access switches across three buildings. A Python script using the requests library calls each switch's RESTCONF interface to create VLAN 210 and set its name. On the first run, 21 switches succeed, 12 return HTTP 401, and 5 return HTTP 403. The script authenticates with a shared service account, svc_auto, that is defined locally with privilege 15, and the switches have ip http secure-server enabled but no ip http authentication local. A junior engineer reruns the entire script three times to fix the failures. Later you find that two switches now have VLAN 210 named DATA-OLD instead of DATA-NEW, and one access port description was overwritten. The service account password is rotated monthly, and the script has no retry or logging. Your manager wants the rollout finished today without touching the other 21 switches.
1. Explain why the same script produced both 401 and 403 responses, and what you check on the switch and AAA side to separate the two.
Read the worked answer
401 and 403 are different layers of the AAA decision.
- 401 Unauthorized means the HTTP request failed authentication. With
ip http secure-serverbut noip http authentication local, the HTTP server does not use the local user database for basic auth unless the default login method points there. The service account exists, but the HTTP server falls back to the line/enable method, so the credentials never validate. Checkshow ip http server status,show running-config | include ip http, andshow running-config | section aaa. - 403 Forbidden means authentication succeeded but authorization failed or the account lacks privilege. A privilege-1 account can authenticate via RESTCONF but is denied write operations. Verify
show privilegeand theaaa authorization console/aaa authorization exec defaultconfiguration.
The split across identical hardware proves the issue is per-device config drift, not the script's credential string. Fix the AAA/HTTP method first, then rerun only the 17 failed switches.
2. The reruns caused DATA-OLD and an overwritten port description. Identify the design flaw and rewrite the logic as a safe idempotent operation.
Read the worked answer
The flaw is non-idempotent write logic with blind retries. A POST to a keyed YANG list is a create; when the key already exists the server should return 409, but a retry loop that falls back to PUT with a stale template overwrites the existing leaf values. That explains DATA-OLD and the overwritten description.
Safe pattern:
GETthe current VLAN/interface state, with anETagif supported.- Compare only the fields you own; skip the device if equal.
PUTthe minimal body, orPATCHwith themergeoperation.- Treat 409 as success-if-match, not as a reason to rewrite.
- Log device, URL, status, and payload hash per attempt; cap retries.
Run the corrected script with a dry-run flag against the 21 already-good switches to prove it would make zero changes.
3. Choose between NETCONF/YANG and CLI scraping via Netmiko for this fleet, and justify with two operational criteria.
Read the worked answer
Use NETCONF/YANG, not CLI scraping. Two operational criteria:
- Structured, transactional changes. NETCONF exposes a candidate datastore and
<commit>, so a failed VLAN change does not leave half-applied config. CLI scraping requires parsingshow runand is vulnerable to output formatting changes across IOS-XE releases. - Standard error and capability negotiation. The device advertises YANG models via
<hello>; the script can confirmCisco-IOS-XE-vlansupport before writing. Netmiko/CLI has no equivalent machine-readable schema and no atomic rollback.
For 38 IOS-XE 9200s, enable netconf-yang and restconf, use ietf-interfaces and Cisco-IOS-XE-vlan models, and keep RESTCONF for quick GETs. CLI is acceptable only for a one-off recovery when the management plane is degraded.