NX-OS TACACS+ Command Authorization and Fallback Lockout
A network team manages 12 Cisco Nexus 93180YC-FX switches running NX-OS 10.2(5). AAA is configured with two Cisco ISE TACACS+ servers and local fallback. NOC engineers use personal accounts mapped to the network-operator role. After an ISE device-administration policy change, engineers can authenticate but every show command returns % Authorization failed. The TACACS+ live logs show Authen OK followed by Authorz FAIL. During a later ISE outage, the same engineers cannot log in at all; the local fallback user is disabled and login fallback is not configured. The break-glass network-admin account exists, but its password is sealed and the change freeze starts in 30 minutes. The team wants to restore NOC read-only access quickly without granting network-admin.
1. What is failing: authentication, authorization, or accounting? What evidence and commands confirm it?
Read the worked answer
Authentication is succeeding because the TACACS+ log shows Authen OK and the engineers get a prompt. Authorization is failing because the next step, Authorz FAIL, occurs for every show command and the CLI returns % Authorization failed. Accounting is not the cause; it only records activity. The ISE policy change probably stopped returning the correct shell role or command authorization set for the network-operator role. On NX-OS, check show aaa, show running-config | include aaa, show tacacs+, and use test aaa with a test account. Confirm that aaa authorization commands default points to the TACACS+ group and that the ISE device-administration policy returns a shell profile permitting show commands. If the TACACS+ server returns no command set, NX-OS has nothing to authorize against and denies the command.
2. What configuration changes on NX-OS and ISE restore read-only NOC access while preserving least privilege?
Read the worked answer
On NX-OS, keep authentication and authorization pointed at the TACACS+ group but add a local fallback so a server outage does not lock out NOC. Ensure aaa authentication login default group tacacs+ local and a login fallback such as aaa authentication login fallback error local are present. For command authorization, use aaa authorization commands default group tacacs+ local if local fallback for commands is required. On ISE, fix the TACACS+ device-administration policy so it returns the intended shell role, such as network-operator, and a command set that permits read-only commands including show. Do not return network-admin just to clear the error. If local fallback is used, create a local network-operator user and role with the same read-only command permissions. Test with a dedicated account, confirm show interface brief succeeds, and confirm a configuration command is still denied. That preserves least privilege while restoring NOC visibility.
3. How do you test and avoid locking yourself out during the change freeze?
Read the worked answer
Keep an existing privileged session open while changing AAA; do not close it until a second test session succeeds. Configure and verify local fallback before changing authorization, so an ISE outage does not repeat the lockout. Use a dedicated test account rather than a production NOC account, and test authentication, a permitted show command, and a denied configuration command. Confirm that the break-glass account is usable and that its password is available under the emergency procedure. Apply the change to one switch first, then a small canary group, before the remaining switches. Have a rollback ready that restores the previous AAA configuration from a saved copy. Monitor TACACS+ live logs and NX-OS accounting during the test. If the test session fails, use the still-open privileged session to roll back rather than escalating the NOC role to network-admin.