sudo Works Interactively but Fails Inside the Hardened CI Unit
On Ubuntu 22.04, the release team's CI agent runs as the deploy user inside a systemd service (ci-agent.service) on a shared build host. Administrators added /etc/sudoers.d/90-deploy containing deploy ALL=(root) NOPASSWD: /usr/bin/systemctl restart myapp and validated it with visudo -cf. The command works when an engineer SSHs in as deploy, including with sudo -n. Under the service it fails immediately with sudo: effective uid is not 0, is sudo installed setuid root? and exit code 1. The unit was hardened last sprint: it carries NoNewPrivileges=yes, ProtectSystem=strict, PrivateTmp=yes, and DynamicUser=no. The service must restart myapp after each deployment. The team will not grant a full root shell and cannot run the agent as root. systemctl lives at /usr/bin/systemctl; the sudoers file is mode 0440, root:root; the agent's working directory is its own private tmpfs.
1. Explain why the sudoers rule is not the cause and what the error message actually indicates. How would you prove it under the real unit?
Read the worked answer
The failure happens before sudo ever reads its policy.
sudo is a setuid-root binary; elevation depends on the kernel honouring the setuid bit at execve(). NoNewPrivileges=yes sets PR_SET_NO_NEW_PRIVS on the service process, and the kernel then refuses to grant any new privilege to that process or its children - including setuid. sudo starts with an effective UID equal to deploy, sees geteuid() != 0, and prints exactly that message.
So the sudoers syntax, the mode 0440, and visudo -cf were never in question: no policy lookup is attempted.
Proof, in order:
systemctl cat ci-agent.serviceand its drop-ins - confirmNoNewPrivileges=yesis actually in effect.- Reproduce outside the service:
systemd-run --uid=deploy --property=NoNewPrivileges=yes /usr/bin/sudo -n /usr/bin/systemctl restart myappfails; the same command without that property succeeds. sudo -lasdeployinteractively shows the rule matches, which isolates the environment rather than the policy.journalctl -u ci-agentshows the same exit code with no sudo policy log line.
This is the classic 'validated the file, missed the sandbox' trap.
2. Rank the remediation options by least privilege and by risk, and justify your first choice.
Read the worked answer
Preferred: keep every hardening flag and remove sudo from the path. systemctl restart myapp reaches PID 1 over D-Bus, where authorization is polkit's job, not setuid's. Add /etc/polkit-1/rules.d/50-deploy-myapp.rules granting org.freedesktop.systemd1.manage-units to deploy only when the looked-up unit equals myapp.service, plus the matching action in a .policy file. The agent then runs plain systemctl restart myapp. Least privilege: one verb on one unit, no root shell, no setuid dependency, and the unit keeps NoNewPrivileges=yes.
Acceptable: a narrow privileged helper. A root-owned wrapper with a hard-coded unit name, executed by a dedicated unit that deploy is allowed to start - or a sudo rule after deliberately setting NoNewPrivileges=no on that one unit.
Avoid: clearing NoNewPrivileges on the whole agent. It weakens the entire unit, including any binary the agent executes from a workspace it can write, and re-opens sudo as a privilege-escalation path. You would be trading a real sandbox boundary for one convenience command.
In all cases keep the sudoers entry exact-match - no systemctl *, no wildcards, no ALL on the command - and verify with sudo -ll that the resolved command and arguments match what the agent actually invokes.
3. Describe how you would verify the chosen fix, including a negative test, and one residual risk.
Read the worked answer
- Reproduce the failure under the real unit before changing anything:
systemd-run --unit=ci-test --uid=deploy -p NoNewPrivileges=yes /usr/bin/sudo -n /usr/bin/systemctl restart myappโ expect exit 1 with the same message. - Apply the polkit rule,
systemctl restart polkit, then run the same path without sudo:systemd-run --uid=deploy -p NoNewPrivileges=yes /usr/bin/systemctl restart myappโ expect success, andjournalctl -u polkitto show an authorization granted formyapp.service. - Negative tests. The same user must fail on
systemctl restart sshd,systemctl stop myapp, and a directbusctl call org.freedesktop.systemd1 /org/freedesktop/systemd1 org.freedesktop.systemd1.Manager StartUnit nginx.service s replace. If any succeeds, the rule is too broad - usually a missing unit check. - Confirm the sandbox is intact:
systemd-analyze security ci-agent.servicestill reportsNoNewPrivileges=yes. - Audit trail: one attributable entry per deployment across
journalctl -u ci-agent -u polkit.
Residual risk: polkit rule ordering and interaction with distro defaults - an earlier return polkit.Result.NO can override your later allow, and a vendor update can ship a rule that weakens yours. The negative tests are the control; re-run them after package upgrades, and keep the rule file under configuration management rather than hand-edited on the host.