Eleven Minutes Per Manager
Halden Precision manufactures components across three plants. It employs 6,000 people and 2,500 contractors. Identity is a single Active Directory forest plus four legacy maintenance applications with local accounts. Quarterly access reviews are run by line managers through a spreadsheet; the last cycle certified 99.6% of 41,000 entitlements as correct, and the average manager spent 11 minutes on their review. There are roughly 1,200 privileged accounts, and about 310 of them are not attributable to a named individual, mostly service accounts and shared shift accounts. Plant engineers hold a sealed 'break-glass' envelope for the admin_ot account, but the control room log shows it used at least weekly. HR notifies IT of leavers within three working days; contractor end dates live in agency spreadsheets. The last internal audit found 14 enabled accounts belonging to people who had left, two of them privileged, and no evidence that the OT accounts had ever been reviewed.
1. Which single access-control failure in this environment creates the greatest risk, and how would you sequence remediation without halting plant operations?
Read the worked answer
The largest risk is not any one of the listed defects but their combination around non-attributable privilege: 1,200 privileged accounts, 310 without a named holder, a review that never covers OT, and a leaver process that lags by days. The reason attribution matters more than the others is that every downstream control depends on it. Logging, anomaly detection, investigation, revocation and disciplinary process all assume an action can be tied to a person. Where a shift account is shared, an incident cannot be scoped, a malicious insider cannot be identified, and the account cannot be disabled without stopping production.
Sequencing, given that plants cannot stop:
- Inventory and attribute, read-only. Compile every privileged account with owner, purpose, systems reached, and last use. Do this before changing anything; the first remediation attempt in OT environments usually breaks a process nobody documented.
- Convert the easy cases. Service accounts move to managed service accounts or a vault with rotated credentials; shared shift accounts move to individually named accounts with just-in-time elevation approved by the shift supervisor. Where a shared account is genuinely unavoidable on a legacy application, wrap it in a jump host with session recording so actions are attributable even if the credential is not.
- Bring OT into scope of review. Add the 310 accounts to the certification population with engineering owners, not IT managers, as reviewers.
- Automate the leaver path from the HR system and replace the agency spreadsheets with a contractor register that carries contractual end dates and produces disablement tickets. Measure revocation latency rather than asserting a three-day SLA.
- Compensating controls now, while the above runs: network segmentation between IT and OT, no direct internet path to plant systems, alerting on admin_ot use, and removal of standing vendor access.
The order reflects blast radius and reversibility. Attribution first because it is the precondition; automation second because it removes a recurring failure; segmentation in parallel because it limits harm if the rest is slow.
2. A 99.6% approval rate across 41,000 entitlements at 11 minutes per manager is not evidence of good hygiene. Diagnose what is happening and redesign the certification so it produces real assurance.
Read the worked answer
Eleven minutes is not enough to read, let alone evaluate, a typical manager's entitlement list; the arithmetic alone shows the review is an attestation of the spreadsheet rather than of access. Several failures are likely compounding. Reviewers are given raw entitlement identifiers from four legacy applications they do not administer, so they cannot judge. There is no usage data, so the safe answer is approve. There is no consequence for approving, and often a penalty for revoking, since revocation generates tickets and complaints. And 99.6% approval across a population that includes joiners, movers and contractors is statistically implausible - some proportion of drift is normal in any estate this size.
Redesign around four changes:
- Certify roles and privilege, not raw entitlements. Where a role-based model exists, certify the role assignment; reserve entitlement-level review for privileged, conflicting and exception access. This shrinks the population from 41,000 to something a manager can genuinely assess.
- Risk-based population selection. Every cycle, force review of: all privileged accounts, all segregation-of-duties conflicts, accounts dormant beyond 60-90 days, contractor and third-party accounts, and a random sample of the rest. The sample is what detects rubber-stamping.
- Provide decision support. Last login, last authentication source, whether the entitlement is in the user's role baseline, peer comparison, and a flag for anomalies. A reviewer with this data behaves differently from one with a list of codes.
- Make revocation the default for unjustified access, with an explicit justification field required to retain privileged access, and an automated revocation SLA after the cycle closes.
Measure the process on challenge rate, revocation latency, and percentage of the risk-based population reviewed on time - not on completion percentage. Have security independently re-test a sample of retained privileged access each quarter. If the challenge rate stays near zero after the redesign, the problem is the reviewer population, and certification should move to application owners and system owners who can see the entitlement in context.
3. How should the admin_ot 'break-glass' arrangement be redesigned, given weekly use and OT constraints?
Read the worked answer
The first judgement is definitional. A break-glass credential used weekly is not break-glass; it is an undocumented everyday administrative account with an envelope as its only control. Either the need is routine, in which case it should be provisioned as properly governed privileged access, or it is exceptional, in which case the weekly use indicates an unaddressed operational requirement. Both conclusions lead to the same place: give engineers a legitimate route to the access they demonstrably need, and keep a genuine emergency path separate.
For the routine path: named individual accounts with just-in-time elevation, approval by the shift supervisor or on-call manager, time-boxed to the maintenance window, checked out from a vault so the underlying credential is never known to the engineer, with full session recording through a jump host and alerts on use outside the window. This preserves the workflow while producing attribution and an audit trail.
For the genuine emergency path: distinct credentials, unique per site, stored in a tamper-evident container with a serialised seal and a logbook; use triggers an immediate alert to the on-call security lead and the plant manager; access expires automatically within a defined period; every use is reviewed within five working days by someone independent of the engineer, with a documented reason. Test the seal and the alerting quarterly.
OT constraints shape the compensating controls rather than the principle. Legacy applications may not support federation, so place the vault and jump host in a segmented zone with protocol-aware firewall rules, no direct internet path, and no standing vendor access. If an application truly cannot support named accounts, terminate sessions at the jump host with per-engineer authentication and recording, so the shared credential sits behind an attributable layer.
The measure of success is not the elimination of the envelope but a falling count of unattributed administrative actions, which should be reported to the plant leadership monthly until it reaches zero.