The dual-homed historian: availability constraints in a live utility
You are on day two of a five-day on-site assessment for a water utility. The scope names the corporate IT network (10.10.0.0/16) and 'the SCADA historian at 10.20.4.15'. During reconnaissance you learn the historian is dual-homed: one interface on corporate IT, one on the OT network (10.20.0.0/16), which is not listed. The engineering workstation at 10.20.4.20 has an always-on RDP session to the historian and controls pump setpoint logic. The client's operations manager says 'it's all ours, test whatever you need', but the signed rules of engagement say the testing window is 02:00–05:00 and that no action may affect process availability. At 02:40 your scan of the historian triggers a Modbus exception log alert that the control-room supervisor sees, and she calls you asking whether pumps are about to stop. You have three days left.
1. What do you do in the next 30 minutes, and why?
Read the worked answer
The first action is to stop, not to explain. I would terminate the running scan against 10.20.4.15, then ask the control-room supervisor what she actually saw. A Modbus exception log is a read/parse error and not necessarily a process change, but I cannot establish that from my side of the network. I would ask her to confirm pump states are normal and record her confirmation with a timestamp. Then I would call the named engagement contact - not the operations manager who gave verbal scope - and report what I ran, against which host, what the alert was, and that I have paused all OT-adjacent activity. I would preserve the scan log and the exact request that triggered the exception. The reasoning is that the ROE's availability constraint outranks completing the test, and an unverified claim of 'no impact' is worse than a verified pause. I would not resume anything on 10.20.x until the client and an OT engineer agree in writing. I would also record that the alert may have been coincidental - a scan touching a Modbus register can produce exactly this log - but the burden is on me to prove that, not on the control room to disprove it.
2. The operations manager has verbally authorised you to test the whole OT network. How do you handle this?
Read the worked answer
The operations manager's 'it's all ours' is not authorization. The signed ROE lists corporate IT and one host; the OT network is out of scope, and OT carries safety consequences that generic IT authorization does not cover. I would decline to test 10.20.0.0/16 under the existing document and request a written amendment that names the exact OT hosts and subnets, defines permitted techniques (for example passive capture and review of exported configurations rather than active scanning), names an OT engineer as an approver, sets a maintenance window and a stop condition, and confirms that insurance and indemnity cover OT work. I would explain why: an active scan or exploit against a PLC or historian can change process state, and the no-availability-impact clause would be breached even if nothing visibly fails. If the client cannot amend in time, I document the OT network as out of scope and untested, with the reason, and reflect that in the coverage section. I would not 'just do a light scan' as a compromise - 'light' is not a defined control, and it would create exactly the same liability with none of the protections.
3. How do you finish the engagement in three days, and what do you say in the report about coverage?
Read the worked answer
I would re-plan around risk and evidence. Day three: finish corporate IT coverage on the highest-value targets - domain controllers, remote access, finance and billing systems - rather than spreading effort evenly across a /16. Use the historian's corporate interface only for passive, read-only checks that the ROE permits, or leave it alone. Day four: consolidate evidence, manually validate the top findings, and brief the client. Day five: write the report. The report would include an explicit coverage statement: which subnets were tested, which were sampled, which were untouched and why; what the OT network's exposure looks like from the corporate side, which is in scope and is often the real finding; and the residual risk of not testing OT. I would state clearly that the assessment cannot be represented as full coverage of the utility's estate, and recommend a separate OT assessment under a suitable framework. The judgement is that a defensible partial result with honest limitations is worth more than a broad, shallow scan that risks the plant and the client's regulatory position.