Unauthorized PLC Program Change at 02:14
At 02:14, a SCADA alarm indicates that the program on three of twelve water-treatment PLCs has changed. No change ticket exists, and the only engineer authorized for those PLCs is on leave. The PLCs control chemical dosing, and an incorrect dose could violate permit limits within 30 minutes. The historian shows the changes but has no packet capture. Firewall logs show an RDP session from an engineering laptop at 01:58 to a jump host, then to the PLC subnet. The laptop is still online. The SOC analyst on duty must decide whether to isolate the PLCs, block the laptop, or monitor. The plant has a 24/7 operations center but no dedicated OT IR team; the on-call manager can be reached in 10 minutes.
1. What are your first three actions in the first 15 minutes, and why?
Read the worked answer
First 15 minutes: (1) confirm the alarm with operations and check whether the PLC program change matches any emergency work; call the on-call manager and the authorized engineer. (2) Contain the likely source: disable the switch port for the engineering laptop and block its IP at the firewall, but do not power it off if volatile evidence is needed; preserve firewall logs, jump-host logs, and historian change records. (3) Assess process safety: check dosing rates and permit limits; prepare manual dosing. Do not isolate the PLCs until operations confirms the process can be safely stopped or the dosing is already off-spec. Preserve evidence by copying logs and taking a memory capture of the laptop if policy allows.
2. Should you isolate the PLCs immediately? Justify your answer using safety, availability, and containment.
Read the worked answer
Not automatically. Isolating a PLC may stop chemical dosing or cause a fail-safe that is not safe for the process. Containment should start at the laptop and jump host, because that is where the unauthorized RDP session originated. Isolate the PLC subnet from the jump host and engineering network while keeping the PLC-to-I/O and PLC-to-HMI paths if needed. If the dosing is actively wrong, follow the emergency procedure to put the process in a safe state; that may mean manual control, not simply pulling the PLC. The decision must be joint with operations and documented.
3. What gaps does this incident expose, and what detection improvements would you prioritize?
Read the worked answer
Gaps: no packet capture, no automated PLC change detection, shared jump host, no after-hours change control, and no OT IR playbook. Prioritize a SPAN/TAP on the PLC subnet, alerting on PLC program downloads and RDP to jump hosts, unique accounts, and a playbook that defines safe isolation. Also add a call tree and pre-authorized emergency change process. These reduce time to detect and avoid unsafe containment.