Unauthorized PLC Change During a Production Batch
A plant uses Cisco Cyber Vision sensors on a mirrored SPAN port. At 14:05, Cyber Vision flags a new PLC program download from ENG-WS-03 to PLC-07 on the production line; the engineering change ticket is for a different PLC and expired yesterday. The line is running a batch that cannot be interrupted for 6 hours without scrapping product. ENG-WS-03 has no EDR and runs Windows 7; a USB drive was used at 13:50. Cyber Vision shows SMB and CIP traffic, and the firewall between IT and OT is a Cisco Secure Firewall in routed mode with no IPS on the OT side. The plant manager wants to unplug the engineering workstation immediately; operations wants to finish the batch. The SOC has no OT engineer on shift.
1. What is the immediate containment decision given safety and production constraints?
Read the worked answer
Safety first: do not unplug PLC-07 or stop the line abruptly, because that can damage product or equipment. Isolate ENG-WS-03 from the OT network at the switch port or with a pre-approved ACL, but keep it powered for memory capture. Verify whether PLC-07 actually changed by comparing its program checksum against the Cyber Vision baseline and checking the change ticket. If an unauthorized change is confirmed and safety permits, ask operations to pause between batches, not mid-batch. Block removable media through policy and monitor PLC-07 with Cyber Vision and the SPAN port. Do not run active scans on OT. Notify the plant manager, security leadership, and engineering.
2. How do you collect forensic evidence in an OT environment without disrupting the process?
Read the worked answer
Collect Cyber Vision traffic metadata and retained PCAP if available; capture ENG-WS-03 memory, USB history, Windows event logs, project files, and PLC program backups. Image the workstation before remediation. Capture the PLC program and configuration through the engineering tool only with operations approval and during a safe window. Maintain chain of custody. Because Windows 7 has no EDR, use a live-response toolkit from a write-blocked USB or a controlled network share. Record the timeline: USB insert at 13:50 and download at 14:05. Avoid rebooting until memory is captured, and coordinate any maintenance window with operations so evidence collection does not cause an unplanned outage.
3. What coordination and post-incident actions does this incident require?
Read the worked answer
Establish incident command with the plant manager, operations, engineering, and IT security. Decide containment based on safety and production risk; if the line continues, document the risk acceptance and compensating monitoring. After the batch, isolate and rebuild ENG-WS-03, replace or strictly segment Windows 7, restrict the engineering VLAN, enable firewall inspection on the IT-OT boundary, and enforce USB control. Verify PLC program integrity against a trusted engineering backup. Conduct lessons learned: an expired ticket for a different PLC should trigger an alert. Update Cyber Vision baselines and change management. Report to management and regulators if safety or product quality was affected, and preserve evidence for a possible insider investigation.