OT/ICS Incident Response: Containing Threats in Operational Technology

Cybersecurity

IT-style IR procedures fail in OT environments. From historian server compromise to HMI hijack, this playbook covers ISA/IEC 62443-aligned response steps for operational technology incidents.

By Arjun Raghavan, Security & Systems Lead, BIPI · October 3, 2024 · 11 min read

The Industroyer2 attack on Ukrainian energy infrastructure in 2022 and the Volt Typhoon pre-positioning campaign in US critical infrastructure underscore a consistent reality: adversaries are patient, OT networks are inadequately segmented, and IR teams trained exclusively on IT environments will make the situation worse. Pulling a PLC offline during active production can trigger physical consequences that dwarf the cost of the malware itself.

Why IT IR Playbooks Break in OT Environments

• Availability is the primary security objective in OT. In IT, confidentiality comes first. Shutting down a process to contain a threat may cause more harm than the threat itself.
• Patch cycles in OT are measured in years, not weeks. Vendor-supported equipment running Windows XP embedded is common.
• OT protocols (Modbus, DNP3, PROFINET, EtherNet/IP) are not encrypted and were not designed with authentication in mind.
• Many OT systems cannot tolerate a network scan. Running Nmap against a PLC can crash it.
• Forensic imaging of a PLC or historian server is fundamentally different from imaging a Windows workstation and requires vendor support.

Segmentation Failure: The IT/OT Boundary

The most common entry path for OT incidents is not a zero-day against a PLC. It is a flat or poorly segmented network where IT ransomware pivots across the Purdue Model boundary into the OT DMZ and beyond. In the 2021 Oldsmar water treatment incident, the attacker accessed the HMI directly over TeamViewer, a remote access tool running on a machine that was also connected to the IT network without a firewall between them.

Historian Server Compromise: What to Do and What Not to Do

Historian servers (OSIsoft PI, AspenTech, eDNA) sit at the IT/OT boundary and aggregate process data for business reporting. They are attractive targets because they have IT-like operating systems, are often domain-joined, and have read access into the OT network. When a historian is compromised, you must determine whether the attacker has used it as a pivot point into the control network before isolating it.

1. Check historian connectivity: Does it have active connections to Level 1 or Level 2 devices at the time of discovery? Use netstat and firewall logs, not a network scan.
2. Review authentication logs for the historian service account. Has it been used to authenticate to PLC web interfaces or engineering workstations?
3. Snapshot the historian VM if it is virtualized before any containment action. This preserves process data and forensic state simultaneously.
4. Notify the process control engineer on shift. They must know the historian is being isolated so they can switch to local HMI readings.
5. Isolate the historian at the firewall level, not by pulling a network cable. A sudden loss of connectivity can cause cascading alarms in some DCS configurations.

HMI Hijack: Indicators and Immediate Response

A compromised HMI is the most operationally dangerous scenario in an OT incident. An attacker with write access to an HMI can change setpoints, open or close valves, and disable safety interlocks. Detection often comes from process anomalies (unexpected setpoint changes, alarm floods) rather than from security tooling.

• Immediate: Have the control room operator switch affected HMI stations to local control at the PLC panel. This breaks the software path to the compromised interface.
• Verify that safety instrumented systems (SIS) are still operating independently and have not been disabled. SIS bypass is a hallmark of sophisticated OT attacks.
• Do not reboot the HMI during active operations. A reboot can cause the PLC to enter a failsafe state that may require a manual plant restart.
• Capture a memory image of the HMI workstation before any remediation. HMI malware is often fileless or stored in non-standard locations.
• Engage the HMI vendor's emergency support. Many SCADA vendors (Wonderware/AVEVA, FactoryTalk, Ignition) have 24-hour critical support lines.

When an HMI is compromised, the control room operator's judgment takes precedence over IT containment decisions. Physical process safety is non-negotiable.

ISA/IEC 62443 Response Framework

ISA/IEC 62443-2-1 defines requirements for an IACS (Industrial Automation and Control System) security management system, including incident response. Unlike NIST CSF, it explicitly addresses the zone-and-conduit model, security levels, and the distinction between IT and OT response procedures.

Evidence Collection in OT Environments

• Collect PLC ladder logic exports and project files. Changes to logic are the equivalent of malware implants in OT.
• Extract DCS event logs and alarm historian data. Process anomalies recorded in the historian may predate the security alert by days or weeks.
• Capture network traffic from the OT DMZ if a tap or span port is available. Most OT networks lack EDR, making network forensics the primary data source.
• Document all setpoint values at the time of discovery and compare against the engineering baseline. Unauthorized changes must be recorded for root cause analysis.
• Preserve all copies of HMI screen configurations and tag databases. Attackers who modify tags can cause operators to see false process readings.

Recovery and Revalidation

Returning OT systems to service after a security incident requires engineering sign-off, not just IT sign-off. Every PLC, RTU, and HMI that was on the compromised network segment must have its configuration validated against the last known-good engineering baseline before resuming controlled operation. For critical processes, a phased restart under manual supervision is mandatory.

Read more field notes, explore our services, or get in touch at info@bipi.in. Privacy Policy · Terms.