Objectives

Motivation

Modern electrical grids are increasingly complex cyber-­physical systems, with advanced sensing, control and communication layers overlaid on nonlinear dynamical networks. Real-­time grid operations are aided by closed ­loop feedback control decisions that rely on advanced sensor measurements to dynamically secure the balance between supply and demand. Such tightly coupled cyber­-physical control operation arguably presents an opportunity for malicious cyber agents to sneak into the system and cause disruptions, equipment damages and/or financial losses. Real-­world events such as the Stuxnet [2], the Dragonfly [3], and the 2015 cyber­attacks on the Ukrainian power grid [4] and on the Metcalf substation in the state of California in US [5], demonstrate the vulnerability of power systems worldwide to cyber-attacks. Different, possibly overlapping, classes of cyber­attacks have been investigated in the power systems community, including denial of service (DoS) attacks [6, 7]; gray­hole or packet drop attacks [8]; jamming or link ­failure attacks [6, 9]; and false data injection (FDI) or data integrity attacks [6,7,10–12].

Timely detection and appropriate mitigation of adversarial events in power systems are critical when it comes to dynamically steering the system into “safety.” Any malicious activities on measurements from sensors like Phasor Measurement Units (PMUs) and Supervisory Control and Data Acquisition (SCADA) system, or on control signals from Automatic Generation Control (AGC) can mislead the control center operator into taking wrong control actions, and therefore lead to disruption of operation, financial losses, and equipment damage. There is a lack of in-depth evaluation of the behavior of AGC under such conditions, and at present, there are no effective approaches for detecting or mitigating potential adversarial conditions. This project aims to apply deep learning techniques to dynamically detect adversarial scenarios, and identify in real-time optimal recourse actions that steers the system back to normal operating conditions via a resilient “autopilot” mechanism.

Objectives

The PowerDrone project developed a data­-driven approach towards detection and mitigation of adversarial conditions in the power grid to adaptively steer the system back to secure operational mode. Today the state­-of­-the-art Automatic Generation Control (AGC) use SCADA data, which has a sampling rate of one measurement every 2-­5 seconds and does not capture the dynamics of the system. Additionally, the existing controls are not capable of detection of adversarial scenarios or adaption of controls to mitigate their impacts. We have developed a prototype of “PowerDrone” which is an intelligent controller that will use Reinforcement Learning (RL)­ driven control techniques on high sampling rate PMU data to detect adversarial conditions and mitigate the impacts. Specifically, our effort focused on development of an open-­source data generation framework, detection and localization of stealthy manipulation of PMU measurements, and the development of an RL­-based smart AGC application which is capable of bypassing cyber­-attacks and taking appropriate control decisions to ensure grid stability.

A graph-theoretic model of cyber-physical systems has been developed by abstracting the four layers: physical layer (generators, transmission network, relays), sensor layer (SCADA, PMU), control layer (AGC, network controller), and cyber layer (cyber topology, netflow, firewall logs), as shown in Figure 1.

The following three modules have been developed and integrated into PowerDrone:

  • Red Team: Simulates adversarial actions through cyber and sensor layers which can lead to impacts in the physical layer of the system.
  • Blue Team: Detects adversarial conditions and steers system to safety by mitigation steps which include RL­-based controls and network configuration changes.
  • Systems Layer: Cyber-­physical system model integrated with red team and blue team modules for evaluating performance under various adversarial­ defensive scenarios.
fig 1

Figure 1: Figure showing the Red Team, Blue Team, and Systems Layer modules of PowerDrone (left); and the associated physical, sensor, communication, and control layers of the system (right).

References:

[1] J. Achiam, OpenAI Spinning Up as a Deep RL Researcher, GitHub, GitHub repository (2018).

[2] ICS Advisory (ICSA­10­272­01): Primary Stuxnet Advisory (Sep. 2010). URL https://us-cert.cisa.gov/ics/advisories/ICSA-10-272-01

[3] Alert (TA18­074A): Russian Government Cyber Activity Targeting Energy and Other Critical Infrastructure Sectors (Mar. 2018). URL https://us-cert.cisa.gov/ncas/alerts/TA18-074A

[4] ICS Alert (IR­ALERT­H­16­056­01): Cyber­Attack Against Ukrainian Critical Infrastructure (Feb. 2016). URL https://us-cert.cisa.gov/ics/alerts/IR-ALERT-H-16-056-01

[5] Quadrennial Technology Review 2015 (Sep. 2015). URL https://www.energy.gov/sites/default/files/2015/09/f26/ QTR2015-3A-Cyber-and-Physical-Security_0_0.pdf

[6] S. Pushpak, A. Diwadkar, M. Fardad, U. Vaidya, Vulnerability analysis of large­scale dynamical networks to coordinated attacks, in: 2014 4th Australian Control Conference, IEEE, 2014, pp. 89–94.

[7] M. N. Kurt, Y. Yılmaz, X. Wang, Distributed quickest detection of cyber­attacks in smart grid, IEEE Trans. Information Forensics and Security 13 (8) (2018) 2015–2030.

[8] S. Pal, B. Sikdar, J. H. Chow, An Online Mechanism for Detection of Gray­Hole Attacks on PMU Data, IEEE Trans. Smart Grid 9 (2018) 2498–2507.

[9] M. N. Kurt, Y. Yılmaz, X. Wang, Real­time detection of hybrid and stealthy cyber­attacks in smart grid, IEEE Trans. Information Forensics and Security 14 (2) (2018) 498–513.

[10] R. B. Bobba, K. M. Rogers, Q. Wang, H. Khurana, K. Nahrstedt, T. J. Overbye, Detecting false data injection attacks on DC state estimation, in: Workshop on Secure Control Systems, CPSWEEK, 2010.

[11] Y. Liu, P. Ning, M. K. Reiter, False data injection attacks against state estimation in electric power grids, ACM Trans. Information and System Security (TISSEC) 14 (1) (2011) 1–33.

[12] L. Xie, Y. Mo, B. Sinopoli, Integrity data attacks in power market operations, IEEE Trans. Smart Grid 2 (4) (2011) 659–666.