PNNL

Abstract

Security of cyber-physical systems (CPS) continues to pose new challenges due to the tight integration and operational complexity of the cyber and physical components. To address these challenges, this paper presents a domain-aware, optimization-based approach to determine an effective defense strategy for CPS in an automated fashion – by emulating a strategic adversary in the loop that exploits system vulnerabilities, interconnection of the CPS, and the dynamics of the physical components. Our approach builds on an adversarial decision-making model based on a Markov Decision Process (MDP), that is used to determine the optimal cyber (discrete) and physical (continuous) attack actions over a CPS attack graph. The defense planning problem is modeled as a non-zero sum game between the adversary and defender. We first use a model-free reinforcement learning method to solve the attacker’s problem as a function of the defense strategy. Against the resulting attack policy, the defender’s problem of hardening the network is solved using Bayesian optimization (BO), and this process is repeated multiple times to find approximate best-responses for each player. The results are demonstrated on a ransomware inspired graph in conjunction with a smart building system as the physical process. Extensive numerical studies indicate convergence of both the attacker’s and defender’s objective towards a Nash equilibrium for various values of the defender-specific cost of network hardening.