September 25, 2021
Journal Article

Deep Reinforcement Scheduling of Energy Storage Systems for Real-time Voltage Regulation in Unbalanced LV Networks with High PV Penetration

Abstract

The ever-growing higher penetration of distributed energy resources (DERs) in low-voltage (LV) distribution systems brings both opportunities and challenges to voltage support and regulation. This paper proposes a deep reinforcement learning (DRL)-based scheduling scheme of energy storage systems (ESSs) to mitigate system voltage deviations in unbalanced LV distribution networks. The ESS-based voltage regulation problem is formulated as a multi-stage quadratic stochastic program, with the objective of minimizing the expected total daily voltage regulation cost while satisfying operational constraints. While existing voltage regulation methods are mostly focused on onetime- step control, this paper explores a day-horizon systemwide voltage regulation problem. In other words, the size of action and state spaces are extremely high-dimensional and need to be delicately handled. Furthermore, in order to overcome the difficulty of modeling uncertainties and develop a realtime solution, a learn-to-schedule feedback control framework is proposed by adapting the problem to a model-free DRL setting. The proposed algorithm is tested on a customized 6-bus system and a modified IEEE 34-bus system. Simulation results validate the effectiveness and near-optimality of voltage regulation by ESS in comparison with a deterministic quadratic program solution.

Published: September 25, 2021

Citation

Wang S., L. Du, X. Fan, and Q. Huang. 2021. Deep Reinforcement Scheduling of Energy Storage Systems for Real-time Voltage Regulation in Unbalanced LV Networks with High PV Penetration. IEEE Transactions on Sustainable Energy 12, no. 4:2342-2352. PNNL-SA-163829. doi:10.1109/TSTE.2021.3092961