PNNL

Abstract

Microgrids are useful solutions for integrating renewable energy resources and providing seamless green electricity to minimize our energy footprint. In recent years, extreme weather events happened often worldwide and caused significant economic and societal losses. Such events bring uncertainties to the microgrids energy scheduling problem and increase the challenges of microgrid operation. Traditional optimization approaches suffer from the inaccuracy of the uncertain microgrid model and the unseen events. Existing reinforcement learning-based optimization approaches are also hampered by the limited generalization and the increasing computational burden when stochastic formulations are required to accommodate the uncertainties. This paper proposes a new parallelized reinforcement learning (PRL) method to handle the microgrid energy scheduling problem during uncertain weather events. Specifically, several local learning agents are employed to interact with pertinent microgrid environments in a distributed manner and report outcomes to the global agent, which will optimize microgrid energy resources online during extreme events. The stochastic microgrid energy optimization problem is reformulated to include all possible scenarios with probabilities. The advantage estimate functions are designed with a backward sweep to transfer the outcomes to the value function updating process. Two simulation studies, stochastic optimization, and online optimization are performed to compare with several existing reinforcement learning approaches. The influence of power outage duration is further discussed. Results show that the proposed PRL method can achieve up to 20% improvement of optimization performance and is multiple times faster than the existing approaches.