PNNL

Abstract

Machine learning techniques are promising for enhanced operation of battery energy storage systems (BESSs) for grid applications. This paper presents an innovative optimal BESS dispatch strategy based on the deep deterministic policy gradient approach, incorporating action clipping and reward shaping techniques for secure exploration. To ensure the effectiveness of learning-based dispatch, the proposed method models nonlinear operational characteristics and various degradation effects of BESS. In particular, the varying charging/discharging efficiencies are modeled as functions of battery state of charge and charging/discharging power. In addition to loss of life, degradation effects on energy capacity and efficiencies are also considered. Case studies were performed to illustrate and validate the proposed method. It was found that the proposed control design provides 1.6 times greater cost savings than the control based on the constant efficiency model, while preserving the expected BESS lifespan and avoiding a 45% reduction in lifespan without considering the life loss model.