Interarea oscillations pose significant threats to the security and stability of power systems, and these oscillations have to be damped rapidly to prevent potential system failure. Our previous work introduced a deep reinforcement learning (DRL) strategy on top of the power oscillation damping (POD) control architecture using high-voltage direct current (HVdc) transmissions. This strategy effectively generated POD signals based on the wide-area information, thus modulating the transferred power to damp interarea oscillations. However, the performance degrades when there are significant changes in system structure or operating conditions, highlighting a general adaptability issue in DRL-based control schemes. To solve the issue, we propose a novel meta-evolution strategy framework that integrates both meta-strategy optimization (MSO) and evolutionary strategy (ES)-based DRL in this article. The ES algorithm has fewer hyperparameters that are relatively easy to tune than typical DRL algorithms and identifies the optimal policy quickly by exploring the policy parameter space through parallel computations. The MSO introduces a latent variable projecting the power system parameters under different operation conditions to ensure a fast and efficient adaption to unforeseen scenarios. A modified IEEE 39-bus system with HVdc transmission is studied to validate the performance and effectiveness of the proposed meta-strategy in damping interarea oscillations.

Static Var compensator (SVC) has often been used in the power system industries to regulate the bus voltage and reactive power. With an increasing concern of the power system transient stability, the SVC has also been used to dynamically regulate the power flow for oscillation dampings in recent years. A guided surrogate-gradient-based evolutionary strategy (GSES)-based SVC control approach is proposed in this article to damp power system interarea oscillations. The GSES algorithm is used to train a reinforcement learning agent to learn the best decisions that control the SVC, with an objective of quickly damping interarea oscillations. The GSES algorithm brings many benefits such as easier, more robust, and efficient training with less required hyperparameters. Because the GSES can engage with multiple individual workers during the training process, parallel computation techniques are implemented in this research to accelerate the computational speed. To test the performance of the proposed GSES-based SVC controller, we compared this novel method with a supplementary SVC damping controller on two different test systems. Results indicate that the proposed GSES-based SVC control approach can effectively damp power system interarea oscillations. Especially, in some case studies, the proposed approach successfully saved the test system from a transient-instability-caused breaking down.

One of the key challenges to deep reinforcement learning (deep RL) is to ensure safety at both training and testing phases. In this work, we propose a novel technique of unsupervised action planning to improve the safety of on-policy reinforcement learning algorithms, such as trust region policy optimization (TRPO) or proximal policy optimization (PPO). We design our safety-aware reinforcement learning by storing all the history of “recovery” actions that rescue the agent from dangerous situations into a separate “safety” buffer and finding the best recovery action when the agent encounters similar states. Because this functionality requires the algorithm to query similar states, we implement the proposed safety mechanism using an unsupervised learning algorithm, k-means clustering. We evaluate the proposed algorithm on six robotic control tasks that cover navigation and manipulation. Our results show that the proposed safe RL algorithm can achieve higher rewards compared with multiple baselines in both discrete and continuous control problems. The supplemental video can be found at: https://youtu.be/AFTeWSohILo.

With the challenges of increased grid dynamics and more variability of power generation from renewable energy sources, rapidly increasing complexity in the grid model, and abundant data from measurements and simulations, the requirements for computational analysis have also increased dramatically. This paper proposes a novel big data analysis approach for power system cascading analysis, prevention, and remediation. The developed techniques will be capable of cascading analysis, better assessment of the systems vulnerability level, as well as proposing potential remediation. Case studies using IEEE 118- bus system and a 563-bus system, with comparisons against a commercial tool, validate the advantages of the developed big data approach: accurate prediction, and more importantly, faster and effective correction actions. The developed techniques could be further used for other power system applications.

Deriving generation dispatch is essential for efficient and secure operation of electric power systems. This is usually achieved by solving a security-constrained optimal power flow (SCOPF) problem, which is by nature non-convex, usually nonlinear and thus computationally intensive. The state-of-the-art optimization approaches are not able to solve this problem for large-scale power systems within the power system operation time window (usually 5 minutes). In this work, we developed supervised learning approaches to determine security-constrained generation dispatch within a much shorter time window. More importantly, the physical constraint of only accessing to local measurements and other information in most utilities' real-time operation can not be ignored for the predictive models. The feasibility and accuracy of utilizing only local features (measurements and grid information in one area) to predict optimal local generation dispatch (dispatch of all generators in the corresponding area) in multi-area power systems has been explored. The results showed optimal local generation dispatch can be predicted with local features with high accuracy, which is comparable to the results obtained with global features.