Inverter-based resources (IBRs) possess dynamics that are significantly different from those of synchronous-generator-based sources and as IBR penetrations grow the dynamics of power systems are changing. This article discusses the characteristics of the new dynamics and examines how they can be accommodated into the long-standing categorizations of power system stability in terms of angle, frequency, and voltage stability. It is argued that inverters are causing the frequency range over which angle, frequency, and voltage dynamics act to extend such that the previously partitioned categories are now coupled and further coupled to new electromagnetic modes. While grid-forming (GFM) inverters share many characteristics with generators, grid-following (GFL) inverters are different. This is explored in terms of similarities and differences in synchronization, inertia, and voltage control. The concept of duality is used to unify the synchronization principles of GFM and GFL inverters and, thus, established the generalized angle dynamics. This enables the analytical study of GFM-GFL interaction, which is particularly important to guide the placement of GFM apparatuses and is even more important if GFM inverters are allowed to fall back to the GFL mode during faults to avoid oversizing to support short-term overload. Both GFL and GFM inverters contribute to voltage strength but with marked differences, which implies new features of voltage stability. Several directions for further research are identified, including: 1) extensions of nonlinear stability analysis to accommodate new inverter behaviors with cross-coupled time frames; 2) establishment of spatial–temporal indices of system strength and stability margin to guide the provision of new stability services; and 3) data-driven approaches to combat increased system complexity and confidentiality of inverter models.

The large-scale integration of converter-interfaced resources in electrical power systems raises new threats to stability which call for a new theoretical framework for modelling and analysis. In this paper, we present the intrinsic analogy of a power system to a communication system, which is here called power-communication isomorphism. Based on this isomorphism, we revisit power system stability from a communication perspective and thereby establish a theory that unifies the heterogeneous power apparatuses of power systems and provides a bridge between electromagnetic transient (EMT) and phasor dynamics. This theory yields several new insights into power system stability and new possibilities for stabilization. In particular, we demonstrate that a system of 100% converter-interfaced resources can achieve stable synchronization in small- and large-signal sense under grid-following control which was commonly considered impossible.

Value factorisation is a useful technique for multi-agent reinforcement learning (MARL) in global reward game, however, its underlying mechanism is not yet fully understood. This paper studies a theoretical framework for value factorisation with interpretability via Shapley value theory. We generalise Shapley value to Markov convex game called Markov Shapley value (MSV) and apply it as a value factorisation method in global reward game, which is obtained by the equivalence between the two games. Based on the properties of MSV, we derive Shapley-Bellman optimality equation (SBOE) to evaluate the optimal MSV, which corresponds to an optimal joint deterministic policy. Furthermore, we propose Shapley-Bellman operator (SBO) that is proved to solve SBOE. With a stochastic approximation and some transformations, a new MARL algorithm called Shapley Q-learning (SHAQ) is established, the implementation of which is guided by the theoretical results of SBO and MSV. We also discuss the relationship between SHAQ and relevant value factorisation methods. In the experiments, SHAQ exhibits not only superior performances on all tasks but also the interpretability that agrees with the theoretical analysis. The implementation of this paper is placed on https://github.com/hsvgbkhgbv/shapley-q-learning.

This paper presents a problem in power networks that creates an exciting and yet challenging real-world scenario for application of multi-agent reinforcement learning (MARL). The emerging trend of decarbonisation is placing excessive stress on power distribution networks. Active voltage control is seen as a promising solution to relieve power congestion and improve voltage quality without extra hardware investment, taking advantage of the controllable apparatuses in the network, such as roof-top photovoltaics (PVs) and static var compensators (SVCs). These controllable apparatuses appear in a vast number and are distributed in a wide geographic area, making MARL a natural candidate. This paper formulates the active voltage control problem in the framework of Dec-POMDP and establishes an open-source environment. It aims to bridge the gap between the power community and the MARL community and be a drive force towards real-world applications of MARL algorithms. Finally, we analyse the special characteristics of the active voltage control problems that cause challenges (e.g. interpretability) for state-of-the-art MARL approaches, and summarise the potential directions.

Cooperative game is a critical research area in the multi-agent reinforcement learning (MARL). Global reward game is a subclass of cooperative games, where all agents aim to maximize the global reward. Credit assignment is an important problem studied in the global reward game. Most of previous works stood by the view of non-cooperative-game theoretical framework with the shared reward approach, i.e., each agent being assigned a shared global reward directly. This, however, may give each agent an inaccurate reward on its contribution to the group, which could cause inefficient learning. To deal with this problem, we i) introduce a cooperative-game theoretical framework called extended convex game (ECG) that is a superset of global reward game, and ii) propose a local reward approach called Shapley Q-value. Shapley Q-value is able to distribute the global reward, reflecting each agent's own contribution in contrast to the shared reward approach. Moreover, we derive an MARL algorithm called Shapley Q-value deep deterministic policy gradient (SQDDPG), using Shapley Q-value as the critic for each agent. We evaluate SQDDPG on Cooperative Navigation, Prey-and-Predator and Traffic Junction, compared with the state-of-the-art algorithms, e.g., MADDPG, COMA, Independent DDPG and Independent A2C. In the experiments, SQDDPG shows a significant improvement on the convergence rate. Finally, we plot Shapley Q-value and validate the property of fair credit assignment.

Power electronics based dc transformer is the key equipment for MVDC distribution network interconnection, which requires both high step-ratio and low step-ratio dc-dc conversions to interface dc links at different voltages. This paper reviews the original high step-ratio resonant modular multilevel dc converter (RMMC) for MVDC applications and then presents a step-by-step circuit evolution which drives the high step-ratio RMMC to a low step-ratio RMMC, which inherits all the key operational advantages that were present in high step-ratio RMMC version including soft-switching operation, inherent balancing and high effective switching frequency. Furthermore, based on these two basic RMMCs, a derivative family of RMMCs for both high step-ratio and low step-ratio conversion is provided for different kinds of interconnection requirements in MVDC distribution system. The theoretical analysis for both high step-ratio and low step-ratio RMMCs are verified by a set of medium voltage full-scale simulation examples.