With increasing penetration of Electric Vehicles (EVs) into the transportation system and smart electricity grid, there is a growing need for integrating them into Home Energy Management Systems (HEMS). This integration within HEMS introduces dynamic user behaviors and time-varying charging demand, thus posing challenges for the HEMS. To mitigate these challenges, this paper proposes a charging model for heterogeneous EVs that covers the range of Plug-in Hybrid EVs (PHEVs), Range-Extender EVs (REEVs) and Battery EVs (BEVs) with/without heat pumps. The proposed heterogeneous EV charging model considers weather conditions, estimated mileage and driver’s experience to describe the dynamic charging demand and the anxiety level influencing their behavior. To optimize the HEMS operation, minimizing the energy cost and ensuring comfort, this paper introduces an offline Deep Reinforcement Learning (DRL) algorithm which learns directly from pre-collected datasets, avoiding the cost and safety issues associated with continuous real-world interactions. The algorithm incorporates the Huber loss and a Q-quantile estimator to mitigate performance degradation from dataset anomalies such as data noise, sensor failure and human error, resulting in more robust HEMS optimization strategies. Experimental results demonstrate the method’s effectiveness in reducing total costs and analyze the performance of household devices with two different electricity rates.

The increasing penetration of distributed energy resources has prompted distribution system operators (DSOs) at the retail electricity market level to coordinate with the independent system operator (ISO) at the wholesale market level, for greater benefits. However, interaction mechanisms between the ISO and DSOs, and impacts of prices and power injections, have not been adequately investigated in literature. This article proposes a distributed coordination framework for the ISO and DSOs across wholesale-retail (bi-level) electricity markets, considering their interactions more fairly. Moreover, to mitigate the challenges arising from the interdependence between the ISO and heterogeneous DSOs, a coupled training mechanism based on the response model is devised. This mechanism iteratively trains the ISO and DSOs by solely exchanging prices and power injections, ensuring the demand–supply balance at both retail and wholesale levels. In addition, a deep reinforcement learning algorithm is introduced for the three-stage iterative training process of heterogeneous agents. Results demonstrate the effectiveness of the proposed method and its advantages in terms of lowering energy prices, clearing of cheaper clean resources and thus, improving overall market efficiency.

As the penetration of variable renewable generation increases in power systems, issues, such as grid stiffness, larger frequency deviations, and grid stability, are becoming more relevant, particularly in view of 100% renewable energy networks, which is the future of smart grids. In this context, energy storage systems (ESSs) are proving to be indispensable for facilitating the integration of renewable energy sources (RESs), are being widely deployed in both microgrids and bulk power systems, and thus will be the hallmark of the clean electrical grids of the future. Hence, this article reviews several energy storage technologies that are rapidly evolving to address the RES integration challenge, particularly compressed air energy storage (CAES), flywheels, batteries, and thermal ESSs, and their modeling and applications in power grids. An overview of these ESSs is provided, focusing on new models and applications in microgrids and distribution and transmission grids for grid operation, markets, stability, and control.

Battery energy storage systems (BESSs) can be very beneficial to power systems and microgrids for various applications. With increasing sales of electric vehicles (EV), the availability of used electric vehicle batteries (EVBs) is on the rise, which has received significant attention in recent years. The retired EVBs, after repurposing, can serve as an alternative option to new batteries in a BESS. In addition, when a microgrid operator desires to install a BESS, the optimal decisions such as installation year, energy and power size, replacement year, and the number of cycles to failure, that are normally overlooked, have to be determined. Therefore, this paper proposes a comprehensive and novel framework for planning and operation of the BESS based on repurposed EVBs. A novel linearized BESS sizing model is proposed that obtains the BESS optimal decisions regarding design and operation. Various new, modified, and linearized relationships for the BESS have been included in the planning model to ensure that the replacement year of the BESS is optimally determined. Several classes of EVs with multiple drive cycles are clustered and integrated within the proposed framework.

This paper presents a mathematical model for optimal sizing of battery energy storage systems (BESS) in an unbalanced distribution system. The vanadium radox (VR) and sodium sulfur (NaS) based batteries are considered in this study. Detailed life-cycle cost analysis is carried out to first identify the long-term cost of the batteries, which is then included in the proposed model. Two case studies are carried out: first the effect of the local distribution company (LDC) savings on the battery size is examined, and second the optimal size of the battery is determined by solving a comprehensive optimization problem. The proposed model seeks to maximize the savings of the LDC accrued from BESS operation, i.e., maximize the difference of discharging and charging cost of the BESS, while the investment cost is minimized. The IEEE 13-node test feeder is used to test the proposed model.