Reinforcement learning systems often face tasks requiring the simultaneous optimization of multiple conflicting objectives, where traditional single-policy methods fail to capture the diversity of Pareto-optimal trade-offs. This paper introduces MO-CoERL, a Multi-Objective Cooperative Evolutionary Deep Reinforcement Learning framework that integrates cooperative coevolution with actor–critic learning to address this challenge. The proposed method combines population-based evolutionary exploration with gradient-based policy refinement through a CAPQL backbone, while a global Pareto archive enables hypervolume-guided feedback and diversity maintenance. This cooperative mechanism allows MO-CoERL to achieve stable convergence, broad Pareto coverage, and improved generalization across objectives. Experiments on four continuous-control MuJoCo benchmarks (Hopper, Walker2d, Swimmer, and Ant) demonstrate that MO-CoERL outperforms CAPQL across most benchmarks in convergence speed and front quality. On average, it achieves +41.72% higher Expected Utility Metric (EUM) and a +66.89% improvement in Hypervolume (HV). Notably, MO-CoERL yields up to an 89.52% increase in HV on Hopper and +173.15% on Walker2d, highlighting its robustness in high-dimensional and unstable tasks. These results confirm that cooperative evolution effectively complements actor–critic learning, enhancing both policy diversity and Pareto convergence. MO-CoERL provides a scalable preference-conditioned MORL integrated with cooperative evolution, offering a robust foundation for advancing cooperative and population-based optimization frameworks. • Proposes MO-CoERL, a cooperative evolutionary multi-objective RL framework. • Integrates actor–critic learning with population-based evolutionary exploration. • Uses a global Pareto archive for hypervolume-guided selection and feedback. • Achieves on average +41.72% EUM and +66.89% Hypervolume gain over CAPQL on MuJoCo tasks. • Improves convergence speed, stability, and Pareto diversity across benchmarks.

This paper presents a novel framework for Peer-to-Peer (P2P) energy trading that integrates uncertainty-aware prediction with multi-agent reinforcement learning (MARL), addressing a critical gap in current literature. In contrast to previous works relying on deterministic forecasts, the proposed approach employs a heteroscedastic probabilistic transformer-based prediction model called Knowledge Transformer with Uncertainty (KTU) to explicitly quantify prediction uncertainty, which is essential for robust decision-making in the stochastic environment of P2P energy trading. The KTU model leverages domain-specific features and is trained with a custom loss function that ensures reliable probabilistic forecasts and confidence intervals for each prediction. Integrating these uncertainty-aware forecasts into the MARL framework enables agents to optimize trading strategies with a clear understanding of risk and variability. Experimental results show that the uncertainty-aware Deep Q-Network (DQN) reduces energy purchase costs by up to 5.7% without P2P trading and 3.2% with P2P trading, while increasing electricity sales revenue by 6.4% and 44.7%, respectively. Additionally, peak hour grid demand is reduced by 38.8% without P2P and 45.6% with P2P. These improvements are even more pronounced when P2P trading is enabled, highlighting the synergy between advanced forecasting and market mechanisms for resilient, economically efficient energy communities.

Multi-Objective Reinforcement Learning (MORL) poses significant challenges, primarily due to the necessity of balancing conflicting objectives—a limitation that traditional single-objective approaches fail to address. This paper introduces Multi-Objective Evolutionary Reinforcement Learning (MO-ERL), the first adaptation of Evolutionary Reinforcement Learning (ERL) specifically designed to address the complexities of the multi-objective domain effectively. MO-ERL integrates policy gradient-based reinforcement learning (RL), which optimizes expected utility, with evolutionary algorithms (EAs) that maintain diversity across the Pareto front. This combination leverages RL’s strength in exploitation and EAs’ proficiency in exploration, enabling MO-ERL to effectively navigate the trade-offs inherent in multi-objective optimization problems. Evaluation on multi-objective continuous control tasks using the MuJoCo physics engine demonstrates that MO-ERL outperforms state-of-the-art baselines, achieving up to 62.71% higher hypervolume and 196.28% greater expected utility. These results validate MO-ERL’s ability to balance solution diversity and optimality, setting a new benchmark for solving MORL tasks. • First extension of Evolutionary Reinforcement Learning to multi-objective domain. • Outperforms state-of-the-art (CAPQL, PCN) on multi-objective continuous control. • Up to 62.7% higher hypervolume, 196.3% utility on high-dimensional control tasks. • MO-GA hypervolume approximation reduces computational cost, maintaining performance. • Frequent MORL injections into MO-GA yield smoother learning and faster convergence.

Effective residential appliance scheduling is crucial for sustainable living. While multi-objective reinforcement learning (MORL) has proven effective in balancing user preferences in appliance scheduling, traditional MORL struggles with limited data in non-stationary residential settings characterized by renewable generation variations. Significant context shifts in the environment can invalidate previously learned policies. To address this, we extend state-of-the-art MORL algorithms with the meta-learning paradigm, enabling rapid, few-shot adaptation to shifting contexts. Additionally, we employ an auto-encoder (AE)-based unsupervised method to detect shifts in environmental context. We have also developed a residential energy environment to evaluate our method using real-world data from London residential settings. This study not only assesses the application of MORL in residential appliance scheduling but also underscores the effectiveness of meta-learning in energy management. Our top-performing method significantly surpasses the best baseline, while the trained model saves 3.28% on electricity bills, a 2.74% increase in user comfort, and a 5.9% improvement in expected utility. Additionally, it reduces the sparsity of solutions by 62.44%. Remarkably, these gains were accomplished using 96.71% less training data and 61.1% fewer training steps.

It has been shown that an agent can be trained with an adversarial policy which achieves high degrees of success against a state-of-the-art DRL victim despite taking unintuitive actions. This prompts the question: is this adversarial behaviour detectable through the observations of the victim alone? We find that widely used classification methods such as random forests are only able to achieve a maximum of ≈71% test set accuracy when classifying an agent for a single timestep. However, when the classifier inputs are treated as time-series data, test set classification accuracy is increased significantly to ≈98%. This is true for both classification of episodes as a whole, and for “live” classification at each timestep in an episode. These classifications can then be used to “react” to incoming attacks and increase the overall win rate against Adversarial opponents by approximately 17%. Classification of the victim’s own internal activations in response to the adversary is shown to achieve similarly impressive accuracy while also offering advantages like increased transferability to other domains.

Buildings are the highest consumers of electrical energy globally. Developing effective control strategies to enhance the operation of buildings to increase their energy efficiency has therefore become a very active area of research. This research proposes the use of evolutionary neural networks for the task of Heating Ventilation and Air Conditioning control. A neural network controller is trained using the exponential Natural Evolutionary Strategy algorithm is proposed. Its performance is bench-marked against multiple state of the art control strategies, including: a Genetic Algorithm trained neural network, the popular Deep Q-Network reinforcement learning algorithm and also a threshold based control policy. The results presented demonstrate that evolutionary neural networks are an effective strategy for reducing energy consumption while also minimizing thermal discomfort. The energy consumption and thermal discomfort achieved by evolutionary neural networks, are significantly lower than when using Deep Q-Network. This is particularly true for the Natural Evolutionary Strategy algorithm.

Collecting, analyzing and gaining insight from large volumes of data is now the norm in an ever increasing number of industries. Data analytics techniques, such as machine learning, are powerful tools used to analyze these large volumes of data. Synthetic data sets are routinely relied upon to train and develop such data analytics methods for several reasons: to generate larger data sets than are available, to generate diverse data sets, to preserve anonymity in data sets with sensitive information, etc. Processing, transmitting and storing data is a key issue faced when handling large data sets. This paper presents an “On the fly” framework for generating big synthetic data sets, suitable for these data analytics methods, that is both computationally efficient and applicable to a diverse set of problems. An example application of the proposed framework is presented along with a mathematical analysis of its computational efficiency, demonstrating its effectiveness. Empirical results indicate that the proposed data generation framework provides a reduction in computational time of ≈ 33% when compared to the alternative approach of generating the data set in full.