Federated Deep Reinforcement Learning (FDRL) aims to enable distributed collaborative training of multiple DRL models while preserving privacy. Existing FDRL methods function in static client environments, but real-world scenarios often involve dynamic state transitions, such as noise, which render static model topologies inadequate and result in biased policy loss. This degrades client performance and leads to suboptimal global policies. To address this challenge, we develop a generic solution, referred to as the self-regulating training framework, which can be seamlessly integrated into existing FDRL approaches to address dynamic state transitions. Specifically, we propose a Sparse Training (ST) method that dynamically sparsifies and adjusts the topology of each model during training to maximize model performance and reduce model complexity. Additionally, we introduce an auxiliary model to adaptively regulate the policy loss of client models, mitigating loss bias and facilitating updates that yield improved returns. Experimental results demonstrate that our method enhances six state-of-the-art (SOTA) FDRL approaches across nine tasks in terms of return.

This article addresses the hybrid model predictive control of chiller systems via collaborative neurodynamic optimization. A mixed-integer optimization problem is formulated for the model predictive control of chiller systems to minimize power consumption, subject to various constraints including thermodynamic and energy-conservation constraints. It is then decomposed into a global and a binary optimization subproblem. A collaborative neurodynamic optimization approach is proposed to solve the subproblems sequentially. The approach is based on multiple pairs of projection neural networks and discrete Hopfield networks, assisted with a metaheuristic rule. The effectiveness of the approach is demonstrated through experiments based on the parameters and specifications of a chiller system.

In this article, we present a collaborative neurodynamic optimization approach to distributed chiller loading in the presence of nonconvex power consumption functions and binary variables associated with cardinality constraints. We formulate a cardinality-constrained distributed optimization problem with nonconvex objective functions and discrete feasible regions, based on an augmented Lagrangian function. To overcome the difficulty caused by the nonconvexity in the formulated distributed optimization problem, we develop a collaborative neurodynamic optimization method based on multiple coupled recurrent neural networks reinitialized repeatedly using a meta-heuristic rule. We elaborate on experimental results based on two multi-chiller systems with the parameters from the chiller manufacturers to demonstrate the efficacy of the proposed approach in comparison to several baselines.

In the operation planning of heating, ventilation, and air conditioning systems, optimal chiller loading assigns cooling loads to chillers with minimized power consumption. In this article, a mixed-integer optimization problem is formulated for distributed chiller loading and is then decomposed into two optimization subproblems with binary and continuous variables. A collaborative neurodynamic optimization approach is proposed for distributed chiller loading by solving the formulated subproblems. In the collaborative neurodynamic optimization framework, multiple projection neural networks and discrete Hopfield networks are used for scattered searches and a metaheuristic rule is adopted for reinitializing neuronal states upon their local convergence. Experimental results based on the specifications and parameters of three actual chiller systems are elaborated to substantiate the high performance of the approach.

Chillers are indispensable machines for heat removal and the primary sources of power consumption in heating, ventilation, and air conditioning systems. In this paper, a cardinality-constrained global optimization problem is formulated to minimize power consumption for optimal chiller loading. The formulated problem is solved using a collaborative neurodynamic optimization method based on multiple neurodynamic models. Experimental results based on available actual chiller parameters are elaborated to demonstrate the superiority of the proposed approach to many baseline methods for optimal chiller loading.

This article addresses event-triggered optimal load dispatching based on collaborative neurodynamic optimization. Two cardinality-constrained global optimization problems are formulated and two event-triggering functions are defined for event-triggered load dispatching in thermal energy and electric power systems. An event-triggered dispatching method is developed in the collaborative neurodynamic optimization framework with multiple projection neural networks and a meta-heuristic updating rule. Experimental results are elaborated to demonstrate the efficacy and superiority of the approach against many existing methods for optimal load dispatching in air conditioning systems and electric power generation systems.

A chiller plant is an essential part of a heating, ventilation, and air conditioning system. Chiller plant operation planning is to determine the throughput of active chillers, pumps, and fans in a chiller plant to meet cooling load demands with minimized power consumption. Existing planning methods are limited to chiller plant operation with homogeneous devices subject to constraints for the conservation of energy or with heterogeneous devices without considering the conservation of energy. In this article, a mixed-integer optimization problem is formulated for chiller plant operation planning with heterogeneous devices to minimize power consumption subject to various constraints, including the constraints for the conservation of energy. The formulated problem is reformulated as a global optimization problem and solved via collaborative neurodynamic optimization with multiple projection neural networks. Experimental results based on equipment manufacturers’ specifications are elaborated to demonstrate the significantly higher performance of the proposed approach than four mainstream methods in terms of power consumption wattage.