Optimal control of complex systems often requires access to a high-fidelity model, and information about the (future) external stimuli applied to the system (load, demand, ...).An example of such a system is a cooling network, in which one or more chillers provide cooled liquid to a set of users with a variable demand.In this paper, we propose a Reinforcement Learning (RL) method for such a system with 3 chillers.It does not assume any model, and does not observe the future cooling demand, nor approximations of it.Still, we show that, after a training phase in a simulator, the learned controller achieves a performance better than classical rule-based controllers, and similar to a model predictive controller that does rely on a model and demand predictions.We show that the RL algorithm has learned implicitly how to anticipate, without requiring explicit predictions.This demonstrates that RL can allow to produce high-quality controllers in challenging industrial contexts.

Communication plays a vital role in multi-agent systems, fostering collaboration and coordination. However, in real-world scenarios where communication is bandwidth-limited, existing multi-agent reinforcement learning (MARL) algorithms often provide agents with a binary choice: either transmitting a fixed amount of data or no information at all. This rigid communication strategy hinders the ability to effectively utilize bandwidth. To overcome this challenge, we present the Dynamic Size Message Scheduling (DSMS) method, which introduces finer-grained communication scheduling by considering the actual size of the information being exchanged. Our approach lies in adapting message sizes using Fourier transform-based compression techniques with clipping, enabling agents to tailor their messages to match the allocated bandwidth according to importance weights. This method realizes a balance between information loss and bandwidth utilization. Receiving agents reliably decompress the messages using the inverse Fourier transform. We evaluate DSMS in cooperative tasks where the agent has partial observability. Experimental results demonstrate that DSMS significantly improves performance by optimizing the utilization of bandwidth and effectively balancing information importance.

Many real-world decision problems are characterized by multiple conflicting objectives which must be balanced based on their relative importance. In the dynamic weights setting the relative importance changes over time and specialized algorithms that deal with such change, such as a tabular Reinforcement Learning (RL) algorithm by Natarajan and Tadepalli (2005), are required. However, this earlier work is not feasible for RL settings that necessitate the use of function approximators. We generalize across weight changes and high-dimensional inputs by proposing a multi-objective Q-network whose outputs are conditioned on the relative importance of objectives and we introduce Diverse Experience Replay (DER) to counter the inherent non-stationarity of the Dynamic Weights setting. We perform an extensive experimental evaluation and compare our methods to adapted algorithms from Deep Multi-Task/Multi-Objective Reinforcement Learning and show that our proposed network in combination with DER dominates these adapted algorithms across weight change scenarios and problem domains.

Many real-world reinforcement learning problems have a hierarchical nature, and often exhibit some degree of partial observability. While hierarchy and partial observability are usually tackled separately (for instance by combining recurrent neural networks and options), we show that addressing both problems simultaneously is simpler and more efficient in many cases. More specifically, we make the initiation set of options conditional on the previously-executed option, and show that options with such Option-Observation Initiation Sets (OOIs) are at least as expressive as Finite State Controllers (FSCs), a state-of-the-art approach for learning in POMDPs. OOIs are easy to design based on an intuitive description of the task, lead to explainable policies and keep the top-level and option policies memoryless. Our experiments show that OOIs allow agents to learn optimal policies in challenging POMDPs, while being much more sample-efficient than a recurrent neural network over options.