This paper addresses the sustainable and regulation-compliant control of wastewater treatment plants (WWTPs) characterized by nonlinear process dynamics, external disturbances, and multiple, often competing, effluent-quality constraints (e.g., biochemical oxygen demand, ammonia, nitrate, and phosphorus limits). We propose an Adaptive Lagrangian Soft Actor–Critic (AL-SAC) algorithm that incorporates a Lagrangian relaxation term into the maximum-entropy SAC objective in order to impose hard bounds on these key effluent variables. Lagrange multipliers are updated online based on instantaneous constraint violations, eliminating manual penalty tuning and enhancing convergence stability. AL-SAC algorithm is evaluated on the Benchmark Simulation Model No.1 (BSM1) and is found to provide up to a 26% reduction in energy consumption (i.e., aeration and pumping), improve the effluent quality index (EQI) by 3.94%, and significantly reduce the period over which total nitrogen (TN) and ammonia (SNH) concentrations exceed regulatory thresholds. These results demonstrate AL-SAC’s promise for energy-efficient and fully compliant WWTP operation.

Communication is crucial in multi-agent reinforcement learning when agents are not able to observe the full state of the environment. The most common approach to allow learned communication between agents is the use of a differentiable communication channel that allows gradients to flow between agents as a form of feedback. However, this is challenging when we want to use discrete messages to reduce the message size, since gradients cannot flow through a discrete communication channel. Previous work proposed methods to deal with this problem. However, these methods are tested in different communication learning architectures and environments, making it hard to compare them. In this paper, we compare several state-of-the-art discretization methods as well as a novel approach. We do this comparison in the context of communication learning using gradients from other agents and perform tests on several environments. In addition, we present COMA-DIAL, a communication learning approach based on DIAL and COMA extended with learning rate scaling and adapted exploration. COMA-DIAL uses COMA to learn the action policy while it uses the mechanism introduced in DIAL to learn the communication policy. Using COMA-DIAL allows us to perform experiments on more complex environments. Our results show that the novel ST-DRU method, proposed in this paper, achieves the best results out of all discretization methods across the different environments. It achieves the best or close to the best performance in each of the experiments and is the only method that does not fail on any of the tested environments.

Hyperdimensional Computing (HDC) is an emerging AI algorithm, touted to be an efficient, neuro-inspired and reliable alternative to neural networks for Edge AI. HDC utilizes hypervectors with several thousand elements; the number of elements in these hypervectors denotes the HDC dimension. This dimension can be optimized for improving the efficiency and reliability of HDC inference against errors such as bit-flips, which can be caused by environmental radiation-induced soft errors. We hypothesize that, by reducing the runtime chip area and execution time utilized by HDC inference through lowering dimensionality, both efficiency and reliability against soft error-induced bit-flips can be simultaneously improved while trading off a negligible amount of accuracy and error threshold. We tested our hypothesis by executing an HDC inference algorithm with two different dimension values, 10000 (10k) and 1024, on a commercially available, low-power, bare-metal ARM platform with a Cortex-M4 processor. We conducted the efficiency analysis by measuring the CPU cycles and energy required for executing the algorithm, and the reliability analysis using real-world atmospheric-like neutron radiation from the ChipIr facility in Oxfordshire, UK. Analyses revealed that, by lowering the HDC dimension from 10k to 1024, the reliability of HDC inference against soft error-induced bit-flips was 3.5 times better and efficiency improved by more than 16 times. This innovative observation contrasts the prevailing understanding in the community that increasing the HDC dimension always improves robustness or reliability. To the best of our knowledge, our work is the first to study the reliability of HDC inference using real-world radiation.

Simulating fluid dynamics is challenging due to the computational complexity of processing high-dimensional data, which often requires significant time.Fluid behavior is typically governed by partial differential equations (PDEs), and the complexity escalates when obstacles disrupt the flow, reinforcing vorticity formation.Vorticity describes the local rotational motion of a fluid.In this paper, we present a data-driven approach to automate PDE simulations and develop surrogate models to generate fluid dynamics based on the Kármán vortex street.Our approach aims to generate accurate fluid simulations with faster computation through architectural adjustments.

The Metal-Oxide-Semiconductor Field-Effect Transistor (MOSFET) is a fundamental component in modern electronics, playing a vital role in the amplification and switching of signals within digital circuits. It regulates current flow in response to applied voltage. MOSFETs consist of four terminals: Gate (IG), Drain (ID), Source (IS), and Background or Body (IB). Nevertheless, during the design and manufacturing processes, certain devices are classified as non-functional due to specific abnormalities in their terminals. Identifying the nature of these defects during production can significantly enhance quality control by enabling more accurate detection and correction of faults. This study investigates the use of Artificial Intelligence (AI) and data-driven approaches by applying unsupervised learning techniques to classify the characteristics of faulty MOSFETs. Unsupervised learning enables the analysis of large datasets to examine abnormal devices and distinguish between them based on their distinct properties. By integrating AI-driven diagnostics into the production pipeline, our objective is to establish a system that not only improves yield and operational efficiency but also enhances the reliability of MOSFETs in end-use applications. This approach aims to establish a new standard for precision in quality control semiconductor manufacturing.

Abstract We assess the feasibility of a reusable neural architecture search agent aimed at amortizing the initial time-investment in building a good search strategy. We do this through the use of Reinforcement Learning, where an agent learns to iteratively select the best way to modify a given neural network architecture. This is achieved using a transformer-based agent design trained using the Ape-X algorithm. We consider both the NAS-Bench-101 and NAS-Bench-301 settings, and compare against various known strong baselines, such as local search and random search. While achieving competitive performance on both benchmarks, the amount of training required for the much larger NAS-Bench-301 is only marginally greater than NAS-Bench-101, illustrating the strong scaling properties of our agent. Our agent is able to achieve strong performance, but the choice of values for certain parameters are crucial to ensuring the succesful training of the agent. We provide some guidance for the selection of appropriate values for hyperparameters through a detailed description of our experimental setup and several ablation studies.

Learning to communicate in order to share state information is an active problem in the area of multi-agent reinforcement learning. The credit assignment problem, the non-stationarity of the communication environment and the problem of encouraging the agents to be influenced by incoming messages are major challenges within this research field which need to be overcome in order to learn a valid communication protocol. This paper introduces the novel multi-agent counterfactual communication learning (MACC) method which adapts counterfactual reasoning in order to overcome the credit assignment problem for communicating agents. Next, the non-stationarity of the communication environment, while learning the communication Q -function, is overcome by creating the communication Q -function using the action policy of the other agents and the Q -function of the action environment. As the exact method to create the communication Q -function can be computationally intensive for a large number of agents, two approximation methods are proposed. Additionally, a social loss function is introduced in order to create influenceable agents, which is required to learn a valid communication protocol. Our experiments show that MACC is able to outperform the state-of-the-art baselines in four different scenarios in the particle environment. Finally, we demonstrate the scalability of MACC in a matrix environment.

The conventional approach for controlling the supply temperature in collective space heating networks relies on a predefined heating curve determined by outdoor temperature and heat emitter type. This prioritises thermal comfort but lacks energetic and financial optimisation. This research proposes an adaptive supply temperature control in well-insulated dwellings, responsive to diverse environmental parameters. The approach considers variable electricity prices and accommodates different indoor temperature set points in dwellings. The study evaluates the effectiveness of two Deep Reinforcement Learning (DRL) algorithms, i.e., Proximal Policy Optimisation (PPO) and Deep Q-Network (DQN), across various scenarios. Results reveal that DQN excels in collective space heating systems with underfloor heating in each dwelling, while PPO proves superior for radiator-based systems. Both outperform the traditional heating curve, achieving up to 13.77% (DQN) and 16.15% (PPO) cost reduction while guaranteeing thermal comfort. Additionally, the research highlights the capability of DRL-based methods to dynamically set the supply temperature based on a cloud of set points, showcasing adaptability to diverse environmental factors and addressing the growing significance of indoor heat gains in well-insulated dwellings. This innovative approach holds promise for more efficient and environmentally conscious heating strategies within collective space heating networks.

A reliable public transport system is critical in promoting its preference over private vehicles, subsequently leading to a reduction in CO2 emissions and air pollution. A public transport forecasting model enhances travel schedule efficiency, trip planning and therefore service reliability. Due to the large volume of trajectory data and high computational demands, developing scalable forecasting solutions is valuable. In the context of the TANGENT H2020 project, data-driven methodologies are developed for real-time supply prediction problems. This paper focuses on forecasting solutions for the public transport system for the case study of Rennes. It presents a pipeline for bus travel time prediction which is both accurate and computationally efficient, and which can deal with missing trajectories by defining square grids and a state-of-the-art forecasting model. The results show that as the grid size is incrementally increased, improvements in predictive accuracy are observed up to an optimal point where the accuracy goes beyond state-of-the-art.

Maintaining roads is crucial to economic growth and citizen well-being because roads are a vital means of transportation. In various countries, the inspection of road surfaces is still done manually, however, to automate it, research interest is now focused on detecting the road surface defects via the visual data. While, previous research has been focused on deep learning methods which tend to process the entire image and leads to heavy computational cost. In this study, we focus our attention on improving the classification performance while keeping the computational cost of our solution low. Instead of processing the whole image, we introduce a segmentation model to only focus the downstream classification model to the road surface in the image. Furthermore, we employ contrastive learning during model training to improve the road surface condition classification. Our experiments on the public RTK dataset demonstrate a significant improvement in our proposed method when compared to previous works.

In the field of autonomous driving, collaborative perception has emerged as a promising solution for augmenting the capabilities of individual sensors by enabling vehicles to share their sensor information across each other, thereby enhancing their situational awareness. This paper addresses the limitations of classical perception in autonomous vehicles by proposing a novel intermediate collaborative perception methodology employing graph attention network (GAT) to incorporate multiple feature maps and to selectively emphasize important regions within the feature maps. We construct the graph structure as a set of nodes embedding the ego and the neighboring connected vehicles feature maps, as well as establish edge weights between those nodes based on their relationship to each other which is defined by the attention coefficients. The proposed approach leverages both channel and spatial attention-based aggregation and enables the model to determine inter-feature map relationships at a specific channel and spatial regions, while adaptively highlighting the informative regions. This adaptive highlighting mechanism directs the aggregation algorithm towards the most informative areas within the ego and the received feature maps, thereby enhancing the representation power of the ego vehicle’s feature map leading to improved precision in object detection. We quantitatively and qualitatively evaluate the performance of our proposed approach against existing state-of-the-art in collaborative perception. We validate our methodology using V2XSim, a large-scale multi-agent perception dataset. The results demonstrate that our methodology achieves superior performance in enhancing object detection average precision.

As the need for stand-alone energy harvesting devices increases, the alleviation of the ecological and economic impact of their production and maintenance is possible by increasing battery life while reducing needed battery capacity.However, the increasing energy requirements of far-edge Artificial Intelligence and long-range wireless transmissions in the Internet of Things threaten to demand ever-larger battery capacities for such remote devices.Dynamic adaptation of device operation based on harvestable energy -i.e., energy awarenessis a proposed solution and can be implemented using an energy management unit.Standardizing this unit as a separate, active electronic component with standardized drivers can simplify overall system development and benefit existing devices.Hence, we propose a novel interface that allows decoupling this unit from the rest of the system, independent of the power management unit in use.As a first step, we developed a prototype that uses the proposed interface to make existing, solar energy-based, third-party devices energy-aware with provisions to be cross-compatible with differing power management units.The prototype was evaluated using an air quality sensing device and improved the overall device's transmission rate.

Indoor temperature modeling has been a vital component to develop accurate digital twins and smart controllers for buildings. Hybrid (also known as gray-box) modeling caught significant attention from the literature for this task. Combining the accumulated physical knowledge we have about thermal behavior with modern data-driven techniques promises more accurate and stable prediction models which can be used in various applications. However, methods such as data-driven parameter optimization and constrained training proposed in the literature show practical limitations such as high computational expense and software incompatibilities. In this paper we propose a transfer learning-based hybrid modeling approach where a CNN-LSTM model is pre-trained with the simulation data and then refined with the real-life data, thus, creating a completely data-driven hybrid model. We compared our approach to the same CNN-LSTM architecture trained only on real-life data. We reported significant accuracy and stability increases with the proposed approach.

Combined heat distribution circuits (CHDCs) are increasingly used in apartment buildings. Here only one supply pipe distributes both space heating (SH) and domestic hot water (DHW). Currently, the supply temperature is set to the highest temperature needed by one of the end-users (i.e. 65ºC for DHW), even if low-temperature emitters are used for SH. However, using decentral storage tanks for DHW enable demand-based temperature controls to reduce unnecessary heat losses and poor efficiencies. This research uses reinforcement learning (RL), a machine learning technique, to develop new control strategies for CHDCs with underfloor heating and DHW storage tanks. The agent controls the supply temperature and the mass flow in the hybrid boiler room. Whether the RL agent is able to find the optimal control strategy depends on the definition of its Markov Decision Process (MDP) model elements, namely the states, the possible control actions and the reward function. The results show that an increasing gamma and decreasing learning rate during training leads to better performance and that the agent with the largest flexibility develops a better control strategy that resulted in up to 23% primary energy savings.

In recent years, interest in autonomous shipping in urban waterways has increased significantly due to the trend of keeping cars and trucks out of city centers. Classical approaches such as Frenet frame based planning and potential field navigation often require tuning of many configuration parameters and sometimes even require a different configuration depending on the situation. In this paper, we propose a novel path planning approach based on reinforcement learning called Model Predictive Reinforcement Learning (MPRL). MPRL calculates a series of waypoints for the vessel to follow. The environment is represented as an occupancy grid map, allowing us to deal with any shape of waterway and any number and shape of obstacles. We demonstrate our approach on two scenarios and compare the resulting path with path planning using a Frenet frame and path planning based on a proximal policy optimization (PPO) agent. Our results show that MPRL outperforms both baselines in both test scenarios. The PPO based approach was not able to reach the goal in either scenario while the Frenet frame approach failed in the scenario consisting of a corner with obstacles. MPRL was able to safely (collision free) navigate to the goal in both of the test scenarios.

Convolutional neural networks (CNNs) have accomplished magnificent performance on object classification tasks. This work introduces a novel image classification approach based on feature vector fusion of two CNN architectures using graph attention networks (GAT). In the proposed method we extract feature maps from shallow and deep layers of two CNN architectures. These extracted feature vectors are represented as nodes in a graph, and edges between nodes are constructed based on similarities between the feature vectors. The GAT is used to aggregate and fuse connected nodes based on their importance and relevance to the classification task. We believe that this approach compensates for convolution defects during feature processing when using a single CNN. This paper attempts to show that graph-based deep learning can be used to fuse two CNN architectures and not to push the state-of-the-art of image classification accuracy. Our experimental results prove that GAT can be used to fuse feature vectors resulting from CNN architectures.

Bad weather conditions such as heavy rain, black ice and fog can have a significant impact on road safety. Currently vehicle safety technologies such as the electronic stability program work reactive to hazardous situations. In this paper, we propose the use of crowd-sourced vehicle data to improve road-weather models and provide real-time local warnings for weather-related hazards. We present our initial results from a field test where we used vehicle CAN-bus data and low cost external sensors to observe local weather phenomena. The CAN-bus contains, among others, data on vehicle dynamics such as wheel speeds. Our approach is to isolate anomalies within these signals. Our initial research suggests some anomalies are weather related and can be used to describe local weather phenomena. Furthermore, the externally installed sensors provide more information on which we can build our assumptions. The results show that the gathered measurements are consistent with the reliable observations from road weather stations.