Multi-Agent Reinforcement Learning (MARL) algorithms are widely adopted in tackling complex tasks that require collaboration and competition among agents in dynamic Multi-Agent Systems (MAS). However, learning such tasks from scratch is arduous and may not always be feasible, particularly for MASs with a large number of interactive agents due to the extensive sample complexity. Therefore, reusing knowledge gained from past experiences or other agents could efficiently accelerate the learning process and upscale MARL algorithms. In this study, we introduce a novel framework that enables transfer learning for MARL through unifying various state spaces into fixed-size inputs that allow one unified deep-learning policy viable in different scenarios within a MAS. We evaluated our approach in a range of scenarios within the StarCraft Multi-Agent Challenge (SMAC) environment, and the findings show significant enhancements in multi-agent learning performance using maneuvering skills learned from other scenarios compared to agents learning from scratch. Furthermore, we adopted Curriculum Transfer Learning (CTL), enabling our deep learning policy to progressively acquire knowledge and skills across pre-designed homogeneous learning scenarios organized by difficulty levels. This process promotes inter- and intra-agent knowledge transfer, leading to high multi-agent learning performance in more complicated heterogeneous scenarios.

Distributed decision-making in multi-agent systems presents difficult challenges for interactive behavior learning in both cooperative and competitive systems. To mitigate this complexity, MAIDRL presents a semi-centralized Dense Reinforcement Learning algorithm enhanced by agent influence maps (AIMs), for learning effective multi-agent control on StarCraft Multi-Agent Challenge (SMAC) scenarios. In this paper, we extend the DenseNet in MAIDRL and introduce semi-centralized Multi-Agent Dense-CNN Reinforcement Learning, MAIDCRL, by incorporating convolutional layers into the deep model architecture, and evaluate the performance on both homogeneous and heterogeneous scenarios. The results show that the CNN-enabled MAIDCRL significantly improved the learning performance and achieved a faster learning rate compared to the existing MAIDRL, especially on more complicated heterogeneous SMAC scenarios. We further investigate the stability and robustness of our model. The statistics reflect that our model not only achieves higher winning rate in all the given scenarios but also boosts the agent’s learning process in fine-grained decision-making.

In recent years, reinforcement learning algorithms have been used in the field of multi-agent systems to help the agents with interactions and cooperation on a variety of tasks. Controlling multiple agents simultaneously is extremely challenging as the complexity increases drastically with the number of agents in the system. In this study, we propose a novel semi-centralized deep reinforcement learning algorithm, MAIDRL, for mixed cooperative and competitive multi-agent environments. Specifically, we design a robust DenseNet-style actor-critic structured deep neural network for controlling multiple agents based on the combination of local observation and abstracted global information to compete with opponent agents. We extract common knowledge through influence maps considering both enemy and friendly agents for unit positioning and decision-making in combat. Compared to the centralized method, our design promotes a thorough understanding of the potential influence that a unit has without the need for a complete view of the global state. In addition, this design enables multiagent understanding of common goals, unlike fully decentralized methods. The proposed method has been evaluated on StarCraft Multi-Agent Challenge scenarios in the real-time strategy game, StarCraft II, and the results show that, statistically, the agents controlled by MAIDRL perform better than or as well as those controlled by centralized and decentralized methods.

Commercial parking space occupancy detection systems used to be mostly sensor-based. Very recently, we have seen great success in computer vision techniques which allow us to utilize the CCTV camera feed in real-time. In this paper, we review multiple existing convolutional neural network models of various sizes to analyze the benefits of using each model for parking space occupancy detection. We measure the accuracy, required floating point operations, and parameter counts of each model. We then compare model performance over different conditions such as camera perspectives, weather types, different parking lots, and lighting conditions. Based on our observations and experience, we introduce three novel architectures, two based on the DenseNet architecture - Mini DenseNet and Simple DenseNet, and CoarseNet - a multi-layer perceptron. We compare our proposed models to other models of various sizes. Performance results show that these three models have advantages over existing models in parameter counts, accuracy, and resilience to new camera perspectives.

This paper describes a time-domain approach for estimating the state of health (SOH) and remaining useful life (RUL) of a battery, with particular emphasis on LiFePO4cells. In this methodology, a small test signal is superimposed on top of the battery load to trigger its transient dynamics. The resulting terminal voltage and current are measured, and a nonlinear least-squares routine is used to estimate the impedance parameters of the battery model. These parameter values are then used to track the SOH and RUL over time. Experimental results are presented. The approach requires minimal hardware and could be used to form the basis of a robust on-line monitoring system.