In recent years, black-box distributed optimization (DBO) has been widely studied to solve complex optimization problems in multi-agent systems, such as hyperparameter optimization of distributed machine learning. However, most existing methods use a fixed or diminishing step size to sample and search in the black box optimization space, which makes it challenging to maintain optimization efficiency on different optimization problems. In this work, we propose a multi-agent evolution strategy with cooperative and cumulative step adaptation (). In, each agent executes the algorithm to sample and explores its local objective function, and communicates with other agents to optimize the global objective function cooperatively, which is the sum of local objective functions. To improve the sampling adaptability, we design a cooperative and cumulative step adaptation method (CCSA) consisting of inner adaptation and outer adaptation. By detecting the evolution path of the multi-agent system, CCSA decreases the step size when the evolution directions of agents are conflicting and increases the step size when consistent. In terms of theoretical analysis, we first discuss the working principle of CCSA, and then discuss the system consensus of . In terms of experimental verification, achieves better consensus performance and competitive solution quality compared with state-of-the-art algorithms for DBO.

Distributed optimization has attracted lots of attention in recent years. Thanks to the intrinsic parallelism and great search capacity, evolutionary computation (EC) has the potential for black-box and non-convex distributed optimization. However, due to the decentralization of local objective functions, it is challenging to optimize the global objective function with efficient communication and guaranteed system consensus. To tackle this challenge, we propose a Multi-Agent Swarm Optimization method with adaptive Internal and External learning (MASOIE). In MASOIE, each agent evolves a swarm of particles by internal learning and external learning. Internal learning enables agents to optimize their local objectives, while external learning enables agents to cooperate to achieve a consensus toward the global objective. To improve the consensus ability, we design a special velocity setting of external learning for particle evolution. We provide the theoretical analysis of the system consensus of deterministic MASOIE. To improve communication efficiency, we design an adaptive communication mechanism to adjust the communication interval, enabling agents to explore at the early stage and reach system consensus at the later stage. Empirical studies show that the proposed algorithm achieves stable consensus performance, competitive solution quality and lower communication cost on benchmark functions compared with existing black-box distributed algorithms.

Traffic signal control (TSC) problems have received increasing attention with the development of the smart city. Reinforcement learning (RL) models TSC as a Markov decision process and learns the timing relationship of traffic scheduling from massive historical data. Due to the uncertainty and mutability of TSC problems, existing RL methods face bottlenecks in diversity and are easy to be trapped into local optima. To alleviate this predicament, this paper combines evolutionary optimization and RL to propose an evolutionary algorithm-assisted reinforcement learning (EARL-Light) method for TSC problems. EARL-Light is a population-based algorithm, in which one individual represents a policy and a population of individuals are evolved to search for near-optimal policies. The diversified search ability of evolutionary optimization can help the algorithm get rid of local optima for global optimization and the rapid learning based on the gradient of RL can achieve fast convergence. Extensive experiments on seven real-world traffic datasets demonstrates that EARL-Light achieves shorter travel time with fast convergence.

The emergence of networked systems in various fields brings many complex distributed optimization problems, where multiple agents in the system need to optimize a global objective cooperatively when they only have local information. In this work, we take advantage of the intrinsic parallelism of evolutionary computation to address network-based distributed optimization. In the proposed multiagent co-evolutionary algorithm, each agent maintains a subpopulation in which individuals represent solutions to the problem. During optimization, agents perform local optimization on their subpopulations and negotiation through communication with their neighbors. In order to help agents optimize the global objective cooperatively, we design a penalty-based objective function for fitness evaluation, which constrains the subpopulation within a small and controllable range. Further, to make the penalty more targeted, a conflict detection method is proposed to examine whether agents are conflicting on a certain shared variable. Finally, in order to help agents negotiate a consensus solution when only the local objective function is known, we retrofit the processes of negotiating shared variables, namely, evaluation, competition, and sharing. The above approaches form a multiagent co-evolutionary framework, enabling agents to cooperatively optimize the global objective in a distributed manner. Empirical studies show that the proposed algorithm achieves comparable solution quality with the holistic algorithm and better performance than existing gradient-free distributed algorithms on gradient-uncomputable problems.

Target tracking and path planning using unmanned aerial vehicles (UAVs) have attracted increasing research attention in recent years. The rapid development of communication technology enables the use of multiple UAVs to perform target tracking collaboratively. But it remains challenging to coordinate multiple UAVs in some complicated scenarios, e.g., tracking multiple targets using multiple UAVs. In this paper, we intend to propose a Nash-based evolutionary dynamic optimization algorithm for multi-target tracking using multiple UAVs. Firstly, considering the requirement of balancing the number of UAVs tracking each target, we formulate the tracking problem as a distributed constrained multi-objective dynamic optimization problem using model predictive control (MPC). Secondly, to better track dynamic targets with stochastic behaviors, we design an evolutionary dynamic optimization (EDO) approach to solve the optimization problem. Thirdly, in order to avoid collisions, we combine the EDO approach with Nash optimization. The experimental results show that our approach has better performance than compared algorithms.