We aim to develop a general multi-agent reinforcement learning (MARL) policy that enables a group of robots to efficiently explore large-scale, unknown environments with random pose initialization. Existing MARL-based multi-robot exploration methods face challenges in reliably mapping observations to actions in large-scale scenarios and lack of zero-shot generalization to unknown environments. To this end, we propose a generic multi-task pre-training algorithm (termed TaskExp) to enhance the generalization of learning-based policies. In particular, we design a decision-related task to guide the policy to focus on valuable subspaces of the action space, improving the reliability of policy mapping. Moreover, two perception-related tasks-Location Estimation and Map Prediction-are designed to enhance the zero-shot capability of the policy by guiding it to extract general invariant features from unknown environments. With TaskExp pre-training, our policy significantly outperforms state-of-the-art planning-based methods in large-scale scenarios and demonstrates strong zero-shot performance in unseen environments. Furthermore, TaskExp can also be easily integrated to improve the existing learning-based multi-robot exploration methods.

Most existing decentralized learning methods with differential privacy (DP) guarantee rely on constant gradient clipping bounds and fixed-level DP Gaussian noises for each node throughout the training process, leading to a significant accuracy degradation compared to non-private counterparts. In this paper, we propose a new Dynamic Differentially Private Decentralized learning approach (termed Dyn-D^2P) tailored for general time-varying directed networks. Leveraging the Gaussian DP (GDP) framework for privacy accounting, Dyn-D^2P dynamically adjusts gradient clipping bounds and noise levels based on gradient convergence. This proposed dynamic noise strategy enables us to enhance model accuracy while preserving the total privacy budget. Extensive experiments on benchmark datasets demonstrate the superiority of Dyn-D^2P over its counterparts employing fixed-level noises, especially under strong privacy guarantees. Furthermore, we provide a provable utility bound for Dyn-D^2P that establishes an explicit dependency on network-related parameters, with a scaling factor of 1/sqrt{n} in terms of the number of nodes n up to a bias error term induced by gradient clipping. To our knowledge, this is the first model utility analysis for differentially private decentralized non-convex optimization with dynamic gradient clipping bounds and noise levels.

Collaborative learning enhances the performance and adaptability of multi-robot systems in complex tasks but faces significant challenges due to high communication overhead and data heterogeneity inherent in multi-robot tasks. To this end, we propose CoCoL, a Communication efficient decentralized Collaborative Learning method tailored for multi-robot systems with heterogeneous local datasets. Leveraging a mirror descent framework, CoCoL achieves remarkable communication efficiency with approximate Newton-type updates by capturing the similarity between objective functions of robots, and reduces computational costs through inexact sub-problem solutions. Furthermore, the integration of a gradient tracking scheme ensures its robustness against data heterogeneity. Experimental results on three representative multi-robot collaborative learning tasks show that the proposed CoCoL can significantly reduce both the number of communication rounds and total bandwidth consumption while maintaining state-of-the-art accuracy. These benefits are particularly evident in challenging scenarios involving non-IID (non-independent and identically distributed) data distribution, streaming data, and time-varying network topologies.

This paper considers solving a class of nonconvex-strongly-convex distributed stochastic bilevel optimization (DSBO) problems with personalized inner-level objectives. Most existing algorithms require computational loops for hypergradient estimation, leading to computational inefficiency. Moreover, the impact of data heterogeneity on convergence in bilevel problems is not explicitly characterized yet. To address these issues, we propose LoPA, a loopless personalized distributed algorithm that leverages a tracking mechanism for iterative approximation of inner-level solutions and Hessian-inverse matrices without relying on extra computation loops. Our theoretical analysis explicitly characterizes the heterogeneity across nodes (denoted by $b$), and establishes a sublinear rate of $\mathcal{O}( {\frac{1}{{{{\left( {1 - \rho } \right)}}K}}\!+ \!\frac{{(\frac{b}{\sqrt{m}})^{\frac{2}{3}} }}{{\left( {1 - \rho } \right)^{\frac{2}{3}} K^{\frac{2}{3}} }} \!+ \!\frac{1}{\sqrt{ K }}( {\sigma _{\operatorname{p} }} + \frac{1}{\sqrt{m}}{\sigma _{\operatorname{c} }} ) } )$ without the boundedness of local hypergradients, where ${\sigma _{\operatorname{p} }}$ and ${\sigma _{\operatorname{c} }}$ represent the gradient sampling variances associated with the inner- and outer-level variables, respectively. We also integrate LoPA with a gradient tracking scheme to eliminate the impact of data heterogeneity, yielding an improved rate of ${{\mathcal{O}}}(\frac{{1}}{{ (1-\rho)^2K }} \!+\! \frac{1}{{\sqrt{K}}}( \sigma_{\rm{p}} \!+\! \frac{1}{\sqrt{m}}\sigma_{\rm{c}} ) )$. The computational complexity of LoPA is of ${{\mathcal{O}}}({\epsilon^{-2}})$ to an $\epsilon$-stationary point, matching the communication complexity due to the loopless structure, which outperforms existing counterparts for DSBO. Numerical experiments validate the effectiveness of the proposed algorithm.

The sim-to-real gap poses a significant challenge in RL-based multi-agent exploration due to scene quantization and action discretization. Existing platforms suffer from the inefficiency in sampling and the lack of diversity in Multi-Agent Reinforcement Learning (MARL) algorithms across different scenarios, restraining their widespread applications. To fill these gaps, we propose MAexp, a generic platform for multi-agent exploration that integrates a broad range of state-of-the-art MARL algorithms and representative scenarios. Moreover, we employ point clouds to represent our exploration scenarios, leading to high-fidelity environment mapping and a sampling speed approximately 40 times faster than existing platforms. Furthermore, equipped with an attention-based Multi-Agent Target Generator and a Single-Agent Motion Planner, MAexp can work with arbitrary numbers of agents and accommodate various types of robots. Extensive experiments are conducted to establish the first benchmark featuring several high-performance MARL algorithms across typical scenarios for robots with continuous actions, which highlights the distinct strengths of each algorithm in different scenarios.

We propose a novel hierarchical priority mechanism for deadlock recovery of distributed swarm via on-demand collision avoidance in cluttered dynamic environments. The proposed priority mechanism dynamically assigns certain priority and an optimized detour point for each agent based on its spatial context to avoid deadlocks which are predicted by properly designed deadlock conditions; as a byproduct, this priority mechanism allows us to effectively resolve livelocks as well. The resulting optimization problem is then solved by polar reformulation and alternating minimization methods. Simulation results demonstrate that, in both static and dynamic environments, our method (termed PriDRAM) outperforms the baseline Alternating Minimization Swarm (AMSwarm) method which does not explicitly account for deadlock recovery, with a 10.5% improvement in average smoothness and a 4.8% reduction in flight time. Moreover, for narrow passages, our method shows a superior performance against the Distributed Linear Safe Corridor (DLSC) method, with a more reasonable passing order and an achievement of up to 40% reduction in flight path length. Finally, we verify the efficacy of our proposed method with a Crazyflie 2.1 quadrotor swarm.

In this paper, we show that applying adaptive methods directly to distributed minimax problems can result in non-convergence due to inconsistency in locally computed adaptive stepsizes. To address this challenge, we propose D-AdaST, a Distributed Adaptive minimax method with Stepsize Tracking. The key strategy is to employ an adaptive stepsize tracking protocol involving the transmission of two extra (scalar) variables. This protocol ensures the consistency among stepsizes of nodes, eliminating the steady-state error due to the lack of coordination of stepsizes among nodes that commonly exists in vanilla distributed adaptive methods, and thus guarantees exact convergence. For nonconvex-strongly-concave distributed minimax problems, we characterize the specific transient times that ensure time-scale separation of stepsizes and quasi-independence of networks, leading to a near-optimal convergence rate of $\tilde{\mathcal{O}} \left( \epsilon ^{-\left( 4+\delta \right)} \right)$ for any small $\delta > 0$, matching that of the centralized counterpart. To our best knowledge, D-AdaST is the *first* distributed adaptive method achieving near-optimal convergence without knowing any problem-dependent parameters for nonconvex minimax problems. Extensive experiments are conducted to validate our theoretical results.

Since the scaling of advanced technology nodes is pushing to its physical limit, lithography hotspot detection (LHD) has become more significant than ever in design for manufacturability. Recently, machine learning techniques have been deployed to greatly reduce simulation time for hotspot detection, but high-quality data are required to build a model. Many design companies do not have enough high-quality data and are hesitant to share it for fear of intellectual property theft or model ineffectiveness. Furthermore, using locally trained models with limited and similar data can lead to overfitting and a lack of generalization and robustness when applied to new designs. In this article, we propose a heterogeneous federated learning framework for LHD that can address the aforementioned issues. Our framework can overcome the challenges of nonindependent and identically distributed data and heterogeneous communication, ensuring high performance and good convergence in various scenarios. The proposed framework creates a more robust centralized global submodel through heterogeneous knowledge sharing while keeping local data private. Then, it combines the global submodel with a local submodel for better adaptation to local data heterogeneity. Our experimental results show that the proposed framework outperforms other state-of-the-art methods.

This paper investigates secure control for homogeneous vehicle platoons in the presence of false data injection attacks with low communication and computation costs. We consider a scenario where each vehicle within the platoon transmits a local state vector to multiple neighboring vehicles. By leveraging these shared vectors from both preceding and following vehicles, we propose a novel and effective resilient controller for vehicle platoons against node/communication link attacks. More specifically, each vehicle determines the local state deviation vectors from neighboring vehicles. It then eliminates the vectors that are farthest from the origin, with the number of removed vectors equivalent to the maximum number of attacks. This approach offers a considerable advantage by mitigating the effects of abnormality and manipulation, making it robust against arbitrary information tampering within a pre-defined upper boundary for manipulated broadcast information. Importantly, we establish specific conditions for the proposed resilient design to guarantee the internal stability of the vehicle platoon under attacks. Extensive simulations and experiments involving four TurtleBot3s are conducted to demonstrate the effectiveness of the proposed resilient controller.

The integration of multi-sensor data for accurate and robust state estimation is a promising research area with various practical applications. In this paper, we propose an optimization-based fusion framework that combines the camera, 6-DoF IMU, and UWB sensors for accurate and robust real-time localization. Different from traditional localization strategies, the proposed framework includes a data preprocessing module to deal with the issue of time asynchrony among multi-sensor data and a back-end optimization process that relies on UWB loop closure detection to correct the localization error caused by the UWB measurement noise and VIO drift. Moreover, a state prediction model that takes into account all collected UWB data between two consecutive keyframes is proposed to further improve the localization performance. Experiments on public datasets and real-life scenarios demonstrate the efficiency and robustness of the proposed method.

Autonomous drone racing is becoming an excellent platform to challenge quadrotors' autonomy techniques including planning, navigation and control technologies. However, most research on this topic mainly focuses on single drone scenarios. In this paper, we describe a novel time-optimal trajectory generation method for generating time-optimal trajectories for a swarm of quadrotors to fly through pre-defined waypoints with their maximum maneuverability without collision. We verify the method in the Gazebo simulations where a swarm of 5 quadrotors can fly through a complex 6-waypoint racing track in a$35m\times 35m$space with a top speed of 14m/s. Flight tests are performed on two quadrotors passing through 3 waypoints in a$4m\times 2m$flight arena to demonstrate the feasibility of the proposed method in the real world. Both simulations and real-world flight tests show that the proposed method can generate the optimal aggressive trajectories for a swarm of autonomous racing drones. The method can also be easily transferred to other types of robot swarms.

As technology scaling is approaching its physical limit, lithography hotspot detection has become an essential task in design for manufacturability. Although the deployment of machine learning in hotspot detection is found to save significant simulation time, such methods typically demand non-trivial quality data to build the model. While most design houses are actually short of quality data, they are also unwilling to directly share such layout related data to build a unified model due to the concerns on IP protection and model effectiveness. On the other hand, with data homogeneity and insufficiency within each design house, the locally trained models can be easily over-fitted, losing generalization ability and robustness when applying to the new designs. In this paper, we propose a heterogeneous federated learning framework for lithography hotspot detection that can address the aforementioned issues. The framework can build a more robust centralized global sub-model through heterogeneous knowledge sharing while keeping local data private. Then the global sub-model can be combined with a local submodel to better adapt to local data heterogeneity. The experimental results show that the proposed framework can overcome the challenge of non-independent and identically distributed (non-IID) data and heterogeneous communication to achieve very high performance in comparison to other state-of-the-art methods while guaranteeing good convergence in various scenarios.

We develop a general framework unifying several gradient-based stochastic optimization methods for empirical risk minimization problems both in centralized and distributed scenarios. The framework hinges on the introduction of an augmented graph consisting of nodes modeling the samples and edges modeling both the inter-device communication and intra-device stochastic gradient computation. By designing properly the topology of the augmented graph, we are able to recover as special cases the renowned Local-SGD and DSGD algorithms, and provide a unified perspective for variance-reduction (VR) and gradient-tracking (GT) methods such as SAGA, Local-SVRG and GT-SAGA. We also provide a unified convergence analysis for smooth and (strongly) convex objectives relying on a proper structured Lyapunov function, and the obtained rate can recover the best known results for many existing algorithms. The rate results further reveal that VR and GT methods can effectively eliminate data heterogeneity within and across devices, respectively, enabling the exact convergence of the algorithm to the optimal solution. Numerical experiments confirm the findings in this paper.

This paper proposes a novel family of primal-dual-based distributed algorithms for smooth, convex, multi-agent optimization over networks that uses only gradient information and gossip communications. The algorithms can also employ acceleration on the computation and communications. We provide a unified analysis of their convergence rate, measured in terms of the Bregman distance associated to the saddle point reformation of the distributed optimization problem. When acceleration is employed, the rate is shown to be optimal, in the sense that it matches (under the proposed metric) existing complexity lower bounds of distributed algorithms applicable to such a class of problem and using only gradient information and gossip communications. Preliminary numerical results on distributed least-square regression problems show that the proposed algorithm compares favorably on existing distributed schemes.

In this paper, we consider a simple model of distributed sensor fusion problem in sensor networks with asymmetric links, where the common goal is linear parameter estimation. For the realistic scenario of bandwidth-constrained networks, we propose a least square approach, based on distributed quantized consensus algorithms, to compute the ideal centralized sample mean estimate. Analytical results show that the proposed approach is effective in smearing out the quantization errors, and outperforms the centralized approaches with respect to the estimation performance. Simulation results are provided to validate the analytical results.