Operating large multi-reservoir systems is critical for effective water allocation, hydropower generation, and economic development. However, conventional robust planning-based methods scale poorly beyond single-reservoir or cascaded systems due to computational intractability. Addressing inter-reservoir relation extraction and coordination under conflicting objectives, uncertain inflows, and complex network couplings therefore remains an open challenge. To tackle this problem, we introduce the multi-agent reservoir cooperative operation (MARCO) environment, which integrates multiple objectives, heterogeneous topology, and stochastic inflows in a unified framework. An algorithm is then designed to construct a multi-relational directed acyclic graph (MR-DAG) that encodes the underlying topology through coupled objectives and entity relations. Building on this representation, we propose the multi-agent relational directed acyclic graph transformer (MAR-DAGT), a reinforcement learning algorithm that performs typed message passing for efficient feature extraction and employs acyclic decision-making to exploit causal structure for improved credit assignment. Extensive experiments on MARCO show that MAR-DAGT consistently outperforms other MARL and optimization-based baselines in terms of objective satisfaction and robustness to inflow uncertainty.

This paper investigates multi-coalition games (MCGs) involving players with high-order dynamics, where their decisions satisfy both hard local set constraints and nonlinear coupled inequality constraints. The hard set constraints require that players’ decisions always satisfy the prescribed local set constraints. The coexistence of constraints and high-order dynamics leads to significant challenges for algorithm design and analysis, as the system inertia caused by high-order dynamics prevents direct control of the decisions of players through their inputs. To control the constrained high-order players to perform MCG tasks autonomously and distributedly, based on state feedback, leader-following consensus, dynamic average consensus, projection, and adaptive control methods, a fully distributed algorithm is proposed, where the decisions of all players are ensured to satisfy the set constraints all the time. It allows players to adjust their own parameters by using their procurable feedback errors without requiring direct access to others’ decisions or gradients, eliminating the need for global information and parameters tuning. Furthermore, the convergence of this algorithm is rigorously proven by nonsmooth analysis and Lyapunov stability theory. Finally, the algorithm is applied to the electricity market games (EMGs) of smart grids to demonstrate its effectiveness.

This article focuses on nonconvex federated composite optimization (FCO) problem, where the loss function is nonconvex and contains a nonsmooth regularizer. To resolve this problem, we propose FedRREF, a novel federated learning algorithm that integrates error feedback (EF) with the efficient random reshuffling (RR) technique, resulting in lower computation and communication costs. To the best of our knowledge, FedRREF is the first algorithm to consider RR and biased compression simultaneously in federated learning, especially in nonsmooth and nonconvex settings, and it is shown to have a $\mathcal {O}(1/\sqrt {T})$ convergence rate, where $T$ is the number of communication rounds. Finally, the numerical experiments illustrate the validity of the proposed algorithm.

Game theory has emerged as a fundamental framework for modeling and analyzing strategic interactions and decision-making among multiple agents, and has witnessed rapidly growing impact in cyber–physical systems over the past decade. Its integration with dynamic systems has driven major theoretical and technological advances in a wide range of applications, including smart grids, autonomous driving, robotic swarms, and networked control systems. In particular, distributed games in dynamic systems and their equilibrium learning mechanisms have attracted increasing attention due to their scalability, lightweight information exchange, and real-time implementability. This article provides a comprehensive survey of distributed games in dynamic systems, where agents interact only with local neighbors while collectively achieving global equilibrium and stability. First, the foundational theories of distributed dynamic games under three representative classes of systems: linear dynamic systems, nonlinear dynamic systems, and uncertain dynamic systems, are presented. Then, state-of-the-art distributed equilibrium learning and control methods are reviewed, including gradient-based dynamics, payoff-based learning, best-response dynamics, and learning-based approaches. To demonstrate the practical relevance and impact of distributed games in dynamic systems, representative application domains are discussed in detail. Finally, several promising future research directions are outlined, highlighting open challenges at the intersection of distributed games, learning, and dynamic systems.

Accurate prediction of pedestrian trajectories is crucial for safe motion planning of autonomous vehicles in urban environments. Many existing temporal and generative models are constrained by the long-tailed distribution of the data, which limits their ability to handle random or irregular pedestrian movements. Moreover, few studies have addressed the problem of scoring and probabilistic evaluation of predicted trajectories, despite their importance for downstream decision-making tasks. To address these issues, we propose a regularization-randomization network (R-RNet). The core regularization-randomization (R-R) module enables flexible trajectory prediction across diverse scenarios, while the probability predictor provides trajectory scoring and probability estimation to enhance reliability and utility in subsequent tasks. Besides, a self-attention mechanism is utilized to enhance prediction performance by capturing features from the distribution of the goals. The experimental results on the ETH and the UCY datasets show that R-RNet is capable of making reliable evaluations on output trajectories and achieves competitive results in terms of average displacement error and miss rate, while maintaining a lightweight architecture. Extensive experiments and analyses underscore the critical importance of both regularization and randomization operations. The source code is released athttps://github.com/RoboSafe-Lab/R-RNet

This paper explores nudge schemes for a central regulator aimed at incentivizing players in constrained noncooperative games to reach desired equilibria. Unlike traditional intervention mechanisms where players update their actions by blindly following signals from the regulator, the nudge mechanism comprehensively integrates players’ rational judgment by incorporating trust variables into players’ models. This implies that players update their actions by evaluating the signals from the regulator and their own expectations to the incentive mechanisms. If the regulator’s signals significantly deviate from the players’ expectations, players decrease trust in the regulator and rely more on the their expectations when updating their actions. Conversely, if the signals align closely with their expectations, players tend to increase trust in the regulator and place greater emphasis on the regulator’s signals. It should be noted that each player does not have access to other players’ actions, which means that it updates its action in a distributed manner by only observing the actions of its neighbors through a directed balanced graph. Furthermore, static and dynamic nudges are designed based on different information available to the regulator, which are also extended to an online case with the desired equilibrum being time-varying. Finally, an application to the robot formation control is shown to validate the obtained results.

The cooperative-competitive network has broad application prospects due to its advantages in aligning with practical production and life. This work, for the first time, combines the predictive capability of the proportional-integral-derivative (PID) protocol for error and the flexibility of an additional order parameter in fractional calculus to solve the distributed resource allocation problem in cooperative-competitive networks, and provides a discrete time algorithm for the design. The analysis proves that the algorithms can Mittag-Leffler converge to the optimal solution of the distributed resource allocation problem. Finally, numerical simulations are provided, demonstrating the effectiveness of the algorithm, along with a set of comparative simulation that validate the superiority of the proposed algorithm.

This paper addresses the abstract dynamic programming (DP) in the online scenario, where the abstract DP mapping is time-varying, instead of static. In this case, optimal costs and policies at different time instants are not the same in general, and the problem amounts to tracking time-varying optimal costs and policies, which is of interest to many practical problems. It is thus necessary to analyze the performance of classical value iteration (VI) and policy iteration (PI) algorithms in the online case. In doing so, this paper develops and provides the theoretical analysis for several online algorithms, including approximate online VI, online PI, approximate online PI, online optimistic PI, and approximate online optimistic PI algorithms. It is proved that the tracking error bounds for all algorithms critically depend upon the largest difference between any two consecutive abstract mappings. Meanwhile, examples are presented to illustrate the theoretical results.

Since autonomous driving (AD) is one of the most critical problems in the automobile industry, it has garnered the interest of many academics in recent years. An AD framework or a simulator is used to simulate autonomous vehicles (AVs), it can test modules that an AV needs. There are a large number of researchers studying specific AD simulation tasks, but few studies of systematic AD frameworks and simulators exist. This study distinguishes the functions of AD frameworks and simulators, and it makes a deep study of them, helping promote AV development for researchers, enterprises , and developers. This paper reviews open-source and commercial AD frameworks and simulators, introducing and comparing their features, functionalities, and so on. Additionally, we analyze current research on open-source AD frameworks and simulators according to different applied algorithms and hardware studies and discuss efforts to improve their simulation performance. In the last part, this paper proposes promising research fronts for AD frameworks and simulators for the near future from the viewpoints of hardware deficiencies, AD algorithms, scenario generation, vehicle-to-everything, safety and performance, and co-simulation.

This work solves the countermeasure design problems of distributed resilient output time-varying formation-tracking (TVFT) of heterogeneous multiagent systems (MASs) against general Byzantine attacks (GBAs). Inspired by the concept of Digital Twin, a hierarchical protocol equipped with a twin layer (TL) is proposed, which decouples the above problem into the defense against Byzantine edge attacks (BEAs) on the TL and the defense against Byzantine node attacks (BNAs) on the cyber-physical layer (CPL). First, a secure TL with respect to (w.r.t.) the high-order leader dynamics is designed, which achieves resilient estimation against BEAs. A trusted-node strategy against BEAs is proposed, which promotes network resilience by protecting almost the smallest fraction of crucial nodes on the TL. It is proven that strongly (2f+1) -robustness w.r.t. the above trusted nodes is sufficient for the resilient estimation performance of the TL. Second, a decentralized adaptive and chattering-free controller against potentially unbounded BNAs is designed on the CPL. This controller has the merit of uniformly ultimately bounded (UUB) convergence and an assignable exponential decay rate when converging into the above UUB bound. To the best of our knowledge, this article is the first to achieve resilient output TVFT against GBAs, rather than under GBAs. Finally, the practicability and validity of this new hierarchical protocol are illustrated via a simulation example.

Communication has been seen as a significant bottleneck in industrial applications over large-scale networks. To alleviate the communication burden, sign-based optimization algorithms have gained popularity recently in both industrial and academic communities, which is shown to be closely related to adaptive gradient methods, such as Adam. Along this line, this article investigates faster convergence for a variant of sign-based gradient descent, called scaledsignGD, in three cases: First, the objective function is strongly convex; second, the objective function is nonconvex but satisfies the Polyak–Łojasiewicz inequality; third the gradient is stochastic, called scaledsignSGD in this case. For the first two cases, it can be shown that the scaledsignGD converges at a linear rate. For case third, the algorithm is shown to converge linearly to a neighborhood of the optimal value when a constant learning rate is employed, and the algorithm converges at a rate of$O(1/k+1/k^{2}+1/k^{3})$when using a diminishing learning rate, where$k$is the iteration number. The results are also extended to the distributed setting by majority vote in a parameter-server framework. Finally, numerical experiments are performed to corroborate the theoretical findings.

Autonomous vehicles necessitate prediction of future motions of surrounding traffic participants for safe navigation. However, prediction is challenging due to complex road structures and the multimodality of driving behaviors. Recent approaches usually output a fixed number of predicted trajectories, thus can hardly generalize to situations with more options and match modalities in different situations. They also rely on complex loss design and time-consuming training. This work proposes a novel map-adaptive multimodal trajectory predictor that links driving modalities, driver’s intentions, and a vehicle’s candidate centerlines (CCLs) together, rendering the predictor map-adaptive and the multimodality explainable. The predictor is derived by training an intention-aware unimodal trajectory predictor, which consists of aCCL-based goal predictorand agoal-directed trajectory completer, and aCCL scorerfor estimating the possibilities of a target vehicle (TV) choosing a CCL to follow. This decomposed approach simplifies the training process and reduces the computational resources required, thereby rendering it a faster and more cost-effective alternative. Additionally, the proposed predictor has demonstrated comparable or even superior performance to traditional multimodal predictors in specific applications. Overall, the unimodal predictor presents a promising approach for practical machine learning applications, particularly when computational resources are limited.

By leveraging IoT data, Business Process Management (BPM) systems can sense the physical world situation and make more accurate decisions proactively, but the current BPM systems lack the effective method to take good use of IoT data. In this paper, a situation-aware dynamic binding method for IoT services targeting proactive BPM is proposed to meet this requirement. The method first establishes a prediction model to predict the bindable IoT services under given situations by constructing an encoder-decoder structure network model using bi-directional gated recurrent units (Bi-GRU) and incorporating an attention mechanism. Then our method creates a Dynamic IoT Service Task (DIT) model for BPM. This enables the BPM system to integrate the prediction model with dynamic switching of IoT services and a parallel running architecture for multiple IoT services. A prototype system is developed and a case study is conducted to evaluate the performance of the method. The study focuses on the safety supervision scenario for the transportation of hazardous Liquefied Natural Gas (LNG) by sea. Results show that the prediction model proposed in this paper outperforms other models on multiple indicators. Also, the case study and experimental results verify the effectiveness of the method.

Within the realm of distributed optimization, each node in the network possesses computational capabilities. Nodes perform local calculations on their own data, and by communicating local information (e.g., local gradients) with neighboring nodes, agents collectively achieve a globally optimal solution. In recent years, distributed optimization has garnered interest across diverse disciplines, particularly in situations where communication abilities are limited or data are private. In this paper, a distributed second-order algorithm, based on an augmented Lagrangian function, is proposed with an enhanced diffusion communication strategy. An R-linear convergence rate is established under a relaxed locally restricted strong convexity assumption, along with a widely employed L-smoothness assumption. Finally, the superiority of the algorithm is showcased by a distributed logistic regression example utilizing a synthetic dataset with various parameter settings.

This article studies the problem of distributed framework matching (FM), which originates from the assignment task in multirobot coordination and the matching task in pattern recognition. The objective of distributed FM is to distributively seek a correspondence which minimizes some metrics describing the disagreement between two frameworks (i.e., graphs and their embeddings). In view of the type of the underlying graph in the framework, two formulations, undirected framework matching (UFM) and directed framework matching (DFM), and their convex relaxations, relaxed UFM (RUFM), and relaxed DFM (RDFM), are presented. UFM is converted into a graph matching (GM) problem with the adjacency matrix being replaced by a matrix constructed from the undirected framework under certain graphical conditions, and can be solved distributively. Sufficient conditions for the equivalence between UFM and RUFM, and the perturbation admitting exact recovery of correspondence are established. On the other hand, DFM embeds the configuration of the directed framework via another type of matrix, whose computation is distributed, and can deal with the case of two frameworks with different sizes of node sets. A distributed optimization algorithm for solving RDFM is proposed and its convergence results are established which allows DFM to be solved in a fully distributed manner. Simulation examples on both synthetic data and real world datasets demonstrate the applicability and efficacy of our theoretical results in formation control and object matching problems.

In this article, we consider the problem of using a multirobot system to conduct sweep coverage over a region with uneven and unknown workload distribution. Uneven workload distribution means that a robot has to spend different amounts of time covering a unit area at different locations in the region. Unknown workload distribution means that the amount of workload at any location is unknown prior to the operation, hence online sensing and allocation of workload is needed for better efficiency. In this work, we adopt the formulation in which the entire region is separated into multiple stripes, and a discrete-time distributed workload allocation algorithm is used to allocate workload on a stripe to each robot. Previous works that adopt similar formulations do not provide rigorous stability analysis and experimental verification and lack consideration of practical aspects, such as limited sensor range. This work addresses these weaknesses and bridges the gap between theory and practice. First, compared with the existing works, the convergence of the distributed workload allocation algorithm to the optimal workload assignment is established under a more realistic assumption, and less conservative error bounds are derived, which serve as a better indicator of the effectiveness of the algorithm. Second, we propose a new algorithm that addresses the limited sensor range of robots, which is an important constraint in applications, such as agricultural spraying and building inspection. The stability analysis and error bound of the proposed algorithm are also provided. Third, realistic simulations and actual flight experiments using unmanned aerial vehicles are carried out to demonstrate the practicality and validate the theoretical results.

This article aims to address the problem of distributed energy management for both the generation and demand sides in smart grid. Different from many existing works, we investigate the SWM problem with transmission losses. In addition, instead of transforming the local constraint into an approximate penalty function or the projection set, we consider it as a convex nonsmooth indicator function from a different viewpoint. For such a composite problem consisting of smooth and nonsmooth terms, we propose a distributed proximal primal–dual algorithm based on dual decomposition and operator splitting techniques. Each node performs the algorithm through only local computation and communication with limited information, especially not sharing the sensitive gradient directly. It is also proved that the proposed algorithm leads to the global optima at a convergence rate$O(\frac{1}{k})$with a fixed stepsize. Several simulations verify the theoretical analysis and demonstrate the effectiveness of the proposed algorithm.

Due to the growing awareness of driving safety and the development of sophisticated technologies, advanced driving/driver assistance system (ADAS) has been equipped in more and more vehicles with higher accuracy and lower price. The latest progress in this field has called for a review to sum up the conventional knowledge of ADAS, the state-of-the-art researches, novel applications and standards in real world. With the help of this kind of review, newcomers in this field can get basic knowledge easier and other researchers may be inspired with potential future development possibility. This paper makes a general introduction about ADAS by analyzing its hardware support, computation algorithms and current development state. Different types of perception sensors are introduced from their interior feature classifications, installation positions, supporting ADAS functions, and pros and cons. The comparisons between different sensors are concluded and illustrated from their inherent characters and specific usages serving for each ADAS function. The current algorithms for ADAS functions are also collected and briefly presented in this paper from both traditional methods and novel ideas. Additionally, discussions about current regulations and market state of ADAS in China are reviewed in this paper, and other open issues related to ADAS are also introduced in particular.

This paper considers online convex optimization with long term constraints, where constraints can be violated in intermediate rounds, but need to be satisfied in the long run. The cumulative constraint violation is used as the metric to measure constraint violations, which excludes the situation that strictly feasible constraints can compensate the effects of violated constraints. A novel algorithm is first proposed and it achieves an $\mathcal{O}(T^{\max\{c,1-c\}})$ bound for static regret and an $\mathcal{O}(T^{(1-c)/2})$ bound for cumulative constraint violation, where $c\in(0,1)$ is a user-defined trade-off parameter, and thus has improved performance compared with existing results. Both static regret and cumulative constraint violation bounds are reduced to $\mathcal{O}(\log(T))$ when the loss functions are strongly convex, which also improves existing results. %In order to bound the regret with respect to any comparator sequence, In order to achieve the optimal regret with respect to any comparator sequence, another algorithm is then proposed and it achieves the optimal $\mathcal{O}(\sqrt{T(1+P_T)})$ regret and an $\mathcal{O}(\sqrt{T})$ cumulative constraint violation, where $P_T$ is the path-length of the comparator sequence. Finally, numerical simulations are provided to illustrate the effectiveness of the theoretical results.

This paper studies the dynamic formation control problem for cooperative agents with discrete-time dynamics over directed graphs. Unlike using absolute coordinate, relative coordinate, interagent distance, or interagent bearing to specify the target formation and coordinate agents to achieve the formation, we study a coordination problem where the desired formation varies with time and only its geometric shape is predefined. Matrix-valued Laplacian approach has been adopted to address this problem in the continuous-time setting. However, the discrete-time counterpart is more challenging due to the constraint in information exchange. On the other hand, observe that in many real operations, at a given time instant, the agents will be able to plan their target formation configurations for a period of time ahead. We propose preview-based P-like and PD-like controllers for the formation control. The controllers with proper parameter setting are proved to be effective to address the dynamic formation control problem. Numerical simulations are given to validate the effectiveness of the proposed controllers.

This puts forth an infrastructure-free cooperative relative localization (RL) for unmanned aerial vehicles (UAVs) in global positioning system (GPS)-denied environments. Instead of estimating relative coordinates with vision-based methods, an onboard ultra-wideband (UWB) ranging and communication (RCM) network is adopted to both sense the inter-UAV distance and exchange information for RL estimation in 2-D spaces. Without any external infrastructures prepositioned, each agent cooperatively performs a consensus-based fusion, which fuses the obtained direct and indirect RL estimates, to generate the relative positions to its neighbors in real time despite the fact that some UAVs may not have direct range measurements to their neighbors. The proposed RL estimation is then applied to formation control. Extensive simulations and real-world flight tests corroborate the merits of the developed RL algorithm.

This paper investigates the joint effect of quantization, sampled data, and general Markovian interaction links on consensus networks with a leader under directed graphs. The diversity of edges formed by all the followers and the leader is also considered. Each agent in the network possesses continuous-time general linear dynamics. Each agent's state is measured only at sampling time instants, which is encoded before transmission. Subsequently, the encoded state is transmitted through noiseless digital communication links with Markovian switching rates. For this problem, a sufficient condition is derived to guarantee the convergence of the encoded states, based on which a necessary and sufficient condition is obtained to achieve consensus tracking in the mean-square sense. In addition, two sufficient conditions on coupling gain, one of which is fully distributed, are provided by proposing an optimal linear quadratic regulator-based gain matrix to ensure consensus tracking and then, the analysis of consensus region is presented. Finally, a numerical example is presented for illustrating the effectiveness of the theoretical results.

This paper presents a notion of rigidity, Ratio-of-Distance (RoD) rigidity, for studying when a framework can be uniquely determined by a set of RoD constraints up to similar transformations. Unlike the constraints such as distance, bearing and angle used in existing literature, we aim to develop a framework by a set of RoD constraints (the ratio of distances of a pair of edges joining a common vertex). In particular, triangulated Laman graphs are introduced to provide some sufficient conditions for the RoD rigidity. The proposed RoD rigidity theory is further applied to the RoD-based similar formation control. Numerical simulations are provided to support the analysis.

This paper establishes several bounds on the algebraic connectivity and spectral radius of graphs. Before deriving these bounds, a directed graph with a leader node is first investigated, for which some bounds on the spectral radius and the smallest real part of all the eigenvalues of M = L + D are obtained using the properties of M-matrix and non-negative matrix under a mild assumption, where L is the Laplacian matrix of the graph and D = diag{d1, d2, ..., dN} with di > 0 if node i can access the information of the leader node and 0 otherwise. Subsequently, by virtue of the results on directed graphs, the bounds on the algebraic connectivity and spectral radius of an undirected connected graph are provided. Besides establishing these bounds, another important feature is that all these bounds are distributed in the sense of only knowing the information of edge weights' bounds and the number of nodes in a graph, without using any information of inherent structures of the graph. Therefore, these bounds can be in some sense applied to agent networks for reducing the conservatism where control gains in control protocols depend on the eigenvalues of matrices M or L, which are global information. Also some examples are provided for corroborating the feasibility of the theoretical results.