Ensuring the privacy of local datasets has emerged as an important concern in decentralized learning. However, the inherent privacy-utility tradeoff remains a fundamental challenge for privacy preserving decentralized algorithms. To address this issue, we introduce Positive-Incentive Noise Generator (PING), a novel mechanism designed to eliminate negative impact of privacy noise on convergence while defending against powerful colluding inference attacks. PING leverages network topologies and lightweight encryption-decryption operations to generate correlated noise. Building upon PING, we propose PP-DPIN, a privacy preserving stochastic algorithm tailored for decentralized learning. By integrating differential privacy and differential information entropy, we provide a comprehensive privacy quantification for PP-DPIN, with at least half nodes achieving arbitrarily strong privacy guarantees. Furthermore, convergence rate of PP-DPIN is established under stochastic convex and nonconvex settings, which characterizes the impact of privacy noise and demonstrates the linear speedup relative to the network size. Experiments on computer vision tasks validate PP-DPIN's superior performance and robustness against attacks compared to state-of-the-art methods.

Existing distributed optimization algorithms often rely on the Lipschitz continuity of the objective function's gradient, but in practice, the Lipschitz constant is difficult to estimate, and the global Lipschitz continuity assumption may not hold. In this article, we propose two novel decentralized (proximal) algorithms, adaptive decentralized proximal primal--dual (ADPPD) and adaptive decentralized primal--dual (ADPD), which incorporate specially designed adaptive stepsizes within an improved primal-dual framework for composite optimization problems. These algorithms only require local estimates of cocoercivity and the Lipschitz modulus, eliminating the need for global Lipschitz continuity and avoiding the overly conservative stepsizes associated with a large Lipschitz constant. Moreover, the adaptive stepsizes are independent of the network, making the algorithms highly scalable. We provide detailed theoretical analyses to prove that ADPPD shows an ergodic convergence rate $\mathcal {O}({\scriptstyle \text {}^{\scriptstyle 1}}\hspace {-0.224em}/\hspace {-0.112em}{\scriptstyle k})$ when the smooth term $f_{i}$ and the nonsmooth term $g_{i}$ are convex, and ADPD shows a linear convergence rate when $f_{i}$ is strongly convex. Numerical experiments on least-squares and logistic regression problems confirm our theoretical results, demonstrating that our algorithms achieve faster convergence due to the better utilization of local Lipschitz continuity.

Security concerns in large-scale networked environments are becoming increasingly critical. To further improve the algorithm security from the design perspective of decentralized optimization algorithms, we introduce a new measure: Privacy Leakage Frequency (PLF), which reveals the relationship between communication and privacy leakage of algorithms, showing that lower PLF corresponds to lower privacy budgets. Based on such assertion, a novel differentially private decentralized primal-dual algorithm named DP-RECAL is proposed to take advantage of operator splitting method and relay communication mechanism to experience less PLF so as to reduce the overall privacy budget. To the best of our knowledge, compared with existing differentially private algorithms, DP-RECAL presents superior privacy performance and communication complexity. In addition, with uncoordinated network-independent stepsizes, we prove the convergence of DP-RECAL for general convex problems and establish a linear convergence rate under the metric subregularity. Evaluation analysis on least squares problem and numerical experiments on real-world datasets verify our theoretical results and demonstrate that DP-RECAL can defend some classical gradient leakage attacks.

This article concentrates on solving the k -winners-take-all (k WTA) problem with large-scale inputs in a distributed setting. We propose a multiagent system with a relatively simple structure, in which each agent is equipped with a 1-D system and interacts with others via binary consensus protocols. That is, only the signs of the relative state information between neighbors are required. By virtue of differential inclusion theory, we prove that the system converges from arbitrary initial states. In addition, we derive the convergence rate as O(1/t) . Furthermore, in comparison to the existing models, we introduce a novel comparison filter to eliminate the resolution ratio requirement on the input signal, that is, the difference between the k th and (k+1) th largest inputs must be larger than a positive threshold. As a result, the proposed distributed k WTA model is capable of solving the k WTA problem, even when more than two elements of the input signal share the same value. Finally, we validate the effectiveness of the theoretical results through two simulation examples.

This article addresses distributed optimization problems, in which a group of agents cooperatively minimize the sum of their private objective functions via information exchanging. Building on alternating direction method of multipliers (ADMM), we propose a privacy-preserving and communication-efficient decentralized quadratically approximated ADMM algorithm, termed PC-DQM, for solving such type of problems under the scenario of limited communication. In PC-DQM, an event-triggered mechanism is designed to schedule the communication instants for reducing communication cost. Simultaneously, for privacy preservation, a Hessian matrix with perturbed noise is introduced to quadratically approximate the objective function, which results in a closed form of primal vector update and then avoids solving a subproblem at each iteration with possible high computation cost. In addition, the triggered scheme is also utilized to schedule the update of Hessian, which can also reduce computation cost. We theoretically show that PC-DQM can protect privacy but without losing accuracy. In addition, we rigorously prove that PC-DQM converges linearly to the exact optimal solution for strongly convex and smooth objective functions. Finally, numerical simulation is presented to illustrate the effectiveness and efficiency of our algorithm.

This article focuses on developing distributed optimization strategies for a class of machine learning problems over a directed network of computing agents. In these problems, the global objective function is an addition function, which is composed of local objective functions. Such local objective functions are convex and only endowed by the corresponding computing agent. A second-order Nesterov accelerated dynamical system with time-varying damping coefficient is developed to address such problems. To effectively deal with the constraints in the problems, the projected primal-dual method is carried out in the Nesterov accelerated system. By means of the cocoercive maximal monotone operator, it is shown that the trajectories of the Nesterov accelerated dynamical system can reach consensus at the optimal solution, provided that the damping coefficient and gains meet technical conditions. In the end, the validation of the theoretical results is demonstrated by the email classification problem and the logistic regression problem in machine learning.

As an essential problem in robotics, foraging means that robots collect objects from a given environment and return them to a specified location. On many occasions, robots are required to perform foraging tasks in adversarial environments, such as battlefield rescue, where potential adversaries may damage robots with a certain probability. The longer an individual robot moves through adversarial environments, the higher the probability of being damaged by adversaries. The robot system can gain utility only when the robot brings carried objects back to a predetermined home station. Such a risk of being damaged makes returning home at different locations potentially relevant to the expected utility produced by the robot. Thus, the individual robot faces a dilemma when it responds to the potential risks in adversarial environments: whether to return the carried resources home or continue foraging tasks. In this article, two fundamental environment settings are discussed, homogeneous cases and heterogeneous cases. The former is analyzed as having both the optimal substructure property and the non-aftereffect property. Then, we present a dynamic programming (DP) algorithm that can find an optimal solution with polynomial time complexity. For the latter, it is proven that finding an optimal solution is \( \mathcal {NP} \) -hard. We then propose a heuristic algorithm: A division hierarchical path planning (DHPP) algorithm that is based on the idea of dividing the foraging routes generated initially into a certain number of subroutes to dilute risks. Finally, these algorithms are extensively evaluated in simulations, concluding that in adversarial environments, they can significantly improve the productivity of an individual robot before it is damaged.

Existing studies on the multirobot foraging problem often assume safe settings, in which nothing in an environment hinders the robots’ tasks. In many real-world applications, robots have to collect objects from hazardous environments like earthquake rescue, where possible risks exist, with possibilities of destroying robots. At this stage, there are no targeted algorithms for foraging robots in hazardous environments, which can lead to damage to the robot itself and reduce the final foraging efficiency. A motivating example is a rescue scenario, in which the lack of a suitable solution results in many victims not being rescued after all available robots have been destroyed. Foraging robots face a dilemma after some robots have been destroyed: whether to take over tasks of the destroyed robots or continue executing their remaining foraging tasks. The challenges that arise when attempting such a balance are twofold: (1) the loss of robots adds new constraints to traditional problems, complicating the structure of the solution space, and (2) the task allocation strategy in a multirobot team affects the final expected utility, thereby increasing the dimension of the solution space. In this study, we address these challenges in two fundamental environmental settings: homogeneous and heterogeneous cases. For the former case, a decomposition and grafting mechanism is adopted to split this problem into two weakly coupled problems: the foraging task execution problem and the foraging task allocation problem. We propose an exact foraging task allocation algorithm, and graft it to another exact foraging task execution algorithm to find an optimal solution within the polynomial time. For the latter case, it is proven \( \mathcal {NP} \) -hard to find an optimal solution in polynomial time. The decomposition and grafting mechanism is also adopted here, and our proposed greedy risk-aware foraging algorithm is grafted to our proposed hierarchical agglomerative clustering algorithm to find high-utility solutions with low computational overhead. Finally, these algorithms are extensively evaluated through simulations, demonstrating that compared with various benchmarks, they can significantly increase the utility of objects returned by robots before all the robots have been stopped.

This paper develops a fully distributed self-triggered framework for achieving the second-order consensus in multiagent systems. In this framework, a fully distributed self-triggered scheme is proposed to schedule information transmission for each communication channel. Thus, the communication frequency of each channel is significantly reduced. Here, communication over different channels is independent with each other and, thus, a channel-based control protocol is further proposed. The update times of control protocol could be also lowered in this framework. Here, an iterative evaluation method is constructed to design a fully distributed self-triggered scheme without involving any global information, especially the eigenvalue information of the Laplacian matrix. In addition, by introducing several variables, some sufficient conditions for achieving the second-order consensus and average consensus are obtained, respectively, in a distributed fashion, and the Zeno behavior is successfully eliminated. Finally, a simulation example is provided to verify the theoretical analysis.

In this paper, a continuous-time multiagent system is proposed for solving optimal resource allocation problems with local allocation feasible constraints. In the system, all the primal agents are divided into different groups. We use dual variables which describe the dual agents to represent the groups of the original agents. The groups of dual agents are used to communicate with others on behalf of the primal agents to reduce communication costs. That is to say, primal agents aim to seek their own optimal solutions by using local information. And dual agents represent primal agents to communicate with other agents in different groups by using the whole group information. The two kinds of agents cooperate to find the optimal solution of the problem. In this way, we only need to know the connections of dual agents to design the multiagent network, and do not need to consider the connections of the primal agents. So the communication cost and the amount of variables will be largely reduced especially for large-scale problem. Furthermore, it is proved that the multiagent system can reach consensus with respect to the dual variables. At the same time, the primal variables are convergent to the optimal solutions of the optimization problem under some certain assumptions on the communication network. For large-scale problem if we take the groups as areas, then the system is suitable for multiarea problem. Simulation results are presented to demonstrate the performance of the proposed multiagent system.

A clock network is not only difficult to design, but also challenging to test. For high-performance designs with a rigorous clock-skew requirement, small defects in a clock tree network could lead to unexpected failures in the field and thus need to be identified during the manufacturing test. In this paper, we present a novel flush test procedure to determine if a clock network has any small delay faults. This method does not require any change of the clock network, but it does require a “special test clock signal,” which can be generated on the chip by using only standard cells. Experimental results of transistor-level simulation on benchmark circuits injected with resistive open defects in the layout show that the proposed method is capable of detecting a delay fault as small as 52.8 ps.

This paper is concerned with multiple-objective distributed optimization. Based on objective weighting and decision space decomposition, a collaborative neurodynamic approach to multiobjective distributed optimization is presented. In the approach, a system of collaborative neural networks is developed to search for Pareto optimal solutions, where each neural network is associated with one objective function and given constraints. Sufficient conditions are derived for ascertaining the convergence to a Pareto optimal solution of the collaborative neurodynamic system. In addition, it is proved that each connected subsystem can generate a Pareto optimal solution when the communication topology is disconnected. Then, a switching-topology-based method is proposed to compute multiple Pareto optimal solutions for discretized approximation of Pareto front. Finally, simulation results are discussed to substantiate the performance of the collaborative neurodynamic approach. A portfolio selection application is also given.

This paper presents a collective neurodynamic approach with multiple interconnected recurrent neural networks (RNNs) for distributed constrained optimization. The objective function of the distributed optimization problems to be solved is a sum of local convex objective functions, which may be nonsmooth. Subject to its local constraints, each local objective function is minimized individually by using an RNN, with consensus among others. In contrast to existing continuous-time distributed optimization methods, the proposed collective neurodynamic approach is capable of solving more general distributed optimization problems. Simulation results on three numerical examples are discussed to substantiate the effectiveness and characteristics of the proposed approach. In addition, an application to the optimal placement problem is delineated to demonstrate the viability of the approach.

In this paper, new results on the global synchronization of multiple recurrent neural networks (NNs) with time delays via impulsive interactions are presented. Impulsive interaction means that a number of NNs communicate with each other at impulse instants only, while they are independent at the remaining time. The communication topology among NNs is not required to be always connected and can switch ON and OFF at different impulse instants. By using the concept of sequential connectivity and the properties of stochastic matrices, a set of sufficient conditions depending on time delays is derived to ascertain global synchronization of multiple continuous-time recurrent NNs. In addition, a counterpart on the global synchronization of multiple discrete-time NNs is also discussed. Finally, two examples are presented to illustrate the results.

In this paper, distributed optimization is addressed based on a continuous-time multiagent system in the presence of time-varying communication delays. First, the relationship between optimal solutions and the equilibrium points of the multiagent system with time delay is revealed. Next, delay-dependent and delay-independent sufficient conditions in form of linear matrix inequality are derived for ascertaining convergence to optimal solutions, in the cases of slow-varying delay and fast-varying delay. Furthermore, a set of conditions are also obtained for the delay-free case. In addition, a sampled-data communication scheme is presented based on the conditions for the fast varying delay systems. Simulation results are presented to substantiate the theoretical results. An application for distributed parameter estimation is also given.

A clock network in a 3D-IC is not only difficult to design, but also challenging to test. For high-performance designs with a rigorous clock-skew requirement, studies have shown that small defects in a clock tree network could lead to unexpected failures in the field and thus need to be identified during the manufacturing test. In this paper, we present a novel test method to determine if a clock network has any small delay faults. This method does not require any change of the clock network, and it is capable of detecting a delay fault as small as 50ps through outlier analysis, while locating the FFs affected by the fault.

This paper presents theoretical results on the global exponential synchronization of multiple memristive neural networks with time delays. A novel coupling scheme is introduced, in a general topological structure described by a directed or undirected graph, with a linear diffusive term and discontinuous sign term. Several criteria are derived based on the Lyapunov stability theory to ascertain the global exponential stability of synchronization manifold in the coupling scheme. Simulation results for several examples are given to substantiate the effectiveness of the theoretical results.

This paper is concerned with the global robust synchronization of multiple memristive neural networks (MMNNs) with nonidentical uncertain parameters. A coupling scheme is introduced, in a general topological structure described by a direct or undirect graph, with a linear diffusive term and a discontinuous sign term. First, a set of sufficient conditions are derived based on the Lyapunov stability theory for ascertaining global robust synchronization of coupled MMNNs. Second, a pinning adaptive coupling method is proposed to ensure global synchronization without knowing the bound of parameter uncertainties. Two illustrative examples are discussed to substantiate the theoretical results.