A correction is given to a previously published result concerned with the relationship between a suitably defined matrix seminorm for consensus analysis and a coefficient of ergodicity.

We develop provably safe and convergent reinforcement learning (RL) algorithms for control of nonlinear dynamical systems, bridging the gap between the hard safety guarantees of control theory and the convergence guarantees of RL theory. Recent advances at the intersection of control and RL follow a two-stage, safety filter approach to enforcing hard safety constraints: model-free RL is used to learn a potentially unsafe controller, whose actions are projected onto safe sets prescribed, for example, by a control barrier function. Though safe, such approaches lose any convergence guarantees enjoyed by the underlying RL methods. In this paper, we develop a single-stage, sampling-based approach to hard constraint satisfaction that learns RL controllers enjoying classical convergence guarantees while satisfying hard safety constraints throughout training and deployment. We validate the efficacy of our approach in simulation, including safe control of a quadcopter in a challenging obstacle avoidance problem, and demonstrate that it outperforms existing benchmarks.

Quantum Internet has potential capabilities far beyond the traditional Internet and is thus a promising future platform for communication and computation. Entanglement is a cornerstone of quantum mechanics and forms the basis of numerous quantum applications in the quantum Internet. While existing studies primarily focus on two-user entanglement, a plethora of applications necessitates the leap to multi-user entanglement. This paper tackles the fundamental problem of multi-user entanglement routing in the quantum Internet, aiming to entangle multiple quantum users with a high entanglement rate. We abstract the problem as a novel graph routing problem, which is not readily addressed by existing graph problem solutions due to the unique characteristics of the quantum Internet. To address this problem, we first consider a sufficient condition ensuring a feasible solution's existence and design an algorithm with the optimal solution. Given the NP-Completeness and NP- Hardness of determining a feasible solution's existence and deriving an optimal solution in general cases, respectively, we propose two heuristic algorithms to offer efficient solutions, which are shown, via extensive simulations, to outperform the existing algorithms in terms of entanglement rates.

Quantum networks have emerged as a future platform for quantum information exchange and applications, with promising capabilities far beyond traditional communication networks. Remote quantum entanglement is an essential component of a quantum network. How to efficiently design a multi-routing entanglement protocol is a fundamental yet challenging problem. In this paper, we study a quantum entanglement routing problem to simultaneously maximize the number of quantum-user pairs and their expected throughput. Our approach is to formulate the problem as two sequential integer programming problems. We propose efficient entanglement routing algorithms for these two optimization problems and analyze their time complexity and performance bounds. Evaluation results highlight that our approach outperforms existing solutions in both the number of quantum-user pairs served and network throughput.

Generating a long-distance quantum entanglement is one of the most essential functions of a quantum network to support quantum communication and computing applications. The successful entanglement rate during a probabilistic entanglement process decreases dramatically with distance, and swapping is a widely-applied quantum technique to address this issue. Most existing entanglement routing protocols use a classic entanglement-swapping method based on Bell State measurements that can only fuse two successful entanglement links. This paper appeals to a more general and efficient swapping method, namely n-fusion based on Greenberger-Horne-Zeilinger measurements that can fuse n successful entanglement links, to maximize the entanglement rate for multiple quantum-user pairs over a quantum network. We propose efficient entanglement routing algorithms that utilize the properties of n-fusion for quantum networks with general topologies. Evaluation results highlight that our proposed algorithm under n-fusion can greatly improve the network performance compared with existing ones.

Internet of Things (IoT) is progressively becoming an essential aspect of daily life that can be sensed anywhere and anytime, transforming the traditional lifestyle into a high-tech one. Numerous applications in the edge are brought to life based on IoT infrastructures. Especially, edge computing has witnessed the proliferation and impact of IoT-enabled devices benefiting from the data collection and computation capabilities of IoT. However, establishing an IoT from scratch can be monetarily expensive, and leasing the existing sub-networks confronts the potentially dishonest behavior of service providers. To address these issues, we propose a novel framework of leasing edge IoT networks and analyze the influence of sub-network owners’ dishonest behavior on the network. We model the interaction between the edge user and the owners of sub-networks by a Stackelberg game with a unique equilibrium, jointly analyzing the pricing and data collection mechanisms. The Primal-dual Decomposition algorithm and its theoretical analyses are provided for the corresponding strategies of the edge user and sub-network owners. Evaluations demonstrate that the proposed algorithm in the leasing model can save data collection cost up to 53% compared with existing data collection strategies, and illustrate the difference in network performance compared with the game without dishonest owners.

Edge caching is an effective way to reduce congestion and latency in 5G networks. Non-volatile memory (NVM) devices are developing fast, with the potential of fast access, and higher endurance versus traditional storage devices, to further boost mobile data offloading efficiency in 5G networks. This paper studies how to effectively use the two-layer storage system (NVM-enhanced) in 5G edge caching. We first model an edge caching optimization problem with NVM storage devices included and develop a parallel distributed algorithm with guaranteed convergence in joint caching and routing decisions. A fully decentralized algorithm for scenarios without any coordination is further developed which also guarantees the convergence. Real-world trace-driven simulations and experiments over a small-scale system demonstrate that NVM significantly boosts the performance of edge caching and the proposed algorithms outperform the existing ones.

Quantum networks are considered as a promising future platform for quantum information exchange and quantum applications, which have capabilities far beyond the traditional communication networks. Remote quantum entanglement is an essential component of a quantum network. How to efficiently design a multi-routing entanglement protocol is a fundamental yet challenging problem. In this paper, we study a quantum entanglement routing problem to simultaneously maximize the number of quantum-user pairs and their expected throughput. Our approach is to formulate the problem as two sequential integer programming steps. We propose efficient entanglement routing algorithms for the two integer programming steps and analyze their time complexity and performance bounds. Results of evaluation highlight that our approach outperforms existing solutions in both served quantum-user pairs numbers and the network expected throughput.

This paper considers a federated temporal difference() (TD()) learning algorithm and provides both asymptotic and finite-time analyses. To protect each worker agent cost information from being accessed by possible attackers, we propose a privacy-preserving variant of the algorithm by adding perturbation to the exchanged information. We show the rigorous differential privacy guarantee by using moments accountant and derive an upper bound of the utility loss for the privacy-preserving algorithm. Evaluations are also provided to corroborate the efficiency of the algorithms.

How to serve mobile users in rural areas by 5G networks is challenging due to the sparse distribution of base stations and poor connection to the cloud. Existing solutions focus on transmission frequency implementation such as frequency multiplexing, In this paper, we consider a decentralized caching scheme for two reasons. First, caching contents in the edge is an effective approach to reduce the transmission latency and improve the quality of service for mobile users. Second, the decentralized caching allows base stations to serve mobile users without any coordination of the cloud. Meanwhile, data privacy in the edge is critical for individual users. This paper aims to jointly determine the caching and routing policy in rural areas of 5G networks in a decentralized manner and simultaneously design a proper privacy-preserving mechanism. We tackle the challenges in two progressive steps. First, we design a decentralized algorithm with the convergence guarantee. Furthermore, we enhance the developed decentralized algorithm with a privacy-preserving mechanism based on (local) differential privacy and prove its privacy guarantee. We conduct extensive numerical simulations based on real-world traces to evaluate the proposed algorithms. Results highlight significant performance improvements compared to existing baselines.

Ratio maximization has applications in areas as diverse as finance, reward shaping for reinforcement learning (RL), and the development of safe artificial intelligence, yet there has been very little exploration of RL algorithms for ratio maximization. This paper addresses this deficiency by introducing two new, model-free RL algorithms for solving cost-aware Markov decision processes, where the goal is to maximize the ratio of long-run average reward to long-run average cost. The first algorithm is a two-timescale scheme based on relative value iteration (RVI) Q-learning and the second is an actor-critic scheme. The paper proves almost sure convergence of the former to the globally optimal solution in the tabular case and almost sure convergence of the latter under linear function approximation for the critic. Unlike previous methods, the two algorithms provably converge for general reward and cost functions under suitable conditions. The paper also provides empirical results demonstrating promising performance and lending strong support to the theoretical results.

We study the economic interactions among sellers and buyers in online markets. In such markets, buyers have limited information about the product quality, but can observe the sellers' reputations which depend on their past transaction histories and ratings from past buyers. Sellers compete in the same market through pricing, while considering the impact of their heterogeneous reputations. We consider sellers with limited as well as unlimited capacities, which correspond to different practical market scenarios. In the unlimited seller capacity scenario, buyers prefer the seller with the highest reputation-price ratio. If the gap between the highest and second highest seller reputation levels is large enough, then the highest reputation seller dominates the market as a monopoly. If sellers' reputation levels are relatively close to each other, then those sellers with relatively high reputations will survive at the equilibrium, while the remaining relatively low reputation sellers will get zero market share. In the limited seller capacity scenario, we further consider two different cases. If each seller can only serve one buyer, then it is possible for sellers to set their monopoly prices at the equilibrium while all sellers gain positive market shares; if each seller can serve multiple buyers, then it is possible for sellers to set maximum prices at the equilibrium. Simulation results show that the dynamics of reputations and prices in the longer-term interactions will converge to stable states, and the initial buyer ratings of the sellers play the critical role in determining sellers' reputations and prices at the stable state.

Distributed edge caching has drawn great attention with the fast development of smart edge devices. Caching popular contents in the edge can reduce latency and improve the quality of service of edge mobile users. Meanwhile, the data privacy in the edge is critical to preserve the privacy of individual users and devices. How to jointly determine the caching and routing policy in the edge network in a distributed manner and simultaneously design the proper privacy preserving mechanism are challenging. We tackle these challenges in two progressive steps. First, we design a distributed algorithm which can achieve the global optimum. Second, we propose a privacy-preserving mechanism based on differential privacy and prove the privacy guarantee. We conduct extensive numerical simulations based on real-world requests to evaluate the performance of the proposed distributed algorithm and the privacy mechanism. Results highlight a significant improvement of the proposed distributed algorithm while only up to 10.1% of the total serving cost increased by the privacy mechanism.

To model the cross couplings of multiple topics, we develop a set of rules for opinion updates of a group of agents. The rules are used to design or assign values to the elements of weighting matrices. The cooperative and anticooperative couplings are modeled in both the inverse-proportional and proportional structures. The behaviors of opinion dynamics are analyzed using a nullspace property of the state-dependent matrix-weighted Laplacian matrices and a Lyapunov candidate. Various consensus properties of the state-dependent matrix-weighted Laplacian matrices are predicted according to the interagent network topology and interdependent topical coupling topologies.

This paper considers a mobile data collecting problem in a wireless sensor network with private residual sensor networks for the scenario in which the owners of residual sensor networks may perform dishonest behavior. The interaction between the wireless sensor network operator and the owners of residual sensor networks is modeled by a Stackelberg game which has a unique Stackelberg equilibrium. The influence of the Stackelberg equilibrium caused by the dishonest residual sensor networks owner are analyzed. An algorithm and a theoretical analysis are provided for the corresponding strategies of the operator and owners. Simulations are conducted to illustrate the difference of network performance compared with the game without dishonest residual owners.

This paper studies a data gathering problem in a wireless sensor network containing multiple private residual subnetworks. The interaction between the wireless sensor network operator and the owners of residual sub-networks is modeled by a Stackelberg game, which forms a novel framework for jointly analyzing the pricing, gathering data, and planning routes. It is shown that the game has a unique Stackelberg equilibrium at which the wireless sensor network operator sets prices to minimize total cost, while owners of residual sub-networks respond accordingly to maximize their utilities subject to their bandwidth constraints. An algorithm and theoretical analyses are provided for the corresponding strategies of the operator and owners, and validated by extensive simulations. It is demonstrated that the algorithm achieves lower network cost compared with existing data gathering strategies.

For the purposes of this paper, “gossiping” is a distributed process whose purpose is to enable the members of a group of autonomous agents to asymptotically determine, in a decentralized manner, the average of the initial values of their scalar gossip variables. This paper discusses several different deterministic protocols for gossiping which avoid deadlocks and achieve consensus under different assumptions. First considered is$T$-periodic gossiping which is a gossiping protocol which stipulates that each agent must gossip with the same neighbor exactly once every$T$time units. Among the results discussed is the fact that if the underlying graph characterizing neighbor relations is a tree, convergence is exponential at a worst case rate which is the same for all possible$T$-periodic gossip sequences associated with the graph. Many gossiping protocols are request based which means simply that a gossip between two agents will occur whenever one of the two agents accepts a request to gossip placed by the other. Three deterministic request-based protocols are discussed. Each is guaranteed to not deadlock and to always generate sequences of gossip vectors which converge exponentially fast. It is shown that worst case convergence rates can be characterized in terms of the second largest singular values of suitably defined doubly stochastic matrices.