To overcome the problem of low system resource utilization caused by the lack of exploration of environmental changes in Industrial Internet of Things (IIoT) service orchestration, while ensuring Quality of Service (QoS), we propose an Adaptive Orchestration Algorithm of Service Function Chain (AOASFC) Based on Deep Reinforcement Learning (DRL). Our approach in paper integrates joint deployment and routing information to manage system resources, thereby optimizing orchestration strategies. Furthermore, to enhance the capability of exploring environmental changes, we design a curiosity-driven module that evaluates the environmental changes before and after the DRL agent’s decision-making process, generating intrinsic rewards to guide a more comprehensive exploration process. Our approach effectively mitigates the high bias issue caused by updating value function, because we integrate Proximal Policy Optimization (PPO) with Generalized Advantage Estimation (GAE) and perform weighted averaging on multi-step estimates, optimizing temporal difference learning. In performance comparisons, we have compared DeepCoordblue(a centralized DRL orchestrator) and BSP(a greedy heuristic baseline) algorithm, AOASFC demonstrates superior performance in different traffic arrival patterns of SFC deployment scenarios: it not only improves system throughput by 15.35% and 11.64% respectively, but also keeps end-to-end latency below 50ms while significantly enhancing resource utilization.

With the rapid development of the industrial internet, the value of internal data is increasing. Federated learning, which can protect data privacy, is crucial in this context. However, it faces challenges such as device heterogeneity, data heterogeneity, and single point of failure in industrial internet scenarios. To address these, we propose the Blockchain-integrated Decentralized Asynchronous Federated Learning (BDAFL) algorithm. It leverages blockchain and the Raft consensus algorithm to decouple global model updates from a central server, using multiple servers to aggregate partial model parameters and mitigate single-point failure impacts. For device heterogeneity, BDAFL introduces a weighted mechanism based on dynamic waiting times and update frequencies. To handle data heterogeneity, it uses the Earth Mover’s Distance (EMD) to measure data distribution differences and adjusts local model parameter weights accordingly. Experimental results show that BDAFL improves model accuracy by 1.27% on MNIST, 0.99% on CIFAR-10 and 0.63% on a self-bulid bearing fault dataset compared to similar algorithms, and outperforms them in precision, recall, and F1 scores across all classification categories.

Modern reinforcement learning (RL) frequently suffers from large state and action spaces and delayed feedback in the environment. Recently, feature mapping has become an effective tool to deal with large state and action spaces, where agents use the given features to parameterize the high-dimensional value functions. However, applying feature mapping in an environment with delayed feedback is challenging and unsolved. In the delayed environment, the utilization of feature mapping relies on careful inference of the current environment state from the agent’s observation state and the action sequence. Herein, two consecutive action sequences have overlapping actions, which challenges random-theory-based theoretical regret analysis due to their intricate dependence. In this paper, a new feature-mapping-based framework is proposed to solve the constant delayed Markov Decision Processes (CDMDPs) with m-time-step-delay based on the observation state and action sequence. To address the challenge of statistical coherence brought by the overlapping actions, we design a parameterized-transition CDMDP and a novel method to decouple the statistical coherence brought by the action sequences. The algorithm’s regret bound is $\tilde O\left( {\left( {d + 2 - {\gamma ^m}} \right)\sqrt T /{{\left( {1 - \gamma } \right)}^2}} \right)$ where d is the number of features and γ is the discount factor. It also approaches the lower bound $\Omega \left( {d\sqrt T /{{\left( {1 - \gamma } \right)}^{\frac{3}{2}}}} \right)$. The theoretical analysis demonstrates that our proposed algorithm effectively alleviates the adverse impact of delayed feedback on the agent’s decision-making, and yields a regret bound that is independent of the size of the state and action spaces.

Recently, the study of linear misspecified bandits has generated intriguing implications of the hardness of learning in bandits and reinforcement learning (RL). In particular, Du et al. (2020) shows that even if a learner is given linear features in $\mathbb{R}^d$ that approximate the rewards in a bandit or RL with a uniform error of $\varepsilon$, searching for an $O(\varepsilon)$-optimal action requires pulling at least $\Omega(\exp(d))$ queries. Furthermore, Lattimore et al. (2020) show that a degraded $O(\varepsilon\sqrt{d})$-optimal solution can be learned within $\operatorname{poly}(d/\varepsilon)$ queries. Yet it is unknown whether a structural assumption on the ground-truth parameter, such as sparsity, could break $\varepsilon\sqrt{d}$ barrier. In this paper, we address this question by showing that algorithms can obtain $O(\varepsilon)$-optimal actions by querying $\tilde{O}(\exp(m\varepsilon))$ actions, where $m$ is the sparsity parameter, removing the $\exp(d)$-dependence. We further show (with an information-theoretical lower bound) that this is the best possible if one demands an error $ m^{\delta}\varepsilon$ for $0<\delta<1$. We further show that $\operatorname{poly}(m/\varepsilon)$ bounds are possible when the linear features are "good”. These results provide a nearly complete picture of how sparsity can help in misspecified bandit learning and provide a deeper understanding of when linear features are “useful” for bandit and reinforcement learning with misspecification.

Vehicle-to-Everything network enabled connected autonomous driving has been regarded as a promising solution to realize advanced autonomous driving. However, non-ideal factors in wireless communication and localization severely limit the development. In this work, we propose MoRFF, a Mobility-robust Regional Features Fusion framework for multi-terminal multi-view object detection to realize wireless cooperative perception. To conquer the limited communication bandwidth, stochastic latency, and inaccurate positioning caused by wireless links and mobility, our method features a universal two-stage detection paradigm with deep metric learning, matching the same object from different viewpoints directly on the regional feature maps, and thus helps to greatly reduce the data size to transmit. Our proposed architecture only requires image data without any additional information such as geo-positions, sensor poses, or point clouds from LiDAR, and thus conducive to the promotion of connected autonomous driving. Experimental evaluations show that the proposed algorithm successfully benefits from other viewpoints, increases the detection precision of barely visible objects by 13.42%, and achieves tenfold promotion in communication bandwidth requirements. Furthermore, the proposed algorithm is robust under various communication delays.

This article investigates the grant-free random access mechanism for massive Internet of Things (IoT) devices. By embedding the data symbols in the signature sequences, joint device activity detection and data decoding can be achieved, which, however, significantly increases the computational complexity. Coordinate descent algorithms that enjoy a low per-iteration complexity have been employed to solve this detection problem, but previous works typically employ a random coordinate selection policy which leads to slow convergence. In this article, we develop multiarmed bandit (MAB) approaches for more efficient detection via coordinate descent, which achieves a delicate tradeoff betweenexplorationandexploitationin coordinate selection. Specifically, we first propose a bandit-based strategy, i.e., Bernoulli sampling, to speed up the convergence rate of coordinate descent, by learning which coordinates will result in more aggressive descent of thenonconvex objective function. To further improve the convergence rate, an inner MAB problem is established to learn the exploration policy of Bernoulli sampling. Both convergence rate analysis and simulation results are provided to show that the proposed bandit-based algorithms enjoy faster convergence rates with a lower time complexity compared with the state-of-the-art algorithm. Furthermore, our proposed algorithms are generally applicable to different scenarios, e.g., massive random access with low-precision analog-to-digital converters (ADCs).

Graph neural networks (GNNs) are powerful tools for learning from graph data and are widely used in various applications such as social network recommendation, fraud detection, and graph search. The graphs in these applications are typically large, usually containing hundreds of millions of nodes. Training GNN models on such large graphs efficiently remains a big challenge. Despite a number of sampling-based methods have been proposed to enable mini-batch training on large graphs, these methods have not been proved to work on truly industry-scale graphs, which require GPUs or mixed CPU-GPU training. The state-of-the-art sampling-based methods are usually not optimized for these real-world hardware setups, in which data movement between CPUs and GPUs is a bottleneck. To address this issue, we propose Global Neighborhood Sampling that aims at training GNNs on giant graphs specifically for mixed CPU-GPU training. The algorithm samples a global cache of nodes periodically for all mini-batches and stores them in GPUs. This global cache allows in-GPU importance sampling of mini-batches, which drastically reduces the number of nodes in a mini-batch, especially in the input layer, to reduce data copy between CPU and GPU and mini-batch computation without compromising the training convergence rate or model accuracy. We provide a highly efficient implementation of this method and show that our implementation outperforms an efficient node-wise neighbor sampling baseline by a factor of 2× ~ 4× on giant graphs. It outperforms an efficient implementation of LADIES with small layers by a factor of 2× ~ 14× while achieving much higher accuracy than LADIES. We also theoretically analyze the proposed algorithm and show that with cached node data of a proper size, it enjoys a comparable convergence rate as the underlying node-wise sampling method.

This paper considers the grant-free random access scheme in IoT networks with a massive number of devices. By embedding the data symbols in the signature sequences, joint device activity detection, and data decoding can be achieved, which, however, significantly increases the computational complexity. Coordinate descent algorithms, with a low per-iteration complexity, have been employed to solve the detection problem, but previous works typically employ a random coordinate selection policy which leads to slow convergence. This paper develops a bandit based strategy, i.e., bandit sampling, to speed up the convergence of coordinate descent. We exploit a multi-armed bandit algorithm to learn which coordinates will result in more aggressive descent of the objective function. Both convergence rate analysis and simulation results are provided to show that the proposed algorithm enjoys a faster convergence rate with a lower time complexity compared with the state-of-the-art algorithm.

Social Internet of Things has recently become a promising paradigm for augmenting the capability of humans and devices connected in the networks to provide services. In social Internet of Things network, crowdsourcing that collects the intelligence of the human crowd has served as a powerful tool for data acquisition and distributed computing. To support critical applications (e.g., a recommendation system and assessing the inequality of urban perception), in this article, we shall focus on the collaborative ranking problems for user preference prediction from crowdsourced pairwise comparisons. Based on the Bradley--Terry--Luce (BTL) model, a maximum likelihood estimation (MLE) is proposed via low-rank approach in order to estimate the underlying weight/score matrix, thereby predicting the ranking list for each user. A novel regularized formulation with the smoothed surrogate of elementwise infinity norm is proposed in order to address the unique challenge of the coupled the non-smooth elementwise infinity norm constraint and non-convex low-rank constraint in the MLE problem. We solve the resulting smoothed rank-constrained optimization problem via developing the Riemannian trust-region algorithm on quotient manifolds of fixed-rank matrices, which enjoys the superlinear convergence rate. The admirable performance and algorithmic advantages of the proposed method over the state-of-the-art algorithms are demonstrated via numerical results. Moreover, the proposed method outperforms state-of-the-art algorithms on large collaborative filtering datasets in both success rate of inferring preference and normalized discounted cumulative gain.

This paper concerns the problem of demixing a series of source signals from the sum of bilinear measurements. This problem spans diverse areas such as communication, imaging processing, machine learning, etc. However, semidefinite programming for blind demixing is prohibitive to large-scale problems due to high computational complexity and storage cost. Although several efficient algorithms have been developed recently that enjoy the benefits of fast convergence rates and even regularization free, they still call for spectral initialization. To find simple initialization approach that works equally well as spectral initialization, we propose to solve blind demixing problem via Wirtinger flow with random initialization, which yields a natural implementation. To reveal the efficiency of this algorithm, we provide the global convergence guarantee concerning randomly initialized Wirtinger flow for blind demixing. Specifically, it shows that with sufficient samples, the iterates of randomly initialized Wirtinger flow can enter a local region that enjoys strong convexity and strong smoothness within a few iterations at the first stage. At the second stage, iterates of randomly initialized Wirtinger flow further converge linearly to the ground truth.

Internet-of-Things (IoT) networks are envisioned to typically include a massive number of devices with sporadic and low-latency uplink service needs. This paper presents a blind demixing approach to support the data recovery of multiple simultaneous and unscheduled device transmissions without a priori channel state information (CSI). The proposed joint receiver leverages the group sparse bilinear characteristics of the underlying problem that involves active device detection and data recovery. We exploit the manifold geometry of rank-one matrices in the lifted bilinear equation and apply smoothed ℓ1/ℓ2-norm to induce the group sparsity for active device detection. We further develop a smoothed Riemannian algorithm to solve the sparse blind demixing optimization problem. Numerical results demonstrate the algorithmic advantage and desirable performance of the proposed algorithm.

Massive connectivity has become a critical requirement for Internet-of-Things (IoT) networks, where a large number of devices need to connect to an access-point sporadically. Moreover, low-latency communication and sporadic device traffic are essential to support intelligent services in IoT networks. In this paper, to support low-latency communication for massive devices with sporadic traffic, we present a sparse blind demixing to simultaneously detect the active devices and decode multiple source signals without a priori channel state information in multi- in-multi-out (MIMO) networks. To address the unique challenges of bilinear measurements and sporadic device activity detection, we recast the estimation problem as a sparse and low-rank optimization problem via matrix lifting. We further propose a difference-of-convex-functions (DC) representation for the rank function to guarantee the exact rank constraint, followed by ignoring the non-convex group sparse function. This is achieved by exploiting the difference between nuclear norm and the convex Ky Fan k- norm for a rank function representation. We then develop an efficient DC algorithm to solve the resulting non-convex DC program without regularization parameter. Numerical results demonstrate that the proposed DC approach is able to exactly recover the ground truth signals with reduced sample sizes, as well as achieve better performance against noise compared with the existing convex methods.

In next-generation wireless networks, low-latency communication is critical to support emerging diversified applications, e.g., tactile Internet and virtual reality. In this paper, a novel blind demixing approach is developed to reduce the channel signaling overhead, thereby supporting low-latency communication. Specifically, we develop a low-rank approach to recover the original information only based on the single observed vector without any channel estimation. To address the unique challenges of multiple non-convex rank-one constraints, the quotient manifold geometry of the product of complex symmetric rank-one matrices is exploited. This is achieved by equivalently reformulating the original problem that uses complex asymmetric matrices to the one that uses Hermitian positive semidefinite matrices. We further generalize the geometric concepts of the complex product manifold via element-wise extension of the geometric concepts of the individual manifolds. The scalable Riemannian optimization algorithms, i.e., the Riemannian gradient descent algorithm and the Riemannian trust-region algorithm, are then developed to solve the blind demixing problem efficiently with low iteration complexity and low iteration cost. The statistical analysis shows that the Riemannian gradient descent with spectral initialization is guaranteed to linearly converge to the ground truth signals provided sufficient measurements. In addition, the Riemannian trust-region algorithm is provable to converge to an approximate local minimum from the arbitrary initialization point. Numerical experiments have been carried out in settings with different types of encoding matrices to demonstrate the algorithmic advantages, performance gains, and sample efficiency of the Riemannian optimization algorithms.

We consider the problem of demixing a sequence of source signals from the sum of bilinear measurements. It is a generalized mathematical model of blind demixing with deconvolution, which has wide applications in communication, image processing and dictionary learning, etc. However, state-of-art algorithms for blind demixing either fail to scale to large problem sizes or require proper regularization with tedious algorithmic parameters for optimality guarantees. To address the limitations of exiting methods, we propose a provable nonconvex demixing procedure via Wirtinger flow, much like vanilla gradient descent, to harness the benefits of regularization free, fast convergence rate, and optimality guarantees. This is achieved by exploiting the benign geometry of blind demixing, thereby revealing that Wirtinger flow enforces the iterates in the region of strong convexity and qualified level of smoothness.

In the next generation wireless networks, low-latency communication is critical to support emerging diversified applications, e.g., Tactile Internet and Virtual Reality. In this paper, a novel blind demixing approach is developed to reduce the channel signaling overhead, thereby supporting low-latency communication. Specifically, we develop a low-rank approach to recover the original information only based on a single observed vector without any channel estimation. Unfortunately, this problem turns out to be a highly intractable non-convex optimization problem due to the multiple non-convex rank-one constraints. To address the unique challenges, the quotient manifold geometry of product of complex asymmetric rank-one matrices is exploited by equivalently reformulating original complex asymmetric matrices to the Hermitian positive semidefinite matrices. We further generalize the geometric concepts of the complex product manifolds via element-wise extension of the geometric concepts of the individual manifolds. A scalable Riemannian trust-region algorithm is then developed to solve the blind demixing problem efficiently with fast convergence rates and low iteration cost. Numerical results will demonstrate the algorithmic advantages and admirable performance of the proposed algorithm compared with the state-of-art methods.