ABSTRACT The integration of artificial intelligence (AI) and the Internet of Things (IoT) has given rise to AIoT systems. However, the accelerated proliferation of AIoT ecosystems introduces substantial communication and networking difficulties. Mobile edge computing (MEC) has emerged as a promising solution to address time‐sensitive computational demands. This paper addresses the improvement of information freshness, a critical enabler for real‐time decision‐making in dynamic, resource‐constrained AIoT environments. We formulate the UAV‐assisted MEC system as a Markov decision process with the objective of minimizing the age of information (AoI). To this end, we propose an adaptive federated multi‐agent soft actor‐critic framework for resource scheduling. This framework leverages maximum entropy to enable robust exploration and incorporates an innovative adaptive federated learning mechanism by adopting a trainable network to predict the parameter matrix of federated learning. This enables federated learning to better promote knowledge sharing among multiple agents, thereby accelerating convergence and improving performance. Experimental results validate that our approach significantly outperforms state‐of‐the‐art reinforcement learning based algorithms in AoI minimization, stability enhancement, and task completion volume improvement, thereby advancing the safeguarding of communication and networking in AIoT systems.

With the rapid increase in computing demand for the Internet of Vehicles (IoV), the limited on-board computing resources have gradually become a bottleneck restricting the efficient processing of tasks. Unmanned aerial vehicles (UAVs), with their high mobility and flexible deployment capabilities, have become an ideal platform for assisting IoV task offloading. However, the dynamic environment and the mobility of vehicles pose significant challenges for task offloading and resource allocation in UAV-assist IoV networks. This paper proposes a soft asynchronous actor-critic (SAAC) framework with vehicle trajectory prediction based on deep reinforcement learning (DRL) for UAV-assisted IoV task offloading. Firstly, in order to solve the problem of poor stability of traditional multi-agent deep reinforcement learning algorithms in dynamic environments, this paper combines the asynchronous training of multi-agent systems with the soft update and target network mechanism to improve the anti-interference and stability of the algorithm. Secondly, a module based on a recurrent neural network (RNN) integrated with a sliding window mechanism is proposed to address the uncertainty arising from vehicle mobility. This module enables precise prediction of vehicle trajectories, thereby facilitating forward-looking flight strategy planning for UAVs and reducing task transmission delays. Simulation results show that the proposed algorithm is effective, therefore this paper provides a new framework for the UAV-assisted IoV task offloading.

Statistical heterogeneity poses a formidable challenge to federated learning (FL), resulting in inconsistent training objectives and serious biases in local feature representations among FL clients. Recent research on neural collapse has identified an optimal simplex equiangular tight frame (ETF) for the structure of mean class feature vectors and classifier vectors under a balanced data distribution. However, the amount of data pertaining to each class in every FL client, as well as the distribution of existing classes across all FL clients, are both imbalanced. Therefore, we first confirm the occurrence of minority collapse in imbalanced FL scenarios through experimentation. Then, we propose the FedAMA algorithm to mitigate its adverse impact by enforcing adaptive pushing forces to minority classes in the global training stage and re-adjusting the structure of the mean feature and classifier vectors in the local fine-tuning stage. Finally, we theoretically analyze the global convergence of FedAMA and establish its upper bound. Extensive experiments have also been carried out to demonstrate that FedAMA outperforms existing algorithms in terms of global and personalized model performance, particularly in highly heterogeneous settings.

Existing mobile edge computing (MEC) systems are facing the challenges of limited resources and highly dynamic network environments. How to allocate resources to maintain the efficiency and timeliness of data and tasks is still an open issue. To address this problem, we propose a novel framework for UAV-assisted MEC systems using federated multi-agent reinforcement learning. First, we formulate a joint optimization problem as a multi-agent Markov decision process by jointly minimizing the average age of information and maximizing the number of recent tasks. Second, we design a novel scheduling algorithm for online collaborative resources by adopting multiple agents to learn and make decisions in accordance with the overall interests through federal learning. Finally, an experience replay mechanism for the internal experience pool is introduced to further improve learning efficiency. Experimental results show that our proposed algorithm is superior to the recent typical reinforcement learning-based algorithms. It not only has higher efficiency in task processing and data freshness, but also has more stable performance and adaptability across diverse experimental conditions.

Most existing methods for visual domain adaptation need to convert high-order tensors into one-order high-dimensional vectors through naive vectorization operations. However, they not only destroy the internal spatial structure within the original high-order tensors, but also result in exponentially increasing model parameters. To address these problems, we propose a novel method for visual domain adaptation by representing tensorial features in tensor-train subspace in this paper. Specifically, we firstly provide a theoretical deduction by constructing a tensor-train subspace and proving its linearity and left-orthogonality. Secondly, to extract common tensorial features between source and target domains, we formulate the visual domain adaptation problem into an optimization problem that models the aforementioned common tensor-train subspace between two domains, as well as their corresponding projections. Thirdly, we design a tensor-train subspace representation algorithm (TTSR) to solve the multiple variables optimization problem by optimizing its sub-problems iteratively, so as to process high-order tensorial features. Finally, we evaluate the performance of our proposed TTSR algorithm by conducting extensive experiments on three popular public datasets. The experimental results demonstrate that the TTSR algorithm can improve the classification accuracy of unlabeled target domain than that of baseline algorithms.

The rapid development of edge computing and the Industrial Internet of Things have facilitated near real-time optimization of compute-intensive industrial tasks. Mobile edge computing (MEC) and device-to-device (D2D) offloading are promising ways to achieve near-real-time optimization. In this article, We propose a D2D-assisted MEC computing offloading framework by using deep reinforcement Learning (DRL) with entropy and attention mechanism (DMOEA). DMOEA considers interactions among related entities, including horizontal device-to-device collaboration and vertical device-to-edge offloading. Then, a DRL-based model with multi-actor single-critic structure is designed to solve the offloading strategy. In addition, to further improve efficiency, an attention mechanism is introduced to adapt dynamic changes in network and enhance the exploration ability. The experimental results show that the proposed framework can obtain a fast convergence rate and small oscillation amplitude and also can effectively reduce latency.

Wireless-powered mobile-edge computing (MEC) emerges as a crucial component in the Internet of Things (IoTs). It can cope with the fundamental performance limitations of low-power networks, such as wireless sensor networks or mobile networks. Although computation offloading and resource allocation in MEC have been studied with different optimization objectives, performance optimization in larger-scale systems still needs to be further improved. More importantly, energy efficiency is also a key issue as well as computation offloading and resource allocation for wireless-powered MEC. In this article, we investigate the joint optimization of computation rate and energy consumption under limited resources, and propose an online offloading model to search for the asymptotically optimal offloading and resource allocation strategy. First, the joint optimization problem is modeled as a mixed integer programming (MIP) problem. Second, a deep reinforcement learning (DRL)-based method, energy efficiency computation offloading using probabilistic action generating (ECOPG), is designed to generate the joint optimization policy for computation offloading and resource allocation. Finally, to avoid the curse of dimensionality in large network scales, an action exploration mechanism based on probability is introduced to accelerate the convergence rate by targeted sampling and dynamic experience replay. The experimental results demonstrate that the proposed methods significantly outperform other DRL-based methods in energy consumption, and gain better computation rate and execution efficiency at the same time. With the expansion of the network scale, the improvements become more apparent.

The low-rank matrix completion problem is fundamental in both machine learning and computer vision fields with many important applications, such as recommendation system, motion capture, face recognition, and image inpainting. In order to avoid solving the rank minimization problem which is NP-hard, several surrogate functions of the rank have been proposed in the literature. However, the matrix restored from the optimization problem based on the existing surrogate functions seriously deviates from the original one. In this paper, we first design a new non-convex Schatten capped$p$norm which generalizes several existing non-convex matrix norms and balances between the rank and the nuclear norm of the matrix. Then, a matrix completion method based on the Schatten capped$p$norm is proposed by exploiting the framework of the alternating direction method of multipliers. Meanwhile, the Schatten capped$p$norm regularized least squares subproblem is analyzed in detail and is solved explicitly. Finally, we evaluate the performance of the proposed matrix completion method based on extensive experiments in the field of image inpainting. All the experimental results demonstrate that the proposed method can indeed improve the accuracy of matrix completion compared with the existing methods.

In this work, we propose a deep reinforcement learning (DRL) algorithm DoubleJump which can master the game of Amazons efficiently. To address the bottleneck problem of sparse supervision signal in DRL, we split the neural network into rule network and skill network, using huge amounts of inexpensive data with game rule information and scarce data containing game skill information to train two networks respectively. Besides, we split the three sub-actions of each action into independent states during Monte-Carlo tree search (MCTS), to improve the probability of finding the global optimal state and reduce the average branching factor. The experimental results show our algorithm reaches about 130:70 in the zero-knowledge learning compared with the AlphaGo Zero algorithm, significantly improves the learning speed, and then alleviates the severe dependence on computing resources.

Summary It is well known that virtual network (VN) embedding (VNE) aims to solve how to efficiently allocate physical resources to a VN. However, this issue has been proved to be an NP‐hard problem. Besides, as most of the existing approaches are based on heuristic algorithms, which is easy to fall into local optimal. To address the challenge, we formalize the problem as a mixed integer programming problem and propose a novel VNE method based on reinforcement learning in this article. And to solve the problem, we introduce a pointer network to generate virtual node mapping strategies through an attention mechanism, and design a reward function related to link resource consumption to build the connection between node mapping and link mapping stages of VNE. In addition, we present a policy gradient optimization mechanism to leverage the reward information obtained from the sampled solutions, and design an active search based process to automatically update the parameters of the neural network and to obtain near‐optimal embedding solution. The experimental results show that the proposed method can improve the performance in average physical node utilization and long‐term revenue to cost ratio comparing than that of the existing models.

Internet of Things (IoT) can be used to promote many advanced applications by utilizing the sensed data collected from various settings. To reduce the energy consumption of IoT devices, and to extend the lifetime of network, the sensed data are usually compressed before their transmission through compressed sensing theory. By reconstructing the sensed data at the edge of network with more resourceful devices, such as laptops and servers, the intensive computation and energy consumption of the IoT nodes could be effectively offloaded. However, most of the existing data collection schemes are limited in their scalability, because the unified data reconstruction models of them are not suitable for large-scale surveillance scenarios. In our proposed scheme, the whole network is first partitioned into a number of data correlated clusters based on spatial correlation. Then, a data collection tree is built to collect the compressed data in a hybrid mode. Finally, the data reconstruction problem is modelled as a group sparse problem and solved through using an alternating direction method of multiplier-based algorithm. The performance of data communication and reconstruction of the proposed scheme is evaluated through experiments with real data set. The experimental results show that the proposed scheme can indeed lower the amount of data transmission, prolong the network life, and achieve a higher level of accuracy in data collection compared to existing data collection schemes.

Social networks are critical in terms of information or malware propagation. However, how to contain the spreading of malware in social networks is still an open and challenging issue. In this paper, we propose a novel defending method through big data based influence modeling. We first establish a social interaction graph based on big data sets of the studied object. Based on the graph, we are able to measure direct influence of individuals by computing each node's strength, which includes the degree of the node and the total number of messages sent by each user to her friends. Then, we design an algorithm to construct influence spreading tree using the breadth first search strategy, and measure indirect influence of individuals by traversing the tree. We identify the top k influential nodes among all the nodes via the social influence strength, and propose an immunization algorithm to defend social networks against various attacks. The extensive experiments show that influence can spread easily in social networks, and the greater the influence of initial spread node is, the more impact it is on the malware propagation in social networks. The proposed method provides an effective solution to the prevention of malware or malicious messages propagation in social networks.

Social networks have become an indispensable part of modern life. Signed networks, a class of social network with positive and negative edges, are becoming increasingly important. Many social networks have adopted the use of signed networks to model like (trust) or dislike (distrust) relationships. Consequently, how to rank nodes from positive and negative views has become an open issue of social network data mining. Traditional ranking algorithms usually separate the signed network into positive and negative graphs so as to rank positive and negative scores separately. However, much global information of signed network gets lost during the use of such methods, e.g., the influence of a friend’s enemy. In this paper, we propose a novel ranking algorithm that computes a positive score and a negative score for each node in a signed network. We introduce a random walking model for signed network which considers the walker has a negative or positive emotion. The steady state probability of the walker visiting a node with negative or positive emotion represents the positive score or negative score. In order to evaluate our algorithm, we use it to solve sign prediction problem, and the result shows that our algorithm has a higher prediction accuracy compared with some well-known ranking algorithms.

Summary Efficient and fair resource allocation for multitudinous virtual networks running cloud‐based applications is crucial to archive dynamic resources multi‐tenancy in cloud computing. In order to solve the problem, we propose a novel virtual network embedding (VNE) algorithm to increase revenue and utilization of substrate network as well as to improve acceptance fairness of virtual networks. First, we present a virtual topology pre‐transformation mechanism leveraging reusable technology to reduce topology difference and achieve acceptance fairness. Then, because of the Non‐deterministic polynomial‐time (NP)‐hard characteristics of VNE, we model the problem as an integer linear programming problem and solve the VNE problem with a discrete particle swarm optimization‐based algorithm. The operations and parameters of particles are well redefined according to the VNE context. Finally, an incentive convergence mechanism is proposed to reduce mapping complexity, which can be used to accelerate convergence and to save more bandwidth by exploiting individual candidate nodes' lists. Simulation results prove that our proposed method is superior to the existing similar algorithms in terms of physical resource utilization, acceptance fairness, revenue/cost ratio, and searching efficiency. Copyright © 2016 John Wiley & Sons, Ltd.