Existing mobile edge computing (MEC) systems are facing the challenges of limited resources and highly dynamic network environments. How to allocate resources to maintain the efficiency and timeliness of data and tasks is still an open issue. To address this problem, we propose a novel framework for UAV-assisted MEC systems using federated multi-agent reinforcement learning. First, we formulate a joint optimization problem as a multi-agent Markov decision process by jointly minimizing the average age of information and maximizing the number of recent tasks. Second, we design a novel scheduling algorithm for online collaborative resources by adopting multiple agents to learn and make decisions in accordance with the overall interests through federal learning. Finally, an experience replay mechanism for the internal experience pool is introduced to further improve learning efficiency. Experimental results show that our proposed algorithm is superior to the recent typical reinforcement learning-based algorithms. It not only has higher efficiency in task processing and data freshness, but also has more stable performance and adaptability across diverse experimental conditions.

Most existing methods for visual domain adaptation need to convert high-order tensors into one-order high-dimensional vectors through naive vectorization operations. However, they not only destroy the internal spatial structure within the original high-order tensors, but also result in exponentially increasing model parameters. To address these problems, we propose a novel method for visual domain adaptation by representing tensorial features in tensor-train subspace in this paper. Specifically, we firstly provide a theoretical deduction by constructing a tensor-train subspace and proving its linearity and left-orthogonality. Secondly, to extract common tensorial features between source and target domains, we formulate the visual domain adaptation problem into an optimization problem that models the aforementioned common tensor-train subspace between two domains, as well as their corresponding projections. Thirdly, we design a tensor-train subspace representation algorithm (TTSR) to solve the multiple variables optimization problem by optimizing its sub-problems iteratively, so as to process high-order tensorial features. Finally, we evaluate the performance of our proposed TTSR algorithm by conducting extensive experiments on three popular public datasets. The experimental results demonstrate that the TTSR algorithm can improve the classification accuracy of unlabeled target domain than that of baseline algorithms.

The rapid development of edge computing and the Industrial Internet of Things have facilitated near real-time optimization of compute-intensive industrial tasks. Mobile edge computing (MEC) and device-to-device (D2D) offloading are promising ways to achieve near-real-time optimization. In this article, We propose a D2D-assisted MEC computing offloading framework by using deep reinforcement Learning (DRL) with entropy and attention mechanism (DMOEA). DMOEA considers interactions among related entities, including horizontal device-to-device collaboration and vertical device-to-edge offloading. Then, a DRL-based model with multi-actor single-critic structure is designed to solve the offloading strategy. In addition, to further improve efficiency, an attention mechanism is introduced to adapt dynamic changes in network and enhance the exploration ability. The experimental results show that the proposed framework can obtain a fast convergence rate and small oscillation amplitude and also can effectively reduce latency.

Wireless-powered mobile-edge computing (MEC) emerges as a crucial component in the Internet of Things (IoTs). It can cope with the fundamental performance limitations of low-power networks, such as wireless sensor networks or mobile networks. Although computation offloading and resource allocation in MEC have been studied with different optimization objectives, performance optimization in larger-scale systems still needs to be further improved. More importantly, energy efficiency is also a key issue as well as computation offloading and resource allocation for wireless-powered MEC. In this article, we investigate the joint optimization of computation rate and energy consumption under limited resources, and propose an online offloading model to search for the asymptotically optimal offloading and resource allocation strategy. First, the joint optimization problem is modeled as a mixed integer programming (MIP) problem. Second, a deep reinforcement learning (DRL)-based method, energy efficiency computation offloading using probabilistic action generating (ECOPG), is designed to generate the joint optimization policy for computation offloading and resource allocation. Finally, to avoid the curse of dimensionality in large network scales, an action exploration mechanism based on probability is introduced to accelerate the convergence rate by targeted sampling and dynamic experience replay. The experimental results demonstrate that the proposed methods significantly outperform other DRL-based methods in energy consumption, and gain better computation rate and execution efficiency at the same time. With the expansion of the network scale, the improvements become more apparent.

The low-rank matrix completion problem is fundamental in both machine learning and computer vision fields with many important applications, such as recommendation system, motion capture, face recognition, and image inpainting. In order to avoid solving the rank minimization problem which is NP-hard, several surrogate functions of the rank have been proposed in the literature. However, the matrix restored from the optimization problem based on the existing surrogate functions seriously deviates from the original one. In this paper, we first design a new non-convex Schatten capped$p$norm which generalizes several existing non-convex matrix norms and balances between the rank and the nuclear norm of the matrix. Then, a matrix completion method based on the Schatten capped$p$norm is proposed by exploiting the framework of the alternating direction method of multipliers. Meanwhile, the Schatten capped$p$norm regularized least squares subproblem is analyzed in detail and is solved explicitly. Finally, we evaluate the performance of the proposed matrix completion method based on extensive experiments in the field of image inpainting. All the experimental results demonstrate that the proposed method can indeed improve the accuracy of matrix completion compared with the existing methods.

The goal of cross-domain sentiment classification is to utilise useful information in the source domain to help classify sentiment polarity in the target domain, which has a large number of unlabelled data. Most of the existing methods focus on extracting the invariant features between two domains. But they cannot make better use of the unlabelled data in the target domain. To solve this problem, we present a deep transfer learning mechanism (DTLM) for fine-grained cross-domain sentiment classification. DTLM provides a transfer mechanism to better transfer sentiment across domains by incorporating BERT(Bidirextional Encoder Representations from Transformers) and KL (Kullback-Leibler) divergence. We introduce BERT as a feature encoder to map the text data of different domains into a shared feature space. Then, we design a domain adaptive model using KL divergence to eliminate the difference of feature distribution between the source domain and target domain. In addition, we introduce the entropy minimisation and consistency regularisation to process unlabelled samples in the target domain. Extensive experiments on the datasets from YelpAspect, SemEval 2014 task 4 and Twitter not only demonstrate the effectiveness of our proposed method but also provide a better way for cross-domain sentiment classification.

Summary It is well known that virtual network (VN) embedding (VNE) aims to solve how to efficiently allocate physical resources to a VN. However, this issue has been proved to be an NP‐hard problem. Besides, as most of the existing approaches are based on heuristic algorithms, which is easy to fall into local optimal. To address the challenge, we formalize the problem as a mixed integer programming problem and propose a novel VNE method based on reinforcement learning in this article. And to solve the problem, we introduce a pointer network to generate virtual node mapping strategies through an attention mechanism, and design a reward function related to link resource consumption to build the connection between node mapping and link mapping stages of VNE. In addition, we present a policy gradient optimization mechanism to leverage the reward information obtained from the sampled solutions, and design an active search based process to automatically update the parameters of the neural network and to obtain near‐optimal embedding solution. The experimental results show that the proposed method can improve the performance in average physical node utilization and long‐term revenue to cost ratio comparing than that of the existing models.

Internet of Things (IoT) can be used to promote many advanced applications by utilizing the sensed data collected from various settings. To reduce the energy consumption of IoT devices, and to extend the lifetime of network, the sensed data are usually compressed before their transmission through compressed sensing theory. By reconstructing the sensed data at the edge of network with more resourceful devices, such as laptops and servers, the intensive computation and energy consumption of the IoT nodes could be effectively offloaded. However, most of the existing data collection schemes are limited in their scalability, because the unified data reconstruction models of them are not suitable for large-scale surveillance scenarios. In our proposed scheme, the whole network is first partitioned into a number of data correlated clusters based on spatial correlation. Then, a data collection tree is built to collect the compressed data in a hybrid mode. Finally, the data reconstruction problem is modelled as a group sparse problem and solved through using an alternating direction method of multiplier-based algorithm. The performance of data communication and reconstruction of the proposed scheme is evaluated through experiments with real data set. The experimental results show that the proposed scheme can indeed lower the amount of data transmission, prolong the network life, and achieve a higher level of accuracy in data collection compared to existing data collection schemes.

Social networks are critical in terms of information or malware propagation. However, how to contain the spreading of malware in social networks is still an open and challenging issue. In this paper, we propose a novel defending method through big data based influence modeling. We first establish a social interaction graph based on big data sets of the studied object. Based on the graph, we are able to measure direct influence of individuals by computing each node's strength, which includes the degree of the node and the total number of messages sent by each user to her friends. Then, we design an algorithm to construct influence spreading tree using the breadth first search strategy, and measure indirect influence of individuals by traversing the tree. We identify the top k influential nodes among all the nodes via the social influence strength, and propose an immunization algorithm to defend social networks against various attacks. The extensive experiments show that influence can spread easily in social networks, and the greater the influence of initial spread node is, the more impact it is on the malware propagation in social networks. The proposed method provides an effective solution to the prevention of malware or malicious messages propagation in social networks.

The rapid proliferation of smart mobile devices has significantly enhanced the popularization of the cyber-physical social network, where users actively publish data with sensitive information. Adversaries can easily obtain these data and launch continuous attacks to breach privacy. However, existing works only focus on either location privacy or identity privacy with a static adversary. This results in privacy leakage and possible further damage. Motivated by this, we propose a hybrid privacy-preserving scheme, which considers both location and identity privacy against a dynamic adversary. We study the privacy protection problem as the tradeoff between the users aiming at maximizing data utility with high-level privacy protection while adversaries possessing the opposite goal. We first establish a game-based Markov decision process model, in which the user and the adversary are regarded as two players in a dynamic multistage zero-sum game. To acquire the best strategy for users, we employ a modified state-action-reward-state-action reinforcement learning algorithm. Iteration times decrease because of cardinality reduction from n to 2, which accelerates the convergence process. Our extensive experiments on real-world data sets demonstrate the efficiency and feasibility of the propose method.

Summary Efficient and fair resource allocation for multitudinous virtual networks running cloud‐based applications is crucial to archive dynamic resources multi‐tenancy in cloud computing. In order to solve the problem, we propose a novel virtual network embedding (VNE) algorithm to increase revenue and utilization of substrate network as well as to improve acceptance fairness of virtual networks. First, we present a virtual topology pre‐transformation mechanism leveraging reusable technology to reduce topology difference and achieve acceptance fairness. Then, because of the Non‐deterministic polynomial‐time (NP)‐hard characteristics of VNE, we model the problem as an integer linear programming problem and solve the VNE problem with a discrete particle swarm optimization‐based algorithm. The operations and parameters of particles are well redefined according to the VNE context. Finally, an incentive convergence mechanism is proposed to reduce mapping complexity, which can be used to accelerate convergence and to save more bandwidth by exploiting individual candidate nodes' lists. Simulation results prove that our proposed method is superior to the existing similar algorithms in terms of physical resource utilization, acceptance fairness, revenue/cost ratio, and searching efficiency. Copyright © 2016 John Wiley & Sons, Ltd.

Bluetooth networks are envisioned to provide many promising services and applications. Meanwhile, Bluetooth networks are also increasingly becoming the target of worms. Many emerging worms can utilize the proximity of devices to propagate in a distributed manner, resulting in modeling on worm propagation substantially more challenging. In this paper, we propose an efficient Worm Propagation Modeling scheme, WPM for short. WPM utilizes the two-dimensional (2D) cellular automata to simulate the dynamics of the worm propagation process from a single node to the entire Bluetooth network. The WPM scheme integrates infection factor, which evaluates the spread degree of infected nodes, and resistance factor, which offers resistance evaluation towards susceptible nodes. Moreover, the epidemic state of each node is classified into five types in the WPM scheme, including susceptible, exposed, infected, diagnosed, and recovered. The effectiveness and rationality of the proposed model are validated through extensive simulations.

Greedy routing protocols provide a scalable and cost effective solution for routing packets in mobile ad-hoc networks (MANETs). However, such routing protocols can not be used in the networks where holes exist. We propose a virtual ring-based hole avoiding routing protocol (VRHAR) in MANETs where holes exist. Simulation studies show that the proposed protocol can improve the routing performance more effectively than existing greedy routing protocols.

Node mobility is one of the most important factors that may degrade network performance and restrict network scalability in mobile ad hoc networks. An effective way to reduce the impact of node mobility is to select long lifetime routing paths in the network. We propose a link lifetime-based segment-by-segment routing protocol (LL-SSR) in mobile ad hoc networks, where each node maintains a routing table for its k-hop region. Simulation studies show that LL-SSR has better scalability and higher packet delivery ratio when compared with GPSR.