Abstract The Spark computing framework provides an efficient solution to address the major requirements of big data processing, but data partitioning and job scheduling in the Spark framework are the two major bottlenecks that limit Spark’s performance. In the Spark Shuffle phase, the data skewing problem caused by unbalanced data partitioning leads to the problem of increased job completion time. In response to the above problems, a balanced partitioning strategy for intermediate data is proposed in this article, which considers the characteristics of intermediate data, establishes a data skewing model and proposes a dynamic partitioning algorithm. In Spark heterogeneous clusters, because of the differences in node performance and task requirements, the default task scheduling algorithm cannot complete scheduling efficiently, which leads to low system task processing efficiency. In order to deal with the above problems, an efficient job scheduling strategy is proposed in this article, which integrates node performance and task requirements, and proposes a task scheduling algorithm using greedy strategy. The experimental results prove that the dynamic partitioning algorithm for intermediate data proposed in this article effectively alleviates the problem that data skew leads to the decrease of system task processing efficiency and shortens the overall task completion time. The efficient job scheduling strategy proposed in this article can efficiently complete the job scheduling tasks under heterogeneous clusters, allocate jobs to nodes in a balanced manner, decrease the overall job completion time and increase the system resource utilization.

Abstract Radio frequency identification (RFID) is widely applied due to its fast identification speed and non-contact detection. However, the identification process of RFID tags is susceptible to interference from other tags and environmental factors, resulting in inaccurate identification data. To overcome these problem, this paper proposes an improved sensing data cleaning scheme for object localization in edge computing environment. In tag level data cleaning, we use adaptive sliding window and further consider dynamic tags and read rate in continuous reading cycle to adjust the window size timely and appropriately. In the reader level data cleaning, we estimate the tag number based on Chebyshev’s inequality through Markov chain for cyclic control and optimize different time slot lengths to improve the recognition rate. We build an edge computing environment and combine the proposed tag-level cleaning method and reader-level cleaning method to form a comprehensive RFID data cleaning process. Comparative experimental results show that the RFID data cleaning method proposed in this paper can effectively reduce redundant and missing data and improve the accuracy of tag recognition.

Computation offloading in UAV-assisted MEC has been an interesting research issue. However, in the temporary hotspot areas with traffic explosion, effective solutions that support user privacy security and higher quality of service (QoS) are still lacking. Compared with the previous works, we first constructed a distributed federated learning-based computation offloading method for user privacy security to maximize the normalized weighted sum of task response delay and user energy consumption. We formulated a computation offloading problem considering the UAV hover position, resource allocation, computation offloading decision, and user transmission power allocation, which is a challenging time-series mixed integer nonlinear programming problem. Then, we proposed an improved federated deep deterministic policy gradient (IF-DDPG) algorithm to address it effectively, which improved the traditional federated DDPG algorithm in empirical pooling and exploring noise. Finally, experiment results show that the IF-DDPG can significantly decrease the task response delay and user energy consumption.

Intelligent Connected Vehicles (ICVs) need to obtain real-time traffic data from nearby ICVs or remote content providers to ensure safe driving. However, providers are hesitant to share their data due to privacy and benefits concerns. To ensure privacy while improving efficiency of obtaining data, we proposed smart contract-based data sharing among ICVs, and content delivery between ICVs and remote content provider. To solve low willingness to vehicles due to untrustworthy third-party platforms, we use smart contracts to implement access control during data upload and transaction. Then, we propose a one-to-many sharing model based on Stackelberg game to model the interaction between consumers and owners. Consumers adjust their reward strategies with the owners’ optimal strategies to maximize its utility, thus obtaining the nash equilibrium solution. To provide reliable quality of service (QoS) and security guarantee for content delivery, smart contracts regulate the delivery process, facilitating automatic execution under specific conditions. Transaction records audited and stored on blockchain enhance transparency and trustworthiness. Utilizing a delivery utility model that considers benefits, costs, and mining profits, proposed quantum particle swarm optimization (QPSO) algorithm is used to find the optimal solution. We built an EdgeChain testbed, and used BDD-100K dataset to evaluate the performance in utility, access delay, etc. Compared to CTM and MFPA, proposed data sharing algorithm achieves maximum consumer utility. Compared to LRU, PCCM and MARL, when content is 400, proposed content delivery algorithm reduces average access delay by 30.88%, 18.92% and 4.86%, and reduce backhaul load by 50.04%, 47.23% and 3.16%.

In this paper, we plan to use ground-based stations in mobile edge computing (MEC) and unmanned aerial vehicles (UAVs) to provide communication and computation offloading services in disaster areas. However, optimizing the initial number and three-dimensional position of deployed UAVs is a prerequisite for providing computing services to users. Additionally, due to the limited battery and computing power of UAVs, it is a major challenge to rationally design the UAV trajectory during the computational offloading period to ensure communication quality for mobile users and reduce the energy consumption for completing tasks. Thus, we propose a cooperative computation offloading strategy with on-demand deployment of multi-UAV in UAV-aided MEC. The strategy utilizes the predicted user trajectory for UAV deployment on the premise of the minimum path loss of users. Then, to minimize total energy consumption for completing tasks, a joint optimization problem comprising user association strategy, computing resource allocation strategy, and UAV trajectory is proposed, which is a mixed-integer nonlinear program (MINLP). Therefore, to find the suboptimal solution, we use the block coordinate descent method to solve the problem. Numerical results show that the proposed algorithm can efficiently reduce the path loss by up to 18.55% and the total energy consumption by 18.28% compared to the benchmarks.

The convergence of unmanned aerial vehicle (UAV)-aided mobile edge computing (MEC) networks and blockchain transforms the existing mobile networking paradigm. However, in the temporary hotspot scenario for intelligent connected vehicles (ICVs) in UAV-aided MEC networks, deploying blockchain-based services and applications in vehicles is generally impossible due to its high computational resource and storage requirements. One possible solution is to offload part of all the computational tasks to MEC servers wherever possible. Unfortunately, due to the limited availability and high mobility of the vehicles, there is still lacking simple solutions that can support low-latency and higher reliability networking services for ICVs. In this article, we study the task offloading problem of minimizing the total system latency and the optimal task offloading scheme, subject to constraints on the hover position coordinates of the UAV, the fixed bonuses, flexible transaction fees, transaction rates, mining difficulty, costs and battery energy consumption of the UAV. The problem is confirmed to be a challenging linear integer planning problem, we formulate the problem as a constrained Markov decision process. Deep Reinforcement Learning (DRL) has excellently solved sequential decision-making problems in dynamic ICVs environment, therefore, we propose a novel distributed DRL-based P-D3QN approach by using Prioritized Experience Replay strategy and the dueling double deep Q-network (D3QN) algorithm to solve the optimal task offloading policy effectively. Finally, experiment results show that compared with the benchmark scheme, the P-D3QN algorithm can bring about 26.24% latency improvement and increase about 42.26% offloading utility.

In vehicular edge computing, Unmanned Aerial Vehicles (UAVs) have become a feasible solution for addressing high deployment costs faced by base stations in congested roads during peak hours. However, UAVs cannot cache all requested content due to limited storage. Hence, we propose a content caching strategy based on user preference predictions. To address resource consumption and user privacy concerns during the training process, we propose a user preference prediction model based on hierarchical federated learning training. Specifically, we employ a hierarchical clustering approach to partition user vehicles and UAVs into multiple clusters and utilize hierarchical federated learning to train prediction models within each cluster. Furthermore, to tackle the joint optimization problem of content caching and bandwidth allocation, we propose I-MADDPG, an improved multi-agent deep deterministic policy gradient algorithm. It determines the next continuous action based on the reward value at the current moment and the average reward value in the iteration period as reference parameters. The experimental results demonstrate that the proposed algorithm has significantly enhanced training efficiency compared to the baselines. Additionally, it has improved cache hit rate and reduced content request delay through effective resource allocation.

Massive map data transmission and the strict demand for the privacy of high-precision maps have brought significant challenges to the cache of high-precision maps in intelligent connected vehicles (ICV). Federal learning (FL) was introduced to reduce the pressure on the edge network and protect privacy. But the high dynamics of cars and limited resources lead to low accuracy and high training delay. We propose a joint optimization scheme of participant selection and resource allocation for federated learning. In each time slice, vehicles are determined whether to participate in training, which minimizes long-term training delay with limited energy consumption. To meet the delay and privacy requirements of high-precision map caching, we present an edge cooperative caching scheme based on federated deep reinforcement learning (F-DRL), which aims to achieve dynamic adaptive edge caching while protecting user privacy. The collaborative caching model is formulated as a Markov decision process (MDP). Dueling Deep Q Network (Dueling-DQN) is used to solve the optimal strategy, and federal learning is used for training. Enough comparative experiments to evaluate the performance of the proposed schemes. The aspects of reliability, cache hit rate, and training accuracy prove that the method effectively improves the training parameters of federated learning while meeting a high-precision map cache’s delay and reliability requirements.

The purposes are to explore the safety performance of the Maritime Transportation System (MTS) based on Digital Twins (DTs) Internet of Things (IoT) and develop maritime transportation towards intelligence and digitalization. Because the comprehensive operational security of modern MTS is not yet mature, historical transportation data of the Maritime Silk Road are acquired and preprocessed. Afterward, DTs are introduced, and relay nodes are added to data transmission paths to construct a maritime transportation DTs model based on relay cooperation IoT. Eventually, this model's security performance is validated through simulation experiments. Relay security analysis suggests that interference information is a vital guarantee to assist in information non-disclosure, from which the constructed model can harvest energy to increase the data transmission power, thereby improving communication performance and secrecy rate. Outage probability analysis reveals that the simulated and the theoretical results are almost the same; moreover, given the system's multi-hop paths in the same environment, the more the relays and the greater the fading index, the better the system performance and the lower the outage probability. Once the iterations reach a particular number, the node secrecy rate becomes optimal and cannot cause excessive burden to the system; besides, the power distribution can establish a new equilibrium when the nodes are in different locations, so that system security performance gets improved. The simulated value is closest to the actual result under 100% successful transmission probability and 0.01~0.05 λ value. To sum up, the constructed maritime transportation DTs model presents extraordinary transmission and security performance, providing an experimental basis for intelligent and secure maritime transportation in the future.

Abstract In recent years, with the continuous development of internet of things and cloud computing technologies, data intensive applications have gotten more and more attention. In the distributed cloud environment, the access of massive data is often the bottleneck of its performance. It is very significant to propose a suitable data deployment algorithm for improving the utilization of cloud server and the efficiency of task scheduling. In order to reduce data access cost and data deployment time, an optimal data deployment algorithm is proposed in this paper. By modeling and analyzing the data deployment problem, the problem is solved by using the improved genetic algorithm. After the data are well deployed, aiming at improving the efficiency of task scheduling, a task progress aware scheduling algorithm is proposed in this paper in order to make the speculative execution mechanism more accurate. Firstly, the threshold to detect the slow tasks and fast nodes are set. Then, the slow tasks and fast nodes are detected by calculating the remaining time of the tasks and the real-time processing ability of the nodes, respectively. Finally, the backup execution of the slow tasks is performed on the fast nodes. While satisfying the load balancing of the system, the experimental results show that the proposed algorithms can obviously reduce data access cost, service-level agreement (SLA) default rate and the execution time of the system and optimize data deployment for improving scheduling efficiency in distributed clouds.

Abstract There are many research problems in cloud replica management such as low data reliability, unbalanced node load and large resource consumption. The strategy and status of replica creation, replica placement and replica selection are analyzed. The replica creation based on access tendency (DRC-AT), the replica placement based on user request response time and storage capacity (DRP-RS) and the replica selection based on response time (DRS-RT) are proposed. The DRC-AT algorithm introduces the two parameters of file popularity and period value of file popularity, calculates the file access tendency periodically and decides the creation and deletion of the replica of the file according to the size of the file access tendency. The DRP-RS algorithm evaluates the user’s request response time and storage capacity to select the best node set to place the replica. The DRS-RT algorithm returns to the user the node with the strongest service capability that contains the user’s requested data. Experiments show that the algorithm can improve the speed of data reading by the client, improve the resource utilization, balance the load of the node and improve the overall performance of the system.

In recent years, Sina microblog becomes an important social network platform for users to not only form personal website but share, deliver and keep interest messages via the relationships between users. However, with larger number of users in social networks, the problem of ‘information overload’ has been increasingly highlighted. In this case, community discovery has been the urgent requirements in microblog social networks. In this paper, a microblog community discovery algorithm based on tag correlation and interactive behaviors is presented. For the selection of exact user tags, the model of maximal marginal relevance is used and improved to enrich the variousness and novelty of user interest tags. The method of edge-similarity computation is improved to suit for microblog community discovery. Hadoop platform is applied for data process, which greatly decreases the running time of experiments. Finally, extensive experiments are provided to demonstrate the performance of proposed algorithm. The results indicate that the performance of proposed algorithm is better than that of benchmark algorithms for microblog social networks. The proposed algorithm for community discovery can identify the users belonging to multi communities or no community so that the results of community discovery are more accurate and more suitable for actual environment.

The paper proposes an efficient load-balancing aware cloud resource scheduling approach for mobile users.The proposed approach augments local cloud service pools with public cloud to increase the probability of meeting the service level agreements.The proposed problem is divided by public cloud service scheduling and local cloud service scheduling.The system status information is used in the hybrid mobile cloud system such as the preferences of mobile applications, energy, server load in cloud data center to improve resource utilization and quality of experience of mobile user.Therefore, the system status of hybrid mobile cloud is monitored continuously.The mathematical model of the system and optimization problem is given.An example for hybrid cloud-based mobile multimedia application is also presented.Through extensive experiments, the paper evaluates our algorithm and other approaches from the literature under different conditions.The results of the experiments show our approach has better performance than the approaches from the literature.