Satellite computing, as an emerging edge computing paradigm, extends computing and networking services into space. Due to the internal design constraints of low-Earth orbit (LEO) satellites and the challenges posed by the external environment, satellite computing faces inherent limitations, including severely constrained resources, non-rechargeable batteries, poor heat dissipation, and highly dynamic operating conditions, leading to unreliable and unsustainable quality of service. To address the above challenges and fully realize the potential of satellite computing, this paper investigates temperature- and energy-aware dynamic task scheduling and computing resource allocation, aiming to optimize service latency, reduce onboard energy consumption, and enhance operational profit. Solving this problem requires coordinating task scheduling and resource allocation, balancing communication and computation latency, and addressing the challenge of a vast search space. To solve the above challenges, we first formulate this problem as a repeated Stackelberg game by developing temperature and energy models. Through theoretical analysis, we show that this game leads to a convex optimization framework that exhibits exponential complexity. To accelerate the search for the Stackelberg equilibrium solution, we propose a dynamic task scheduling algorithm based on the interior point method, which reduces the computational complexity to polynomial order. Trace-driven simulations demonstrate that the proposed algorithm reduces task scheduling latency by 28.4% and improves utility by 13% on average.

Executing 5G core network functions on satellites has become crucial to enhance satellite network management and service capabilities. The User Plane Function (UPF) is responsible for efficient data traffic forwarding and is envisioned as a key and pioneering core network function that will be deployed on satellites. However, managing and providing services with satellite UPFs face dual challenges. Limited satellite resources constrain the user scale that a satellite UPF can service, resulting in an unguaranteed service delay. Moreover, the extremely rapid mobility of satellites renders it difficult for satellite UPFs to provide seamless services. To address the above challenges, this paper presents the first-of-its-kind service optimization scheme for satellite UPFs in terms of switch control, state migration, and traffic routing. To provide guaranteed service delay, we provide a theoretical analysis based on the M/G/1 queue model, demonstrating the service delay-resource consumption trade-off. A satellite UPF switch control scheme is integrated into the service optimization process, which can decrease satellite UPF service delay while saving satellite resources by adjusting the switch control parameters. To provide seamless services, we propose a satellite UPF-oriented state-aware service migration and traffic routing (UPF service optimization) algorithm. A policy network-based reinforcement learning approach is employed to dynamically perceive the satellite network’s state as well as the satellite UPF switch state. Building upon the optimization of service delay through satellite UPF switch control, the processes of state-aware state migration and traffic routing are further employed to reduce delay, ensuring seamless service effectively. Experiments reveal that the proposed algorithm outperforms other benchmark algorithms under different metrics. The service delay is reduced by an average of 23.2% compared with other algorithms.

The proliferation of users and data traffic poses substantial pressure on resource management in the Internet of Things (IoT). In addition to beneficially allocating scarce network resources, it also needs to meet differentiated users’ Quality-of-Service (QoS) requirements, such as low delay, high security, etc. The distributed management architecture of blockchain and its inherent security features bring inspiration to resource management in the IoT. In this article, we propose a blockchain-enabled resource orchestration scheme for IoT by deep reinforcement learning (DRL), where the IoT edge server and the end user can reach a consensus on the allocation of network resources based on blockchain theory. Moreover, relying on the policy network, the intelligent agent can be trained by these resource attributes to fully perceive the change of the network’s state and hence make dynamic resource allocation decisions. Finally, simulation results show that the proposed resource orchestration scheme has good performance in comparison to other security resource allocation algorithms. The average revenue, the user request acceptance rate, and the profitability are increased by an average of 8.5%, 1.8%, and 11.9%, respectively, compared with other algorithms.

The development of intelligent cyber–physical systems (ICPSs) in the virtual network environment is facing severe challenges. On the one hand, the Internet of Things (IoT) based on ICPSs construction needs a large amount of reasonable network resources support. On the other hand, ICPSs are facing severe network security problems. The integration of ICPSs and network virtualization (NV) can provide more efficient network resource support and security guarantees for IoT users. Based on the above two problems faced by ICPSs, we propose a virtual network embedded (VNE) algorithm with computing, storage resources, and security constraints to ensure the rationality and security of resource allocation in ICPSs. In particular, we use the reinforcement learning (RL) method as a means to improve algorithm performance. We extract the important attribute characteristics of the underlying network as the training environment of the RL agent. The agent can derive the optimal node embedding strategy through training, so as to meet the requirements of ICPSs for resource management and security. The embedding of virtual links is based on the breadth first search (BFS) strategy. Therefore, this is a comprehensive two-stage RL-VNE algorithm considering the constraints of computing, storage, and security 3-D resources. Finally, we design a large number of simulation experiments from the perspective of typical indicators of VNE algorithms. The experimental results effectively illustrate the effectiveness of the algorithm in the application of ICPSs.

Network virtualization (NV) is a technology with broad application prospects. Virtual network embedding (VNE) is the core orientation of VN, which aims to provide more flexible underlying physical resource allocation for user function requests. The classical VNE problem is usually solved by the heuristic method, but this method often limits the flexibility of the algorithm and ignores the time limit. In addition, the partition autonomy of physical domain and the dynamic characteristics of virtual network request (VNR) also increase the difficulty of VNE. This article proposed a new type of VNE algorithm, which applied reinforcement learning (RL) and graph neural network (GNN) theory to the algorithm, especially the combination of graph convolutional neural network (GCNN) and RL algorithm. Based on a self-defined fitness matrix and fitness value, we set up the objective function of the algorithm implementation, realized an efficient dynamic VNE algorithm, and effectively reduced the degree of resource fragmentation. Finally, we used comparison algorithms to evaluate the proposed method. Simulation experiments verified that the dynamic VNE algorithm based on RL and GCNN has good basic VNE characteristics. By changing the resource attributes of physical network and virtual network, it can be proved that the algorithm has good flexibility.

Summary Unmanned aerial vehicle (UAV) has a broad application prospect in the future, especially in the Industry 4.0. The development of Internet of Drones (IoD) makes UAV operation more autonomous. Network virtualization technology is a promising technology to support IoD, so the allocation of virtual resources becomes a crucial issue in IoD. How to rationally allocate potential material resources has become an urgent problem to be solved. The main work of this paper is presented as follows: (a) In order to improve the optimization performance and reduce the computation time, we propose a multidomain virtual network embedding algorithm (MP‐VNE) adopting the centralized hierarchical multidomain architecture. The proposed algorithm can avoid the local optimum through incorporating the genetic variation factor into the traditional particle swarm optimization process. (b) In order to simplify the multiobjective optimization problem, we transform the multiobjective problem into a single‐objective problem through weighted summation method. The results prove that the proposed algorithm can rapidly converge to the optimal solution. (c) In order to reduce the mapping cost, we propose an algorithm for selecting candidate nodes based on the estimated mapping cost. Each physical domain calculates the estimated mapping cost of all nodes according to the formula of the estimated mapping cost, and chooses the node with the lowest estimated mapping cost as the candidate node. The simulation results show that the proposed MP‐VNE algorithm has better performance than MC‐VNM, LID‐VNE, and other algorithms in terms of delay, cost and comprehensive indicators.

Traditional ground wireless communication networks cannot provide high-quality services for artificial intelligence (AI) applications such as intelligent transportation systems (ITS) due to deployment, coverage and capacity issues. The space-air-ground integrated network (SAGIN) has become a research focus in the industry. Compared with traditional wireless communication networks, SAGIN is more flexible and reliable, and it has wider coverage and higher quality of seamless connection. However, due to its inherent heterogeneity, time-varying and self-organizing characteristics, the deployment and use of SAGIN still faces huge challenges, among which the orchestration of heterogeneous resources is a key issue. Based on virtual network architecture and deep reinforcement learning (DRL), we model SAGIN’s heterogeneous resource orchestration as a multi-domain virtual network embedding (VNE) problem, and propose a SAGIN cross-domain VNE algorithm. We model the different network segments of SAGIN, and set the network attributes according to the actual situation of SAGIN and user needs. In DRL, the agent is acted by a five-layer policy network. We build a feature matrix based on network attributes extracted from SAGIN and use it as the agent training environment. Through training, the probability of each underlying node being embedded can be derived. In test phase, we complete the embedding process of virtual nodes and links in turn based on this probability. Finally, we verify the effectiveness of the algorithm from both training and testing.

The continuous expanded scale of the industrial Internet of Things (IIoT) leads to IIoT equipments generating massive amounts of user data every moment. According to the different requirement of end users, these data usually have high heterogeneity and privacy, while most of users are reluctant to expose them to the public view. How to manage these time series data in an efficient and safe way in the field of IIoT is still an open issue, such that it has attracted extensive attention from academia and industry. As a new machine learning paradigm, federated learning (FL) has great advantages in training heterogeneous and private data. This article studies the FL technology applications to manage IIoT equipment data in wireless network environments. In order to increase the model aggregation rate and reduce communication costs, we apply deep reinforcement learning (DRL) to IIoT equipment selection process, specifically to select those IIoT equipment nodes with accurate models. Therefore, we propose a FL algorithm assisted by DRL, which can take into account the privacy and efficiency of data training of IIoT equipment. By analyzing the data characteristics of IIoT equipments, we use MNIST, fashion MNIST, and CIFAR-10 datasets to represent the data generated by IIoT. During the experiment, we employ the deep neural network model to train the data, and experimental results show that the accuracy can reach more than 97%, which corroborates the effectiveness of the proposed algorithm.

At present, the traditional heuristic method to solve the problem of virtual network embedding (VNE) is still the mainstream. In the environment of network virtualization (NV), a more efficient VNE algorithm is needed to serve the construction of smart city. Using heuristic algorithm to solve the problem of VNE does not meet its development requirements. In this paper, a VNE algorithm based on node probability is proposed by using reinforcement learning (RL) algorithm. The algorithm extracts three attributes of each substrate node to form a feature matrix, which is used as the input of the policy network to train the agent. The purpose is to deduce the mapping probability of each node and rank the base nodes according to this probability, then embed the virtual nodes in this order. Finally, the breadth first search (BFS) strategy is used to map the links. Simulation results show that our algorithm is superior to a representative algorithm based on node ranking in terms of the acceptance rate of virtual network requests (VNR), long-term revenue consumption ratio and long-term average revenue.