Recently, Named Data Networking (NDN) has garnered widespread attention in academia as an innovative network architecture, offering solutions to challenges such as the vulnerability of end-to-end connections in IP-based networks. In NDN, nodes utilize a “pull-push” architecture, exchangingInterestandDatapackets for communication. This architecture is particularly well-suited for highly dynamic, topology-varying unmanned aerial vehicle (UAV) swarm networks, known as UAV Named Data Networking (UNDN). However, in UNDN, due to constraints such as the lightweight design and limited energy of UAVs, the UAVs may exhibit selfish behaviors, opting not to share data in order to conserve their own energy consumption. This behavior results in degraded network performance, as the lack of cooperation among UAVs can hinder efficient data sharing and communication. Therefore, an effective incentive mechanism needs to be proposed. In this paper, we formulate the content-sharing process in UNDN as a double auction market for data exchange. To tackle the problem of asymmetric information between content consumers and producers, we propose an Iterative Double Auction algorithm (IDAA). This algorithm introduces a virtual central broker to guide both parties in conducting honest auctions. Furthermore, we develop a diffusion model-based reinforcement learning algorithm (DiffRL-DA) to derive optimal auction policies, with the goal of better capturing market behaviors and overcoming the limitations of the IDAA. Finally, simulation results verify the efficacy of our proposed mechanisms.

In the era of Artificial Intelligence of Things (AIoT), Unmanned Aerial Vehicles (UAVs) are increasingly deployed in emergency scenarios to provide intelligent environmental sensing and rapid situational awareness. However, the massive raw sensory data generated by UAVs can easily overwhelm bandwidth-limited communication infrastructures, undermining the timeliness and reliability of disaster response. To address this challenge, we propose a cloud-edge-end collaborative semantic communication(SemCom) framework that enables UAVs, mobile edge vehicles, and cloud-based command centers to jointly deliver resilient and scalable AIoT services. In this architecture, UAVs perform real-time environmental sensing, vehicles act as mobile edge nodes for semantic filtering and resource coordination, while the cloud executes global decision-making. To optimize semantic data trading under incomplete information, we integrate matching theory with a multi-armed bandit (MAB) model and design a Matching-UCB algorithm, which allows UAVs and vehicles to dynamically learn preferences through repeated interactions. Theoretical analysis proves that Matching-UCB achieves sub-linear regret and guarantees stable matching outcomes. Simulation results further demonstrate that our semantic-aware AIoT framework reduces transmission load by more than 99% and achieves optimal matching with the highest social welfare and lowest regret among benchmarks.

The swarm of Unmanned Aerial Vehicles (UAVs) has garnered considerable attention, particularly in scenarios with critical situations or limited communication infrastructure. In such cases, Mission UAVs (MUs) often be deployed in clusters to provide communication services. However, the high mobility of MUs always leads to frequent changes in swarm network topology, posing challenges for the network performance. To tackle this issue, we deploy additional Relay UAVs (RUs) with a two-stage adaptive learning-based approach. In the first stage, we employ a Delaunay triangulation-based algorithm to optimize RUs’ position and construct the initial topology. In the second stage, we implement a centralized learning and decentralized execution (CTDE) reinforcement learning framework to ensure continuous network connectivity and optimize performance throughout the task cycle. To further enhance cooperation among RUs, we introduce a sequential update technique coupled with an entropy regularization term during the policy network updates. Finally, extensive simulation results demonstrate the effectiveness of our proposed algorithms.

The rapid advancement of UAV swarm networks has enabled their widespread application across various domains, including disaster relief, environmental monitoring, and intelligent transportation. Collaboration among UAVs within a swarm is vital for efficient resource utilization and optimal performance across these diverse applications. To address diverse service demands, deploying service function chains (SFC) in UAV swarm networks facilitates the real-time implementation of services through efficient resource allocation and UAV cooperation, thereby enhancing network reliability and efficiency. However, traditional SFC deployment strategies struggle to achieve reliability and efficiency due to dynamic topology and limited resources. Additionally, Stochastic Network Calculus (SNC) derives end-to-end latency, guaranteeing quality of service (QoS) in UAV swarm networks. To navigate this issue, we propose a cooperative dynamic SFC deployment algorithm that combines hierarchical proximal policy optimization (HPPO) with an edge-enhanced dynamic graph attention network (EDGAT) for real-time network state extraction. The simulation results validate the effectiveness of our proposed algorithm, showcasing improvements in deployment success rate and long-term average revenue.

The rapid growth of 5G and 6G networks has introduced complex traffic patterns and stringent real-time demands. Traditional SDN architectures struggle to meet the low-latency and dynamic requirements of wireless environments due to high communication overhead and rigid hardwares. Programmable switches, with their ability to dynamically cus-tomize data plane behavior, offer a more flexible solution for real-time traffic management at the network edge. However, most existing solutions rely on offline models with limited real-time detection capabilities, resulting in increased overhead and suboptimal performance. In this paper, we present Gendetect, a generative-AI enabled lightweight traffic detection architec-ture for programmable wireless gateways. Gendetect employs generative knowledge distillation to train decision tree-based models, enabling efficient online training and adaptive updates. By generating synthetic training data in real-time, it reduces the need for frequent control plane interactions, mitigating north-south overhead. Additionally, a feature selection mechanism optimizes resource utilization, balancing table entry consumption and detection accuracy. Extensive simulations demonstrate that Gendetect significantly improves traffic detection performance while reducing match-action table entries, making it well-suited for dynamic and resource-constrained wireless networks.

Recently, unmanned aerial vehicle (UAV) swarm networks have garnered considerable interest from both academia and industry, with applications spanning disaster response and logistics. These environments are complex and demand efficient, stable network performance under highly dynamic conditions. Clustering is a promising solution to manage UAVs by creating a hierarchical structure. We propose a distributed method, Distracted Density Peaks Clustering (DDPC), which uses local information to build a decision graph and identify density centers. Additionally, a dynamic maintenance strategy enhances adaptability, and simulations confirm its effectiveness.

Currently, beamforming antenna array technologies are of utmost importance in 5G communication systems. These technologies are essential for optimizing the coverage and signal quality of the cellular network. However, the optimization of broadcast beams presents significant challenges due to the complex strategy profile space. Each beam can be configured with different widths and heights, making it difficult for conventional algorithms to handle. To address this issue, we propose a novel approach called Multi-Agent Moth-Flame Reinforcement Learning (MAMF-RL) algorithm for broadcast beam optimization. MAMF-RL combines reinforcement learning and moth-flame optimization algorithms to interactively search for the optimal broadcast beams. By decomposing the problem into multiple single-sector antenna configuration problems, MAMF-RL effectively reduces the algorithm complexity. We conducted experiments utilizing real data in an 18-sector wireless coverage area. To evaluate the performance of our proposed method, we compared it with traditional methods such as the particle swarm algorithm. The results demonstrate that our MAMF-RL model achieves an average coverage rate of 1.82% higher and a 13.74% lower overlapping coverage rate compared to traditional methods.

The rapid advancement of the UAV swarm network has made its widespread application across a multitude of domains. However, the inherently dynamic nature of the network often gives rise to intermittent connectivity issues, leading to a significant reduction in the data transmission capacity. To address this challenge, this study explores the integration of Information-centric Network (ICN) with the delay-tolerant network (DTN). This design aims to enhance message delivery rates by caching content data packets in UAV nodes. Building upon this architecture, we study the congestion control and load balancing problem. We design an on-demand collaborative communication routing algorithm. In our design, we first propose a routing decision model that incorporates multiple routing metrics to capture the dynamic evolution patterns of network nodes, effectively controlling local congestion issues. Subsequently, we employ Lyapunov optimization techniques to achieve a network load balancing. By integrating the Lyapunov drift function, we ensure the stability of a feasible solution space within the model. Additionally, considering the high communication overhead caused by the sparse communication characteristics of DTN, we deploy a Multi-Agent Incentivized Communication (MAIC) algorithm to optimize routing scheduling strategies. Within the MAIC framework, each agent develops unique models for its teammates to generate customized information and minimize network information redundancy. Simulation results demonstrate that this algorithm effectively ensures a congestion control and a load balancing within the UAV swarm network while maintaining communication overhead in routing computations at a minimal level.

Over the past decade, low-Earth-orbit (LEO) satellite networks have emerged as a critical infrastructure in communication systems, providing wide coverage, high reliability, and global connectivity. Recently, the development of 6G technologies has challenged the LEO satellite networks to guarantee deterministic scheduling for time-sensitive services. However, traditional deterministic networking techniques fall short for LEO satellite networks. First, these techniques impose strict time constraints, but in LEO satellite networks, delay and jitter typically range in the tens of milliseconds, which exceed these limits and render them infeasible. Second, the dynamic topologies of LEO satellite networks challenge the inflexible scheduling strategies generated by these techniques, leading to sub-optimal performance and potential strategy failures. To tackle the first problem, we propose a Cycle Specified Queuing and Forwarding (CSQF) based deterministic flows scheduling mechanism. It relaxes strict time constraints by employing cyclic multi-queue scheduling, enabling more flexible and reliable long-distance transmission. For the second problem, we propose a learning-based swarm intelligence method for deterministic flows scheduling in dynamic LEO satellite networks. It includes an algorithm that combines a Dynamic Graph Convolutional Network (DGCN) with an Adaptive Ant Colony Optimization (ACO) algorithm, referred to as the DGCN-ACO algorithm. The DGCN captures the dynamic feature of the network and generates the heuristic information. The Adaptive ACO utilizes the heuristic information and considers each flow's attribute to generate multi-path scheduling strategies for each deterministic flow, as well as updates the DGCN. The experiment results demonstrate the effectiveness of our proposed algorithm.

In-band network telemetry (INT) is an emerging network measurement technique that offers real-time and fine-grained visualization capabilities for networks. However, the utilization of INT for network measurement introduces additional overheads to the network. The process of data collection consumes extra bandwidth resources, and adjustments to the data collection scheme can impact network stability. Additionally, the INT orchestration scheme requires adaptation to dynamics in the network to improve measurement accuracy. Therefore, striking a balance between accuracy and stability becomes a critical problem. In this paper, our focus lies in the trade-off between measurement accuracy and network stability. We consider the long-term orchestration of multiple telemetry tasks, rationally deploying distinct telemetry tasks to different application flows. To address the challenge, we propose a self-adaptive Dynamic INT Orchestration scheme, D-INTO. Specifically, we formulate a stochastic optimization problem for dynamic INT orchestration. Then we employ Lyapunov optimization to decouple the stochastic optimization problem and use surrogate Lagrangian relaxation to construct a polynomial-time approximation algorithm. Theoretical analysis and experimental results demonstrate that our proposed D-INTO outperforms existing schemes in terms of adaptability to the network dynamics.

In recent years, there has been significant attention in UAV Named Data Networking (UNDN) from both industry and academia. This network paradigm adopts a “request-reply” communication model that allows UAVs to access desired content without the need for specific information regarding the geographical location or IP address of the content producer. This IP-independent design is well-suited for dynamic UAV swarms, but it presents challenges in establishing matching policies between content consumers and producers. This is because that during the distributed decision-making process in content sharing, consumers cannot possess private information regarding producers, and producers may lack the motivation to distribute content. As a result, a revelation and incentive mechanism is needed to be formulated in the system. In this paper, a resource-efficient content-sharing mechanism is proposed to address the aforementioned challenges. First, we propose a contract-based mechanism to incentivize content producers to share content and reveal their private information at the same time. The problem of obtaining the optimal contract is discussed in both cases of information asymmetry and complete information. Then, the Gale-Shapley (GS) algorithm is adopted to make a stable many-to-one matching between content consumers and content producers. The simulation results verify the feasibility, effectiveness and energy efficiency of the proposed mechanism.

The Digital Twin (DT) represents a rapidly advancing technological innovation within the Industrial Internet of Things (IIoT) domain. DT leverages the power of simulation, machine learning, and data mining to facilitate optimal decision-making for physical objects. However, the creation of a dynamic and living digital counterpart comes at a considerable cost. It requires continuous massive data updating and processing every time the physical object changes. As most data collected by IIoT devices are in their original form, such as images and videos, transmitting such data to remote cloud computing will result in large delays. Furthermore, data processing is often a computationally intensive operation, such as image recognition and video coding, making it impractical to perform processing tasks directly in IIoT devices. To overcome this problem, we introduced the Multi-access/mobile Edge Computing (MEC) architecture to enhance capabilities of DT-enabled IIoT devices. IIoT devices can leverage the extra computing resources in MEC to process raw data, transmitting only the calculation results to update the digital counterpart. To efficiently allocate resources between IIoT devices and MEC, we propose a double auction-based resource allocation scheme. The IIoT devices can purchase computing power from MEC, and an iterative double auction scheme is applied to achieve system efficiency within this market. Furthermore, we propose the Win or Learn Fast Algorithm Policy Hill Climbing (Wolf-PHC) algorithm, which enables agents to improve their strategies continuously through participation in auctions. Simulation results demonstrate that this algorithm accelerates the process of market equilibrium convergence.

With the ongoing advancement of information and communication technologies, the communication technologies for UAV swarm networks have undergone rapid development, especially in the context of large-scale UAV network deployments. In recent years, UAVs have found wide-ranging applications in both military and civilian domains. However, the inherent complexity and high dynamic nature of UAV swarm activities present substantial challenges to traditional routing algorithms, prompting the need for the design and implementation of efficient and sustainable routing solutions. To address these challenges, this paper introduces a UAV swarm routing algorithm based on evolutionary game theory, with a particular focus on energy efficiency and resource optimization. We leverage evolutionary game theory to enhance cooperation among nodes and adopt a strategy update rule that imitates the best-performing agents. In the proposed algorithm, nodes engage in continuous packet forwarding and participate in game interactions with neighboring nodes, adjusting their strategies based on accumulated gains. This strategy not only significantly enhances network lifetime and improves the packet delivery rate but also optimizes energy consumption and resource utilization, aligning with sustainable computing principles. To validate the effectiveness of the proposed routing method, we conduct extensive simulation experiments within a designed and implemented system model under different environmental contexts. The analysis confirmed the accuracy and effectiveness of the proposed routing method, highlighting its exceptional performance in terms of the network survival time, the number of successfully transmitted packets, and the adaptability in dynamic scenarios.

Over the past decade, the Unmanned Aerial Vehicle (UAV) swarm has emerged as a disruptive force reshaping our lives and work. Benefiting from its fast and flexible deployment capabilities, UAV swarms have been widely applied to emergency communications. In the event of damaged ground communication base stations, UAV swarms can quickly reconstruct an emer-gency communication network. However, considering the limited coverage power of a single UAV node, it underscores the need for effective coordination among swarm units as well as diligent planning of a coverage trajectory. In this paper, we propose a cooperative intelligence-based UAV swarm approach for establishing emergency communications. We model a multi-UAV base station-assisted emergency communication scenario as a team Markov game model. To achieve cooperative collaboration among multiple UAVs, we propose a Q-function mixing network based coverage trajectory planning algorithm. Our experimental results demonstrate the superior convergence speed and throughput of the proposed algorithm.

In order to ensure reliable transmission of important traffic in industrial networks, time-sensitive network (TSN) technology and fifth-generation mobile communication technology (5G) are introduced into the industrial network. However, there are still challenges in integrating TSN networks with 5G networks, especially in terms of end-to-end scheduling in hybrid systems. Considering the diverse range of traffic types and their end-to-end transmission requirements within the industrial Internet, we propose a differentiated traffic scheduling model and develop a population generation algorithm, termed Genetic Algorithm (GA) based two-stage population generation algorithm (PTPG). Notably, the algorithm utilizes a non-target training approach to generate the initial population and integrate Proximal Policy Optimization (PPO) to improve algorithm convergence and facilitate the inheritance of advantages across generations. The simulation results demonstrate notable enhancements in end-to-end delay, the number of occupied queues, and algorithm convergence status compared to other algorithms.

Unmanned aerial vehicle (UAV) swarms have found extensive applications owing to their flexibility, mobility, cost-effectiveness, and capacity for collaborative and autonomous service delivery. Empowered by intelligent algorithms, UAV swarm can exhibit cohesive behaviors and autonomously coordinate to achieve collective objectives. Nonetheless, in real-world scenarios with uncertainty and stochasticity, its performance suffers from the unstable information exchange among UAVs and inefficient data sampling. In this paper, we introduce a metaverse-based UAV swarm system, where monitoring, observation, analysis, and simulation can be realized collaboratively and virtually. Within the metaverse, virtual service providers (VSPs) utilize digital twin (DT) to generate and render virtual sub-worlds, while providing diverse virtual services. In particular, the VSP trains the learning model using high-fidelity data from the physical world, formulates optimal decisions for diverse tasks, and returns these decisions to the UAV swarm for the execution of the corresponding tasks. Since synchronization between two worlds needs frequent data exchange, we employ the semantic communication technique in our system which could reduce communication latency by transmitting only the semantic information. In such design, UAVs as workers are employed to collect data and provide extracted semantic information to the VSPs. Moreover, we propose a hierarchical framework to investigate the reliability and sustainability of the metaverse-based UAV swarm system. In the lower layer, we design a worker selection scheme to determine reliable UAVs for data synchronization. In the upper layer, we consider deep learning (DL)-based auction as the incentive mechanism for resource allocation in semantic information trading between UAV swarm and VSPs.

In recent years, the rapid growth of Internet of Things (IoT) devices and applications has posed significant challenges for existing Mobile Edge Computing (MEC) architectures. The inherent latency uncertainties in MEC architectures make it difficult to support latency-sensitive applications such as autonomous vehicles. Additionally, the increasing number of connected devices has led to substantial challenges in terms of limited throughput for MEC servers. With the recent advancements in programmable network hardware, such as SmartNICs and programmable switches, the Network-based Computing (NBC) paradigm has gained widespread attention. Leveraging line-rate processing capabilities, NBC offers a promising solution for high throughput and low latency processing. This paper aims to explore the potential benefits and challenges of incorporating NBC into existing MEC architectures. The feasibility of our proposed architecture is evaluated using two use cases, Linear Quadratic Regulator (LQR) control and Complex Event Processing (CEP), demonstrating significant improvements in latency performance.

The flying ad hoc networks (FANETs) have attracted a large amount of attention from both academia and industry. Benefiting from the flexibility, the FANETs have been widely deployed in various scenarios, ranging from agricultural production to emergency rescue. However, in FANETs, the mobility of unmanned aerial vehicles (UAVs) has led to critical challenges for the stability of communications. Especially, the routing flooding mechanism extremely limits the scalability of FANET. To overcome these technical challenges, constructing a hierarchy and clustering structure in FANETs is considered a promising solution. In this paper, we propose the fission spectral clustering (FSC) strategy for UAV swarm networks. We model the UAV clustering problem as a graph cut problem. The time-sequential attributes weight of nodes and edges will be input to the FSC algorithm. Then, it will construct the Laplace matrix and calculate the first k-th eigenvectors of it. We apply the K-Means algorithm into this feature space to cut the graph by clustering the eigenvectors. Each cluster will constantly fission with this strategy until it satisfies the size and structure constraints in the UAV clusters. Some simulations are implemented to evaluate our proposed algorithm in comparison to the other state-of-the-art solutions.

Recently, diverse emerging scenarios have precipitated a substantial surge in the variety of devices and applications, which has consequently imposed more stringent demands on Quality of Service (QoS) prerequisites. As a burgeoning emerging network measurement method, In-band network telemetry (INT), can provide detailed metrics for QoS by obtaining fine-grained network status information. However, INT only outlines device-level operations, which fails to provide an entire network view for monitoring. To address this, INT orchestration based on network topology and application requirements to achieve network-level monitoring is necessary. In this paper, we propose an INT orchestration model that efficiently measures the entire network while minimizing measuring overhead. The model outputs the probe path and collects requirements for the devices it passes through. Our method effectively reduces network bandwidth consumption caused by INT process and ensures telemetry items remain fresh. Experiment results support the effectiveness of our approach.

As an emerging technology, Software Defined Networking (SDN) shows great potential in the management of complex networks. Its centralized control and programmability provide flexible network traffic control, making networks more intelligent. Meanwhile, Unmanned Aerial Vehicles (UAVs) have gained popularity in various industries for their flexibility, maneuverability, and ability to access hard-to-reach areas. In this paper, we present a new network architecture and dynamic controller assignment algorithm for vehicular networks. The decoupling of the control plane and the data plane is realized through SDN, and the controller is deployed on the UAV to dynamically control the vehicle. As for the dynamic controller assignment problem, we establish a model by the evolutionary game algorithm, and dynamically assign controllers based on vehicle positions and traffic loads on the controllers. Extensive simulation results confirm the effectiveness of the method.

The combination of digital twin (DT) and Internet of Vehicles (IoV) has gained significant attention from both academia and industry in recent times. DT can establish a high fidelity virtual representation of IoV based on the real-time sensor data, and feedback the decision policy, therefore generating possible improvements. Especially, as the advance of Mobile Edge Computing (MEC) technique, it has the potential to facilitate digital twin¡¯s computationally intensive tasks. However, how to schedule the MEC computing resource is the key to efficient operation of the whole system. Therefore, this study aims to investigate pricing considerations and resource management that exist between the vehicle and MEC server in order to mitigate this issue. Specifically, we model the interaction between the MEC server and vehicles as a Stackelberg game, where the leader (i.e., the MEC service provider) sets prices, and then the vehicles act as followers. By leveraging information about social interactions from other vehicles, utility functions are formulated by the vehicles. Additionally, the study analyzes the existence and uniqueness of the Stackelberg equilibrium, and proposes a dynamic iterative algorithm to find the appropriate Nash equilibrium for the proposed Stackelberg game. Experimental results demonstrate that the proposed scheme effectively formulates suitable prices and meets computational requirements.

The past few years have witnessed an exponential growth of diverse Internet of Things (IoT) devices as well as compelling applications ranging from industrial production to medical care. Dramatic advances in IoT technology not only brought enormous economic opportunities but also challenges (e.g., privacy and security vulnerabilities). Recently, with the appearance of blockchain technology, the integration of IoT and blockchain (BCoT) is considered a promising solution to address these issues. Blockchain provides a secure and scalable data management framework for IoT devices. However, the huge computation and energy cost of the consensus process in blockchain prevents it from being directly applied as a generic platform. To overcome this challenge, in this article, we propose a cloud mining pool-aided BCoT architecture, where the IoT devices can rent the computing resources from the cloud mining pools to offload the mining process. Based on this architecture, we study the mining pool selection problem and analyze the colony behaviors of IoT devices with different pooling strategies. We propose a centralized evolutionary game-based pool selection algorithm for the sake of maximizing the system utility. Considering the non-cooperative relationship among multiple miners, we also propose a lightweight distributed reinforcement learning algorithm, named the ‘WoLF-PHC’ algorithm.

With the rapid development of the Industrial Internet of Things (IIoT), massive IIoT devices connect to industrial networks via wired and wireless. Furthermore, industrial networks pose new requirements on communications, such as strict latency boundaries, ultra-reliable transmission, and so on. To this end, time-sensitive networking (TSN) embedded fifth-generation (5G) wireless communication technology (i.e., TSN-5G networks), is considered the most promising solution to address these challenges. TSN can provide deterministic end-to-end latency and reliability for real-time applications in wired networks. 5G supports ultra-reliable and low-latency communications (uRLLC), providing increased flexibility and inherent mobility support in the wireless network. Thus, the integration of TSN and 5G provides numerous benefits, including increased flexibility, lower commissioning costs, and seamless interoperability of various devices, regardless of whether they use a wired or wireless interface. Nonetheless, the potential barriers between the TSN and 5G systems, such as clock synchronization and end-to-end traffic scheduling, are inevitable. Time synchronization has been studied in many works, so this paper focuses on the end-to-end traffic scheduling problem in TSN-5G networks. We propose a novel integrated TSN and 5G industrial network architecture, where the 5G system acts as a logical TSN-capable bridge. Based on this network architecture, we design a Double Q-learning based hierarchical particle swarm optimization algorithm (DQHPSO) to search for the optimal scheduling solution. The DQHPSO algorithm adopts a level-based population structure and introduces Double Q-learning to adjust the number of levels in the population, which evades the local optimum to further improve the search efficiency. Extensive simulations demonstrate that the DQHPSO algorithm can increase the scheduling success ratio of time-triggered flows compared to other algorithms.

LEO satellite networks have become a necessary supplement to terrestrial networks aiming to provide worldwide, ubiquitous connectivity, especially in complicated areas (e.g., mountains, oceans, and disaster areas) where terrestrial network infrastructures are typically sparingly distributed or unavailable. However, the increasing computation-intensive Internet-of-Things (IoT) applications (e.g., real-time remote monitoring, intelligent transportation) require not only efficient and reliable communication but also massive computing capabilities. Constrained by the battery and computing resources, the computing tasks and data of applications have to be transmitted to remote cloud servers. This bandwidth limitation and high transmission delay in LEO networks will reduce the quality-of-service (QoS) of IoT applications. Recently, the combination of LEO networks and edge computing (i.e., Satellite Mobile Edge Computing, SMEC) offers significant opportunities to address these problems. The IoT devices can directly get the computing resources directly from satellites rather than remote servers, thus avoiding long-distance transmission. Considering the resource constraints on satellites, offloading policy plays a crucial role in whole system performance. In this paper, we design a hybrid offloading architecture, which applies a centralized training and distributed execution framework. Also, we propose a multi-agent actor-critic reinforcement learning algorithm, where a centralized “critic” is augmented with the global network state to ease the training procedure of distributed user equipments (UE) by evaluating the benefits of their decisions, while the UEs can adjust their policies according to the critic’s evaluation and choose their own decisions relying on their observations.

The Unmanned Aerial Vehicle (UAV) swarm has attracted widespread attention from both academia and industry. It has been widely adopted in disaster recovery, military communication, agricultural production, and industrial automation. In critical situations or places where communication infrastructure is lacking, deploying a UAV swarm network is a cost-effective solution. However, considering the high speed of UAV devices, designing an effective routing mechanism has been a challenging problem. In this paper, enlightened by the recent success of multi-agent reinforcement learning, we propose a multi-agent policy gradients-based UAV routing algorithm. We adopt a centralized training and decentralized executing framework, where a centralized training platform is implemented to guide the policy updating of each UAV node. Moreover, we introduce a counterfactual baseline scheme in our algorithm to improve the convergence speed. Extensive simulation results validate the effectiveness of the proposed algorithms compared to the state-of-the-art schemes.

Emerging latency-sensitive applications (e.g., industrial control, in-vehicle networks) require that the networks guaranteed data delivery with low, bounded latency. To meet this requirement, the IEEE 802.1 Working Group developed the time-sensitive networks (TSN) standard to enable deterministic communication on standard Ethernet. TSN technology is developed to enable deterministic communication using traffic scheduling and shaping technology. However, while the TSN standards define the mechanisms to handle scheduled traffic, it does not specify algorithms to compute fine-grained traffic scheduling policy. Current TSN flow scheduling schemes largely rely on a manual process, requiring knowledge of the traffic pattern and network topology features. Inspired by recent successes in applying reinforcement learning in online control, we propose a deep reinforcement learning aided no-waiting flow scheduling algorithm in TSN. Extensive simulations are performed to verify that our algorithm can find the optimal solution in an acceptable time.

Nowadays, the compelling applications of the Internet of Things (IoT) bring unexpected economic benefits to our daily lives. But at the same time, it also poses huge challenges to service providers. Diverse proprietary hardware (i.e., firewall and code conversion) have to be deployed in networks for meeting different applications’ requirements. Recently, network functions virtualization (NFV) is considered a promising technique. In the NFV-enabled architecture, network services can be implemented via a set of orderly virtual network functions (VNFs) on standardized compute nodes, which is termed service function chains (SFCs). However, with the explosion of IoT applications, embedding multiple SFCs in a shared NFV-enabled infrastructure becomes a challenging problem. Centralized schemes suffer from the scalability and private issue, while distributed schemes suffer from the nonconvergence problem. In this article, we propose a hybrid intelligent control architecture, which adopts the centralized training and distributed execution paradigm. A centralized critic is introduced to ease the training process of the distributed network nodes. Besides, considering the competitive behavior of users, we formulate the resource allocation problem as a multiuser competition game model. Based on this, we proposed a multiagent reinforcement learning-based SFCs deployment algorithm.

With the growth of the number of Internet of Things (IoT) devices and the emergence of new applications, satisfying distinct QoS in the same physical network becomes more challenging. Recently, with the advance of network functions virtualization and software-defined networking (SDN) technologies, the network slicing technique has emerged as a promising solution. It can divide a physical network into multiple virtual networks, therefore providing different network services. In this article, to meet distinct QoS in industrial IoT, we design a network slicing architecture over the SDN-based long-range wide area network. The SDN controller can dynamically split the network into multiple virtual networks according to different business requirements. On this basis, we proposed a deep deterministic policy gradient (DDPG) based slice optimization algorithm. It enables LoRa gateways to intelligently configure slice parameters (e.g., transmission power and spreading factor) to improve the slice performance in terms of QoS, energy efficiency, and reliability. In addition, to accelerate the training process across multiple LoRa gateways, we leverage the transfer learning framework and design a transfer learning-based multiagent DDPG algorithm.

Resource allocation is a difficult online decision-making problem in Software Defined Networks (SDN). Traditional algorithms usually formulated the network resource topologies as simple matrices or edge lists (e.g., adjacency matrices). However, the rapid growth of the network scale and data volume brings new challenges to the scalability and efficiency of these models. Recently, network representation learning (NRL) has been widely adopted in network modeling, which can embed network nodes and links into low-dimensional vectors. Specifically, as one of NRL techniques, knowledge graph (KG) integrates both pieces of knowledge and their relations, and therefore capturing relational information in the network. Therefore, in this paper, we adopt KG for SDN representation and propose a novel resource allocation scheme based on the relational information learned by KG embedding. We first extract relational information between network nodes, links and requests, and then embed them into low-dimensional vectors. Based on these vectors, we calculate resource allocation schemes by consecutively selecting relay nodes and links in consideration of both available resources and their relational vectors. The extensive simulations are conducted to evaluate our proposed algorithm in comparison to state-of-the-art schemes.

In recent years, satellite communications have played an increasingly important role in daily life. With the explosive growth of new businesses, the expectations for the performance and reliability of satellite communications are greater than ever. However, due to the unique characteristics of satellite node (e.g., fast transmission speed, saturation of resources), it brings unprecedented challenges for load balance in multiple satellite paths. In this paper, to overcome this issue, we proposed a multi-agent reinforcement learning aided load balance architecture. We formulate the load balance in satellites communications as a partially observable Markov decision process (POMDP). Besides, we adopt a multi-agent reinforcement algorithm named Collaborate Q-learning (CollaQ) in our architecture. In addition, some stimulation are performed to evaluate the correctness of our architecture and algorithm.

The past few years have witnessed the compelling applications of the Internet of Things (IoT) in our daily life. The explosive growth of the number of IoT devices also presents a great challenge in network security, especially the DDoS attack. Current DDoS defense mechanisms adopted out-of-band architecture, which is accomplished by a process that receives monitoring data from routers and switches, then analyzes that flow data to detect attacks. However, facing IoT devices growing rapidly, this out-of-band architecture confronted with limited processing capacity, bandwidth resources, and service assurance problems. Recently, with the development of the programming switch, it opens up new possibilities for in-network DDoS detection, where the detection algorithms could be directly implemented inside the routers and switches. Benefit from switch processing performance, the in-network mechanism could achieve high scalability and line speed performance. Therefore, in this article, we design a machine learning-based in-network DDoS detection framework. We implement the lightweight variational Bayes algorithm in each switch to detect the anomaly traffic. Besides, considering the shortage of training data in each switch, a centralized platform is introduced to synchronize parameters among distributed switches to realize collaborative learning. Extensive simulations are conducted to evaluate our proposed algorithm in comparison to some state-of-the-art schemes.

Load balancing is a difficult online decision-making problem in the current network. Recently, with the development of the programmable data-plane, it is feasible to perform flexibly load balance directly inside the network. This in-network load balance scheme can quickly adapt to the volatility of network traffic. However, previous in-network solutions are largely relying on the manual process. Inspired by recent successes in applying machine learning in online control, automating the in-network load balance process is thus appealing. But as a distributed control system, it behooves us to ask the critical question: “Can the distributed switches learn globally optimal scheduling policy and still be deployed in a distributed fashion to allow rapid reaction in real-time?” To tackle this question, we adopt a centralized learning and distributed execution framework and propose a multi-agent actor-critic reinforcement learning algorithm in this paper. The centralized “critic” is reinforced with the global network state and joint actions of all agents to ease the training process whilst distributed switches can take actions relaying on their local observations. In addition, a baseline scheme is introduced to solve the credit assignment problem in the multi-agent system. The extensive simulations are conducted to evaluate our proposed algorithm in comparison to state-of-the-art schemes.

In the past few years, satellites have been widely influenced in our daily life, from Global Positioning System(GPS) to Military investigation. Due to the limitation of computation power and insufficient energy, the satellite has to offload their data to the ground station for further processing. However, long-range transmission has a great impact on many real-time services, such as hotspot tracing. As a remedy, in this paper, we introduce space stations to offload computation tasks of Low Earth Orbit(LEO) satellites to reduce the transmission delay. We formulate the problem of computing resource allocation between LEO satellites and space stations based on double auction mechanisms. Then, we describe an algorithm for searching Nash equilibrium based on Experience-Weighted Attraction(EWA) which is executed after each participant. Auction participants can obtain information from other adversaries and accumulate experience and reflection. So they can complete transactions with other participants in a fuzzy environment and maximize the overall benefit. Simulation results show the convergence and effectiveness of the algorithm.

Past few years have witnessed the compelling applications of the satellite communications and networking in our daily life. Due to the extremely high moving speeds and limited networking resources of LEO satellites, how to optimize inter-satellite traffic has received amount of attention from both academia and industry. In this paper, we proposed a hybrid satellites network traffic control paradigm. In our architecture, the centralized platform collect the global state and the joint action from each agent during the training phase to ease the training, and during execution, the each agent can return the action to the local state through the trained policy. Besides, we adopt a multiagent actor-critic algorithms named Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments(MADDPG) to our architecture. In addition, some simulation results are presented to evaluate the correctness of our architecture and algorithm.

In recent years, we have witnessed the compelling application of the Internet of Things (IoT) in our daily life, ranging from daily living to industrial production. On account of the computation and power constraints, the IoT devices have to offload their tasks to the remote cloud services. However, the long-distance transmission poses significant challenges for latency-sensitive businesses, such as autonomous driving and industrial control. As a remedy, mobile edge computing (MEC) is deployed at the edge of the network to reduce the transmission delay. With the MEC joining in, how to allocate the limited computing resource of MEC is a critical problem to guarantee efficient working of the whole IoT system. In this article, we formulate the resource management among MEC and IoT devices as a double auction game. Also, for searching the Nash equilibrium, we introduce the experience-weighted attraction (EWA) algorithm performing behind each participant. With this AI method, auction participants acquire and accumulate experience by observing others' behavior and doing introspection, which accelerates the trading policy's learning process of each agent in such an opaque environment. Some simulation results are presented to evaluate the convergence and correctness of our architecture and algorithm.

The past few years have witnessed a wide deployment of low earth orbit (LEO) satellites communications and networking. With the explosive growth of new businesses, satellite network is expected to provide global coverage and high bandwidth availability service. Toward this end, Multipath TCP(MPTCP) is a promising transport protocol to use in LEO satellites networks. MPTCP can not only achieve seamless handover, but also enhance throughput by using multiple paths transmission mechanism. However, following the improvement of the performance and scalability, it also brings unprecedented challenges for congestion control of multiple sub-flows. Especially, currently works on the congestion control largely relies on a manual process which presents a poor performance in the high-dynamic complexity network environment. Inspired by the recent success of applying machine learning in many challenging control decision domains, such as video game, self-driving, we employ deep deterministic policy gradient for learning the optimal congestion control strategies by interacting with the underlying network environment. Some simulation results demonstrated the effectiveness and feasibility of our architecture and algorithms.

Past few years have witnessed the compelling applications of the remote sensing satellites in our daily life, ranging from the weather forecast to military surveillance. Due to the computation and power constraints, the LEO satellites have to download the remote sensing data to the ground stations for further processing. However, the long-distance transmission and the ionospheric interference is problematic for supporting the latency-sensitive remote sensing services, such as hotspot detection, hotspot tracing. As a remedy, in this paper, the space stations are introduced to offload the computation task of the remote sensing satellites to reduce the transmission delay. With the space station joining in, a three-tier intelligent remote sensing satellites operation system is constructed. In order to perform well, we study the computation resource allocation strategies in this three-tier system. We model the resource management and pricing problems among three players as a Stackelberg game, where the space stations act as the leaders, the ground stations as the followers, and the LEO satellites as the sub-followers. For searching the Nash equilibrium of this game, we apply ’WoLF-PHC’ algorithm for learning the optimal resource management strategies. In addition, some simulation results are presented to demonstrate the feasibility and performance of our architecture and algorithm.

Past few years have witnessed the compelling applications of the blockchain technique in our daily life ranging from the financial market to health care. Considering the integration of the blockchain technique and the industrial Internet of Things (IoT), blockchain may act as a distributed ledger for beneficially establishing a decentralized autonomous trading platform for industrial IoT (IIoT) networks. However, the power and computation constraints prevent IoT devices from directly participating in this proof-of-work process. As a remedy, in this treatise, the cloud computing service is introduced into the blockchain platform for the sake of assisting to offload computational task from the IIoT network itself. In addition, we study the resource management and pricing problem between the cloud provider and miners. More explicitly, we model the interaction between the cloud provider and miners as a Stackelberg game, where the leader, i.e., cloud provider, makes the price first, and then miners act as the followers. Moreover, in order to find the Nash equilibrium of the proposed Stackelberg game, a multiagent reinforcement learning algorithm is conceived for searching the near-optimal policy. Finally, extensive simulations are conducted to evaluate our proposed algorithm in comparison to some state-of-the-art schemes.

The past few years have witnessed a wide deployment of software defined networks facilitating a separation of the control plane from the forwarding plane. However, the work on the control plane largely relies on a manual process in configuring forwarding strategies. To address this issue, this paper presents NetworkAI, an intelligent architecture for self-learning control strategies in software defined networking networks. NetworkAI employs deep reinforcement learning and incorporates network monitoring technologies, such as the in-band network telemetry to dynamically generate control policies and produces a near optimal decision. Simulation results demonstrated the effectiveness of NetworkAI.