In beyond 5G multi-cell networks, cell-edge users generally experience poor communication quality due to their greater distance from the base station (BS), and increased interference from neighboring cells, which severely impacts their user experience. To address this issue, achieving fair and efficient resource scheduling is key to ensuring the quality of service for edge users. Therefore, this paper formulates a joint optimization problem of spectral subband selection and power control, aiming to maximize the proportional fair sum rate of the multi-cell network. However, most existing algorithms require instantaneous global channel state information, which results in poor scalability and is impractical, especially in highly dynamic wireless network environments with user mobility. Noting that the considered problem can be modeled as a decentralized partially observable Markov decision process, we propose a multi-agent deep reinforcement learning (MADRL) scheme based on a hierar chical centralized training and distributed execution framework (MADRL-HE), enabling agents to make spectral subband and transmit power selections using only local information and some outdated non-local information. Simulation results demonstrate that the proposed scheme features excellent scalability and fast convergence. Moreover, its proportional fair sum rate performance consistently outperforms the existing MADRL scheme in dynamic environments and surpasses centralized iterative optimization schemes in most dynamic scenarios.

This paper considers a 5G dense cellular network scenario where wireless channels are dynamically changing. To address the resource allocation problems in 5G dense networks, a distributed real-time and fair resource allocation algorithm based on a single deep Q-network (DQN) and multiple local deep neural networks (DNNs) architecture (DRFRA-SDML) is proposed. Specifically, each downlink is modeled as an agent. Each downlink is equipped with a local deep neural network (DNN), allowing each agent to input locally observed information into the local DNN to select spectral subband and transmission power in real-time. In the core network, the global experience replay buffer is utilized to collect local experiences collected by all local agents to train a global weight vector of the train DNN, which is shared by all local DNNs. This ensures that the computational complexity of each local agent does not depend on the size of the cellular network, resulting in excellent scalability. To achieve fairness between user devices, a reward function based on fairness has been designed. The simulation results demonstrate that DRFRA-SDML significantly outperforms IFP-FRA and FP-FRA-D in terms of sum rate in most scenarios, while ensuring fairness among user devices and achieving markedly better real-time performance.

With the emergence of 5G technology, the demand for data communication between mobile users has significantly increased, and the number of cellular network infrastructure and mobile devices has also grown rapidly. However, a large number of base stations conducting wireless communication simultaneously inevitably brings serious interference due to the limited spectrum resources and dense distribution. Since the channels in the 5G dense cellular network with mobile users are complex and it is difficult to capture the channel state, the power allocation scheme adapting to a dynamic environment has become an important issue. In this article, a multiagent deep Q-network (DQN) distributed algorithm based on environmental awareness (MADQN-EA) is proposed. Specifically, the downlink between each base station and the user is treated as an agent, and a multiagent distributed approach is developed to improve the scalability of the algorithm. In response to the time-varying nature of the 5G dense cellular network environment, an environmental awareness training method is adopted. This method provides the agent with the opportunity to observe more changes in the 5G dense cellular network environment during the training process. This design significantly enhances the robustness of the proposed algorithm under the changing channel conditions. The proposed MADQN-EA is compared to fractional programming with a perfect CSI (FP-PC), multiagent DQN with experienced instance transfer (MADQN-EIT), and the random power selection scheme (Random). Simulation results show that MADQN-EA is robust against dynamic environment and achieves a higher sum rate performance.

Considering the harsh and complex environments in Underwater Wireless Sensor Networks (UWSNs), where the malicious nodes exist, the secure routing design is investigated in this paper. The malicious nodes are a serious threat to the security of sensor networks. This paper proposes a secure routing protocol, named Trust Based Routing (TBR), that focus on how to evaluate and find the malicious nodes and then determine the reliable routing. The core idea of TBR is that a new trust management model is designed by considering the various historical behaviors of nodes in the interaction process, which is able to assess the trust value of a node based on past behaviors between nodes and protect against potential attacks in the network. And the extra factors such as the remained energy of nodes, the distance between nodes and the trustiness are included in the routing criterions. Finally, a strategy for selecting relay nodes is proposed based on these considerations. Simulation results show that the proposed TBR algorithm can effectively defend against attacks from inside the network and is also more efficient and reliable compared to other existing routing protocols.

Reliable bidirectional communication between the control center and manufacturing devices (MDs) along with efficient resource allocation are critical for the Industrial Internet of Things (IIoT). However, due to the limited network resources of IIoT devices and the intertwined nature of communication and computation resources, achieving efficient optimization of these resources presents significant challenges. In this article, we propose a multilayer communication architecture for the IIoT based on edge computing and cooperative communication technologies, where resource-constrained MDs upload data to edge servers (ESs) for processing. To address the issues of resource scarcity and coupling, we establish a bandwidth release model to analyze and quantify the relationship between computation and communication resources. This elucidates the interaction mechanisms between “transmission-computation” performance. Furthermore, to prevent excessive data upload to ESs, which could lead to node overload and network congestion, we employ game theory to develop a pricing mechanism for resource allocation, where ESs charge for computation services. Specifically, we construct a Lagrangian framework to determine a resource allocation scheme, aiming to maximize the utility of the factory. Simulation results indicate that compared to common methods employed in existing works, our strategy enhances factory utility by 29.79%.

In recent years, underwater acoustic sensor networks (UASNs) have played an increasingly important role in ocean exploration. However, underwater sensor networks suffers from severe propagation attenuation, limited energy and sensor mobility compared with terrestrial networks. Also, the value of sensing data is quite different in some applications of underwater data collection. To tackle these challenges, this paper proposes a hierarchical collection strategy based on value of information (VoI). Taking account of the mobility of nodes close to sea level, we divide the network into two layers according to the Ekman drift current model. In the upper layer, the nodes move violently with the sea water. We adopt opportunistic routing to allow these nodes to search for the appropriate next hop nodes actively. Meanwhile, nodes in the lower layer are clustered. According to the rarity of the data received in the historical data, we propose a novel mathematical formula to measure the VoI of the data, and define the ratio of received data value to energy consumption as the evaluation index of network energy efficiency. AUV-aided transmission and multi-hop transmission are utilized separately to determine the tradeoff between energy consumption and network performance. The choice of transmission mode depends on the VoI in a cluster. Simulation results indicate that the proposed strategy shows satisfactory performance in improving energy efficiency.

With the rapid development of artificial intelligence and Unmanned Aerial Vehicle (UAV) technology, AI-based UAVs are increasingly utilized in various industrial and civilian applications. This paper presents a distributed Edge-Cloud collaborative framework for UAV object detection, aiming to achieve real-time and accurate detection of ground moving targets. The framework incorporates an Edge-Embedded Lightweight (${{\rm{E}}^{2}}\rm{L}$) object algorithm with an attention mechanism, enabling real-time object detection on edge-side embedded devices while maintaining high accuracy. Additionally, a decision-making mechanism based on fuzzy neural network facilitates adaptive task allocation between the edge-side and cloud-side. Experimental results demonstrate the improved running rate of the proposed algorithm compared to YOLOv4 on the edge-side NVIDIA Jetson Xavier NX, and the superior performance of the distributed Edge-Cloud collaborative framework over traditional edge computing or cloud computing algorithms in terms of speed and accuracy.

Smart Grid needs real-time monitoring of power equipment status and optimization of power distribution. In order to meet the needs of mass communication and data transmission, smart grid needs sufficient spectrum resources to ensure fast and reliable data transmission. However, the limitation of spectrum resources and the interference caused by multisystem coexistence have become the bottleneck of the development of smart grid. This article pointed out the defect of the existing spectrum resources management mode. Based on the Taguchi Quality Assessment Theory, we developed a cost model for a utility company, which used a decoding and forward cooperative relay strategy to transmit downlink horizontal propagation to a data aggregation unit. This article analyzed the bilateral auction process between utility company and licensed user to maximize social welfare, and determined the optimal spectrum transaction to improve the utilization of spectrum resources.

The rapid growth of the Industrial Internet of Things (IIoT) has brought attention to the critical issue of communication resource allocation. In industrial environments, efficiently utilizing communication resources to meet the demands of large-scale device connectivity and data transmission poses a significant challenge. This paper proposes a novel approach based on relaying with Intelligent Reflective Surfaces (IRS) to address the scarcity of communication resources in the IIoT. The method leverages decode-and-forward (DF) relaying and Intelligent Reflective Surfaces (IRS) to support data transmission over wireless channels and introduces a price mechanism to tackle the resource allocation benefit problem. Moreover, we employ the Non-dominated Sorting Genetic Algorithm-II (NSGA-II), a multi-objective optimization genetic algorithm, to achieve equitable resource allocation and balance the dual-objective benefit problem between edge servers and factories.

With the development of cloud manufacturing, the automation of storage scheduling becomes popular in the steel industry. However, the high customization of steel plates makes the storage scheduling too complex to be optimized. To overcome this challenge, we propose to utilize the big data of steel plate orders and warehouse status to optimize the storage scheduling of steel plates. The agglomerative hierarchical clustering is first adopted to reduce the complexity of excessive steel plate specifications, then an optimization problem is defined to formulate the storage scheduling of steel plates with safety. A two-stage heuristic (TSH) algorithm is proposed to solve the optimization problem with low complexity. In TSH, steel plates are first assigned to multiple stacks, and then the arrangement of each stack is determined. Experiments are executed based on a cloud manufacturing platform for steel plates production and storage, and the results prove the effectiveness of the proposed works.

In smart grid, serious packet loss often occurs in the process of information interaction between the Utilities and customers, which results in supply-demand deviation and further increases the cost of the Utilities. In order to improve the information transmission performance of communication networks, the Utilities purchase relay service from telecom operator to help data aggregator units (DAU) transfer information to gateway (GW), so as to improve communication quality and reduce the cost of the Utilities. Second, in order to solve the problem of telecom operators’ energy reduction and reduce the cost of purchasing energy, we utilize energy supply point (ESP) to collect the surplus energy of retail customers for energy supply, and telecom operator pays a certain amount of remuneration to ESP in exchange for ESP to continuously supply energy to telecom operator. Then, we establish a three-stage game method and system model between the Utilities, telecom operator and ESP, and propose the relay power allocation and energy transaction pricing strategy. Due to the real-time change of energy demand, we consider two situations of energy oversupply and conservative supply, and use the backward induction method and iterative algorithm to obtain the equilibrium solution of the Stackelberg game, including the unit energy price of ESP, total power of telecom operator, the proportion of transmission power allocated to the relay service, and payment scheme of the Utilities. Simulation results show that the proposed algorithm can quickly and accurately converge to the optimal solution of the problem, and the method can improve the stability of demand-side regulation, reduce the cost of the Utilities and increase the profit of telecom operator.

With the advent of Industry 4.0 and the popularization of smart terminal equipment, the interaction between industrial field information systems and production equipment has intensified. To meet the real-time transmission of time-triggered flow and the coordinated transmission of best effort flow, time-sensitive network-related technologies are used to implement flow queue forwarding by strictly following the gate control list. First, response-time analysis method is proposed to predict the upper bound of delay under a scheduling model following IEEE 802.1Qbv. Second, according to response-time analysis, a deadline monotonic scheduling algorithm with temporary priority expansion is proposed to divide the priority into more levels, which is not limited by the queue type, so as to ameliorate the transmission sequence of switch export flow. Finally, an adaptive priority adjustment scheduling algorithm with temporary priority expansion is designed to construct the optimal scheduling method further improving the scheduling success rate and reducing worst-case end-to-end delays. Compared with similar algorithms, the algorithm proposed improves the scheduling success rate by at least 30%, reduces the total delay by at least 21% and the TT flow delay by 11% in high network utilization conditions.

This paper studied the problem of autonomous channel matching for device-to-device (D2D) pairs in a multiuser cellular network. The goal of each D2D pair is to match an optimal wireless channel to maximize its reward. The reward is defined as the rate of the D2D pairs and limited by the SINR (Signal to Interference plus Noise Ratio) of the cellular user on the current channel. This strategy maximizes a certain network D2D throughput in a distributed manner without requiring online coordination or message exchange among users. We describe this problem as a random non-cooperative game with multiple players (D2D pairs), where each player becomes a learning agent, whose task is to learn its best strategy (based on locally observed information). Then, we designed a multi-user learning algorithm based on double deep Q-network (DDQN), which converged to Nash equilibrium (NE) of mixed strategy. After simulation verification, the algorithm can enable each user to obtain a best strategy to obtain a high communication rate through online or offline learning. And the algorithm has a faster convergence speed compared to the similar method.

Femtocell network is regarded as the potential and effective technique to improve the capacity and coverage of traditional cellular networks. One of the challenges is how to access the network and manage the interference among different users. Compared with other access strategies, hybrid access strategy allows femtocell base stations (FBSs) to provide preferential access to femtocell users (FUEs) while other users can access nearby FBS with specific restrictions. In this paper, a robust downlink power control scheme is studied in two‐tier femtocell networks, where femtocells share the same frequency with macrocell. A hierarchical game framework that takes the different users' requirements into consideration is constructed. In addition, as the link gains are actually uncertain in dynamic environment, probabilistic constraints are used to describe the uncertainty. Then two sub‐problems are obtained to maximise the sum rate of macrocell and femtocells, respectively and guarantee the quality of service (QoS) of different users. To tackle the nonlinear and nonconvex optimization problem, successive convex approximation is introduced. And the practical iterative power allocation algorithm is provided. Finally, numerical results show that the proposed scheme is effective in aspect of energy saving and QoS guarantee under dynamic environment.