At complex intersections, it has become a consensus to address the perception limitations (such as blind spots, the difficulty of accurately obtaining road condition information in distant areas or under adverse weather conditions) faced by single-vehicle intelligence through vehicle-infrastructure cooperation. However, under the constraints of limited spectrum resources, the sharply rising in the number of connected vehicles, and the pressing requirement for effective utilization of electric power, there has been no study on how to design a scheme that assists roadside unit (RSU) completing roadside cooperative message delivery (RCMD) with minimal transmit power. Towards this end, this paper proposes a general and energy-efficient RCMD scheme based on multicarrier multigroup multicasting rate-splitting multiple access. We formulate a joint optimization problem involving the RSU’s transmit power, along with the power and size coefficients of each message. Then we design a deep reinforcement learning-assisted bilevel resource allocation algorithm to solve this problem. Finally, we collect extensive numerical results, using the message delivery success probability and RSU’s transmit power as performance metrics, from the proposed scheme and multiple benchmarks. Results indicate that the proposed scheme not only exhibits strong adaptability (i.e., it can be deployed at any intersection regardless of the number of lanes, vehicles, or traffic complexity), but also significantly reduces the RSU’s transmit power while maintaining high message delivery success probability.

Traditional single-vehicle intelligence system faces challenges such as undetectable spots and perception performance bottlenecks due to limitations in sensor perception angles, ranges, and accuracy, which are particularly pronounced in complex intersection. Vehicle-infrastructure cooperative mechanism has been widely recognized as a promising solution to address challenges faced by single-vehicle intelligence. However, against the backdrop of limited spectrum resources and the sharply rising in the number of connected vehicles, how to efficiently deliver cooperative messages from roadside unit to vehicles is often overlooked. Towards this end, we propose a roadside cooperative message delivery scheme based on multicarrier multigroup multicast rate-splitting multiple access, considering the rarely explored case of transmitting messages with limited size under delay constraint. Then we focus on the critical joint optimization problem of message size and power allocation, with consideration for imperfect channel state information at the transmitter. Subsequently, a multi-agent deep reinforcement learning based resource allocation algorithm is designed to solve this joint optimization problem, exhibiting robustness to dynamic changes in vehicle density and message size. Finally, we analyze through extensive numerical results the impacts of various factors on message delivery success probability.

In the context of 5G and beyond 5G, radio access network (RAN) slicing emerges to enable differentiated services via the instantiation of virtualized logical networks. Despite its promising potential, the resource optimization of RAN slicing confronts significant challenges stemming from the scarcity of spectrum resources, the intricacies of tradeoff between slice service quality and slice security, the mobility of users, and the complexity of wireless interference in multi-cell environments. To address these challenges, we propose a heuristic-assisted multiagent deep reinforcement learning-based resource allocation scheme for RAN slicing. This scheme aims to augment inter-slice resource isolation for security (quantified as isolation rate) while efficiently accommodating diverse requirements across slices (quantified as satisfaction rate). Through extensive numerical results, we exhibit that our proposed scheme adeptly adapts to multiple user mobility patterns, achieving superior performances in terms of satisfaction rate and isolation rate.

Recently, Reduced Capability (RedCap) New Radio (NR) User Equipment (UE), which has smaller cost, lower complexity, and longer battery life than normal NR UE, was introduced by 3GPP in R17. In particular, the UE energy consumption is an important metric of interest. Small Data Transmission (SDT) technology has been proposed by 3GPP to save UE power, which allows UEs to directly transmit small-sized uplink payloads inInactivestate without transitioning toConnectedstate. However, for a large number of RedCap UEs with bursty traffic, how to minimize UE energy consumption while ensuring the reliability of uplink payload transmission is still a challenge. Towards this end, we propose a novel Non-Orthogonal Multiple Access (NOMA)-enhanced SDT scheme to tackle this challenge. Considering that the power level pool is an important component of NOMA to achieve performance gains and the power level pool design problem in multi-cell scenarios is still rarely studied, we present a Multi-Agent Dueling Double Deep Q Network (MAD3QN)-based algorithm to further improve the performances of the proposed scheme by optimizing the power level pool for each gNB with practical imperfect Successive Interference Cancellation (SIC). Extensive numerical comparisons among the proposed scheme and multiple benchmarks show that the proposed scheme (with appropriate user pairing) can significantly improve the transmission reliability of large-scale RedCap UEs with a small cost of energy consumption.

5G and beyond are required to support a variety of emerging services that impose different requirements on the network. Radio Access Network (RAN) slicing is a promising candidate technology, and resource allocation issues related to it have received extensive attention. However, most existing literature on RAN slicing resource allocation either ignores actual network interference or focuses primarily on the QoS of slices but not slicing security. Towards this end, we propose a QoS and security-oriented slicing resource allocation scheme in a multi-cell and multi-slice scenario, where actual link interference is carefully considered. We then formulate a problem of maximizing security (i.e., isolation rate) subject to QoS (i.e., satisfaction rate) constraint. Finally, a multi-agent deep reinforcement learning-based solution is designed to solve this problem, and extensive numerical results demonstrate the superior performance of the proposed scheme.

We investigate in this article joint trajectory design and resource allocation for secure air–ground integrated Internet of Things (IoT) networks with unmanned aerial vehicle (UAV) jamming and device-to-device (D2D) enhancement. By jointly optimizing ground user (GU) scheduling, UAV flight trajectory, and transmit power, we are able to maximize the minimum system secrecy rate of UAV and D2D communications. The formulated optimization problems of the two typical network scenarios, i.e., with and without UAV jamming, are challenging to be solved due to the corresponding nonsmooth and nonconcave objective functions. Therefore, we propose alternating iterative algorithms to solve the problems by employing the successive convex approximation and block coordinate descent methods. Extensive results indicate that the proposed joint optimization schemes can effectively improve the secrecy communication performance under different spatial distributions of GUs.

Power-domain Non-Orthogonal Multiple Access (NOMA) and Ultra-Dense Network (UDN) are promising candidates to cope with the massive access challenge of Machine-Type Communications (MTC). The power level pool is crucial for NOMA to bring performance gains. The existing related literatures rarely consider the power level pool design problem, or only resolve it in the single-cell scenario. However, this problem in multi-cell scenario is more complex and difficult to solve due to the presence of inter-cell interference. Towards this end, we propose a Non-Orthogonal Random Access (NORA) scheme to enable the coexistence of Human-Type Communications (HTC) and MTC for 5G and beyond UDN, where the power level pool design problem in multi-cell scenario is our focus. In order to deal with the complexity caused by multiple optimization objectives and inter-cell interference, we present a Multi-Agent Reinforcement Learning (MARL)-based solution to solve this problem, where each small base station acts as an agent to learn a suitable gap between adjacent power levels. Extensive numerical comparisons demonstrate the superior performances of our proposed scheme in multiple perspectives.

With the fast development of emerging Internet of Everything applications, human-type communications (HTC) and machine-type communications (MTC) will inevitably coexist in future 6G cellular networks. To support massive connectivity while fulfilling diverse requirements of both HTC and MTC, we present a device-to-device aided rate splitting multiple access (RSMA) scheme by encoding both the MTC devices’ messages and HTC users’ common messages into a general common data stream in each cell (or group). Nevertheless, such deploying strategy may bring challenges in complex resource allocation and transmission modes selection. In view of this, we introduce a simple received signal strength (RSS) based transmission modes selection scheme, through which the RSS-threshold selection problem is formulated to maximize the HTC and MTC sum rates. Considering the computational complexity and scalability, we employ a multi-agent reinforcement learning based algorithm for each small base station to choose the optimal RSS threshold thus to achieve maximum sum rate while maintaining massive connectivity. Simulation results reveal our proposed RSMA deploying strategy outperforms non-orthogonal multiple access (NOMA) in system coverage while maintaining high-level system rate. Besides, the proposed MARL based scheme can further improve the system sum rate and coverage for both the HTC and MTC.

To fulfill the stringent requirements brought by human-type communication (HTC) along with massive machine-type communication (MTC), device-to-device (D2D) and non-orthogonal multiple access (NOMA) techniques will inevitably be incorporated into dense cellular networks to cater massive connectivity and maintain high spectral efficiency. However, such combination may lead to very complex network topologies and bring challenge in resource allocation, interference management and transmission mode selection. Note the received signal strength (RSS) is an important factor for cellular and D2D mode selection, it can affect multi-access mode determination in D2D-aided HTC/MTC dense NOMA systems. Therefore, the RSS threshold of each cell has great impact on system performance and should be carefully tuned. To this end, we formulate the RSS-threshold selection problem as a decentralized partially observable Markov decision process to maximize the performance for downlink and uplink communications. Accordingly, we employ a multi-agent reinforcement learning based scheme wherein each small base station acts as an agent and chooses the optimal RSS threshold to achieve maximum sum rate by interacting with the environment continuously. Extensive simulation results reveal our proposed scheme can improve the system sum rate and coverage by enhancing the connectivity of massive HTC and MTC devices via D2D and NOMA techniques.

In 5G vehicle-to-everything (V2X) systems, the scarcity of spectrum resources and the inefficiency of resource allocation make vehicle-to-vehicle (V2V) communications that require stringent latency and high reliability still a challenge. Existing relevant literatures either focus on vehicle-to-infrastructure (V2I) communications, or consider scenarios where the vehicle roles (transmitter or receiver) are fixed in V2V communications, or study the resource allocation within a single cell. What’s more, considering the high-speed movement of vehicles across the coverage regions of multiple cells, the above resource allocation problem becomes even more challenging. Toward this end, we consider in this paper a multi-cell 5G V2X system where V2V links and V2I links coexist and the vehicle roles are not fixed, and propose a sparse code multiple access-based centralized resource allocation scheme, so as to address the above challenges. In view of the fact that our formulated maximizing packet reception ratio problem is a combinatorial optimization problem and is NP-hard, we design a three-stage heuristic yet joint alternating optimization approach to obtain a suboptimal solution. Extensive numerical results demonstrate the superior performances of the proposed scheme in multiple perspectives.

With the rapid development of machine-type communications (MTC), the future communication architecture needs to provide services for both human-type communications (HTC) and MTC with unique characteristics. The huge connections from MTC bring serious challenges to the existing wireless network. Ultra-dense network (UDN), a promising candidate technology, can support massive device access through dense deployment of small base stations (SBSs). Different from the resource management in traditional wireless network with single base station (BS), the resource allocation problem at BS level is more prominent in UDN, and the diversity of devices will make this problem more complicated. In view of this, we investigate the joint optimization of massive access and resource management in the UDN where HTC and MTC coexist. Considering the computational complexity and scalability, we propose a multi-agent deep reinforcement learning based SBS state selection scheme, in which each SBS acts as an agent and selects the optimal state between active and idle by continuously interacting with the environment. In addition, we adopt the power-domain non-orthogonal multiple access to further improve system throughput, and use grant-based and grant-free access manners for HTC and MTC respectively, so as to meet their unique characteristics. Extensive numerical results demonstrate the superior performances of proposed scheme in multiple perspectives.

The explosive growth of machine-type communications (MTC) devices poses critical challenges to the existing cellular networks. Therefore, how to support massive MTC devices with limited resources is an urgent problem to be solved. Bursty traffic is an important characteristic of MTC devices, which makes it difficult for agents to learn useful experience and has a negative impact on model convergence. However, most existing reinforcement learning-based literatures assume that devices have saturate data. Towards this end, we propose two distributed Q-learning aided uplink grant-free non-orthogonal multiple access (NOMA) schemes (including all-devices distributed Q-learning (ADDQ) scheme and portion-devices distributed Q-learning (PDDQ) scheme) to maximize the number of accessible devices, where the bursty traffic of massive MTC devices is carefully considered. In order to reduce the dimension of scheduling space and mitigate the impact of bursty traffic, the idea of grouping devices as well as transmission resources and the intermittent learning mode are adopted in our schemes. Extensive numerical results demonstrate the advantages of proposed schemes from multiple perspectives.

The industrial Internet of Things has become the new driving force for the automobile industry, making people's travel increasingly convenient. However, there are still a multitude of challenges that need to be tackled, including but not limited to illegal driver detection, legal driver identification, and driving behavior evaluation. At present, many researchers have attempted to solve issues of illegal driver detection and legal driver identification by using deep learning network, but there are still quite a few limitations in the collection and analysis of driving behavior data. Moreover, the problem of driving behavior evaluation has been paid little attention. Therefore, in this article we conduct a comprehensive study on driving behavior habits and establish a multitask learning (MTL) network to solve the abovementioned problems. First, we collect original data from a real vehicle and extract the driving behavior characteristics. Then, a novel MTL network composed of long short-term memory network, support vector domain description model and feedforward neural network is established, which achieves illegal driver detection, legal driver identification, and driving behavior evaluation. Extensive experiments illustrate that the proposed MTL network not only supports parallel learning to reduce time and space costs, but also has excellent performances and robustness for the three tasks.

Based on the characteristics of time domain and frequency domain recognition theory, a recognition scheme is designed to complete the modulation identification of communication signals including 16 analog and digital modulations, involving 10 different eigenvalues in total. In the in‐class recognition of FSK signal, feature extraction in frequency domain is carried out, and a statistical algorithm of spectral peak number is proposed. This paper presents a method to calculate the rotation degree of constellation image. By calculating the rotation degree and modifying the clustering radius, the recognition rate of QAM signal is improved significantly. Another commonly used method for calculating the rotation of constellations is based on Radon transform. Compared with the proposed algorithm, the proposed algorithm has lower computational complexity and higher accuracy under certain SNR conditions. In the modulation discriminator of the deep neural network, the spectral features and cumulative features are extracted as inputs, the modified linear elements are used as neuron activation functions, and the cross‐entropy is used as loss functions. In the modulation recognitor of deep neural network, deep neural network and cyclic neural network are constructed for modulation recognition of communication signals. The neural network automatic modulation recognizer is implemented on CPU and GPU, which verifies the recognition accuracy of communication signal modulation recognizer based on neural network. The experimental results show that the communication signal modulation recognizer based on artificial neural network has good classification accuracy in both the training set and the test set.

Large-scale connectivity support is a critical challenge in the massive machine-type communications scenario. Grant-free random access (RA) is a promising solution because it can reduce severe signaling overhead in contention-based RA procedure. However, there will still be collisions due to the random selection of spectrum resources by the devices. Therefore, we propose a distributed Q-learning-assisted grant-free RA scheme to alleviate the collisions between devices. Considering the characteristic of the machine-type communications devices with bursty traffic, the random packet arrival model is adopted in this paper. In order to cope with the difficulties brought by the random transmission of devices to Q-learning, an action reward based on the active probabilities of devices is designed. In addition, we introduce the power domain nor-orthogonal multiple access to further enhance the number of accessible devices. Numerical results demonstrate the advantages of the proposed scheme from the devices' successful access probability.

In this paper, multiple machine learning-enabled solutions are adopted to tackle the challenges of complex sensing model in cooperative spectrum sensing for non-orthogonal multiple access transmission mechanism, including unsupervised learning algorithms (K-Means clustering and Gaussian mixture model) as well as supervised learning algorithms (directed acyclic graph-support vector machine, K-nearest-neighbor and back-propagation neural network). In these solutions, multiple secondary users (SUs) collaborate to perceive the presence of primary users (PUs), and the state of each PU need to be detected precisely. Furthermore, the sensing accuracy is analyzed in detail from the aspects of the number of SUs, the training data volume, the average signal-to-noise ratio of receivers, the ratio of PUs' power coefficients, as well as the training time and test time. Numerical results illustrate the effectiveness of our proposed solutions.