The integration of 5G New Radio (NR) into the manufacturing industry stands as a pivotal aspect of the Industrial Internet of Things (IIoT) paradigm, mitigating constraints posed by traditional wired communication technologies in terms of flexibility, mobility, and adaptability. However, ensuring comparable performance levels presents challenges in the scheduling design of 5G NR networks, particularly due to the emphasis on uplink transmissions within industrial applications. Indeed, current uplink schedulers proposed by the 3rd Generation Partnership Project (3GPP) exhibit performance trade-offs, forcing the network to acquire knowledge of uplink traffic and faltering in the presence of heterogeneous traffic sources, commonplace in industrial environments. In this paper, we thus propose a Reinforcement Learning (RL)-based approach that dynamically assigns users to the most suitable uplink scheduler without prior or acquired knowledge of the different traffic patterns. Through 3GPP-compliant network simulations, we demonstrate the effectiveness of our approach in optimizing network performance while addressing the complexities of heterogeneous IIoT environments and maintaining standard compliance.

In this paper, we propose a novel framework for designing a fast convergent multi-agent reinforcement learning (MARL)-based medium access control (MAC) protocol operating in a single cell scenario. The user equipments (UEs) are cast as learning agents that need to learn a proper signaling policy to coordinate the transmission of protocol data units (PDUs) to the base station (BS) over shared radio resources. In many MARL tasks, the conventional centralized training with decentralized execution (CTDE) is adopted, where each agent receives the same global extrinsic reward from the environment. However, this approach involves a long training time. To overcome this drawback, we adopt the concept of learning a per-agent intrinsic reward, in which each agent learns a different intrinsic reward signal based solely on its individual behavior. Moreover, in order to provide an intrinsic reward function that takes into account the long-term training history, we represent it as a long short-term memory (LSTM) network. As a result, each agent updates its policy network considering both the extrinsic reward, which characterizes the cooperative task, and the intrinsic reward that reflects local dynamics. The proposed learning framework yields a faster convergence and higher transmission performance compared to the baselines. Simulation results show that the proposed learning solution yields 75% improvement in convergence speed compared to the most performing baseline.

In view of the challenges foreseen in futuristic massive IoT (mIoT) scenarios, characterized by a huge deployment of energy-constrained IoT devices, we propose an efficient next-generation multiple access (NGMA) scheme where several innovative solutions combine and integrate. First, we aim to maximize the spectral efficiency of the physical uplink-shared channel (PUSCH) by adopting the most appropriate nonorthogonal multiple access (NOMA) technique and by introducing an efficient contention-based approach for data transmission in the unused PUSCH resources. Second, on the basis of the traffic load, the proposed scheme dynamically sets the UpLink (UL) radio resources dimensioning by searching the optimal tradeoff between lowering the collision probability in the physical random access channel (PRACH) and providing enough radio resources in the PUSCH to permit the data transmission to all the succeeded access requests. Third, it is enriched by a series of strategic procedures to make feasible the traffic load estimation, even when the PRACH is in overload condition. Fourth, we exploit artificial intelligence technologies to further optimize the performances obtained. The proposed scheme is compared with other benchmark schemes available in the literature. It outperforms them in the most important metrics for the mIoT scenario (i.e., spectral efficiency and energy consumption), especially in high load conditions, which is the expected requirement for Beyond 5G and 6G networks.

To tackle the heterogeneous requirements of beyond 5G (B5G) and future 6G wireless networks, conventional medium access control (MAC) procedures need to evolve to enable base stations (BSs) and user equipments (UEs) to automatically learn innovative MAC protocols catering to extremely diverse services. This topic has received significant attention, and several reinforcement learning (RL) algorithms, in which BSs and UEs are cast as agents, are available with the aim of learning a communication policy based on agents' local observations. However, current approaches are typically overfitted to the environment they are trained in, and lack robustness against unseen conditions, failing to generalize in different environments. To overcome this problem, in this work, instead of learning a policy in the high dimensional and redundant observation space, we leverage the concept of observation abstraction (OA) rooted in extracting useful information from the environment. This in turn allows learning communication protocols that are more robust and with much better generalization capabilities than current baselines. To learn the abstracted information from observations, we propose an architecture based on autoencoder (AE) and imbue it into a multi-agent proximal policy optimization (MAPPO) framework. Simulation results corroborate the effectiveness of leveraging abstraction when learning protocols by generalizing across environments, in terms of number of UEs, number of data packets to transmit, and channel conditions.

With the emerging of the massive Machine Type Communication usage scenario, the legacy Random Access (RA) procedures need to be fully renewed to meet the huge number of accesses and the energy-constrained device requirements. In this context, the strategy of transmitting tagged preambles as access requests is gaining importance. In fact, it can reduce the number of signaling transmissions per access attempt, the overall number of phases in the access procedure, and, accordingly, energy consumption per device. However, the effectiveness of detection strategies of tagged preambles by the gNB receiver plays a fundamental role on the RA procedure performance. Up to now, the performance of these advanced detection procedures has been obtained only by running a large number of simulations that are typically highly time-consuming. To the best of our knowledge, this paper presents the first analytical model to analyze the correct detection of both the preamble and the tag transmitted by each device in the presence of interference, due to other preambles and tags, and of noise. The high accuracy of the proposed model is verified through simulations. In addition, we show how our analytical study can be a good tool to investigate and derive innovative detection strategies.

The massive machine-type communication (mMTC) usage scenario, also known as the massive Internet of Things (mIoT), involves a huge number of MTC devices having high requirements on increased battery lifetime, which autonomously transfer small amounts of data with relaxed delay requirements, without human intervention. The current cellular network is unsuitable for this scenario, due to the limited uplink resources allocated to the physical random access channel (PRACH) and to the physical uplink shared channel (PUSCH). With this in mind, in this article, we propose a new framework, customized for massive MTC services, that includes a joint control of the dynamic resource allocation between the PRACH and the PUSCH, and a new random access (RA) procedure based on an adaptive access class barring (ACB) scheme that appropriately spreads RA reattempts in time. In addition, to further increase the transmission efficiency, we adopt the sparse code multiple access (SCMA) technique for PUSCH resources, because SCMA results as the most promising nonorthogonal multiple access (NOMA) technique to support massive MTC connectivity with small-size data. The simulation results show that the proposed control framework significantly improves the number of succeeded communications in comparison with the other proposals available in the literature. Furthermore, an energy consumption model is introduced to show the considerable energy saving achieved by the MTC devices which adopt our joint control scheme.

With the increment in users' data demand and the emergence of applications with stringent Quality of Service requirements, important improvements in cellular network architecture need to be made. In this paper we explore the symbiosis of three key communication technologies in 5G: Device-to-Device communications, 60 GHz unlicensed band transmissions and adaptive beamforming techniques. It provides notable energy saving, improvement in the system spectral efficiency, high achievable data rate and strong interference reduction. Despite these benefits, there are some issues to be analyzed: a limited communication range due to the high path loss, and the interferences among D2D communications, especially if users' density is high. To manage these problems, on the basis of a mmWave network architecture for high TIE density indoor environments, we propose a new centralized control for radio access and an efficient multi-criteria scheduling algorithm using greedy graph vertex-coloring techniques. We aim to enhance transmission efficiency, exploiting concurrent transmissions, maximize throughput and minimize end-to-end delay, while taking into account to keep the computational load low.