Intent-based network automation is a promising tool that enables easier network management; however, certain challenges must be addressed effectively. These are: 1) processing intents, i.e., identification of logic and necessary parameters to fulfill an intent, 2) validating an intent to align it with current network status, and 3) satisfying intents via network optimizing applications. This paper addresses these points via a three-fold strategy to introduce intent-based automation for modern 5G architectures. First, intents are processed via a lightweight Large Language Model (LLM). Secondly, once an intent is processed, it is validated against future incoming traffic volume profiles (high or low). Finally, a series of network optimization applications has been developed. With their machine learning-based functionalities, they can improve certain key performance indicators such as throughput, delay, and energy efficiency. In the final stage, using an attention-based hierarchical reinforcement learning algorithm, these applications are optimally initiated to satisfy the intent of an operator. Our simulations show that the proposed method can achieve at least a 12% increase in throughput, a 17.1% increase in energy efficiency, and a 26.5% decrease in network delay compared to the baseline algorithms.

Federated learning (FL) is a promising technique for learning-based functions in wireless networks, thanks to its distributed implementation capability. On the other hand, distributed learning may increase the risk of exposure to malicious attacks where attacks on a local model may spread to other models by parameter exchange. Meanwhile, such attacks can be hard to detect due to the dynamic wireless environment, especially considering local models can be heterogeneous with non-independent and identically distributed (non-IID) data. Therefore, it is critical to evaluate the effect of malicious attacks and develop advanced defense techniques for FL-enabled wireless networks. In this work, we introduce a federated deep reinforcement learning-based cell sleep control scenario that enhances the energy efficiency of the network. We propose multiple intelligent attacks targeting the learning-based approach and we propose defense methods to mitigate such attacks. In particular, we have designed two attack models, generative adversarial network (GAN)-enhanced model poisoning attack and regularization-based model poisoning attack. As a counteraction, we have proposed two defense schemes, autoencoder-based defense, and knowledge distillation (KD)-enabled defense. The autoencoder-based defense method leverages an autoencoder to identify the malicious participants and only aggregate the parameters of benign local models during the global aggregation, while KD-based defense protects the model from attacks by controlling the knowledge transferred between the global model and local models. The simulation results demonstrate that the proposed attacks can degrade the network performance by 34% and 77%, and lead to lower throughput and energy efficiency. On the other hand, our proposed defense schemes can effectively protect the system from attacks. The system performance can be recovered to approximately 95% of a secure system by using the proposed KD-based defense.

In 5G networks, network slicing has emerged as a pivotal paradigm to address diverse user demands and service requirements. To meet the requirements, reinforcement learning (RL) algorithms have been utilized widely, but this method has the problem of overestimation and exploration-exploitation trade-offs. To tackle these problems, this paper explores the application of self-play ensemble Q-learning, an extended version of the RL-based technique. Self-play ensemble Q-learning utilizes multiple Q-tables with various exploration-exploitation rates leading to different observations for choosing the most suitable action for each state. Moreover, through self-play, each model endeavors to enhance its performance compared to its previous iterations, boosting system efficiency, and decreasing the effect of overestimation. For performance evaluation, we consider three RL-based algorithms; self-play ensemble Q-learning, double Q-learning, and Q-learning, and compare their performance under different network traffic. Through simulations, we demonstrate the effectiveness of self-play ensemble Q-learning in meeting the diverse demands within 21.92% in latency, 24.22% in throughput, and 23.63% in packet drop rate in comparison with the baseline methods. Furthermore, we evaluate the robustness of self-play ensemble Q-learning and double Q-learning in situations where one of the Q-tables is affected by a malicious user. Our results depicted that the self-play ensemble Q-learning method is more robust against adversarial users and prevents a noticeable drop in system performance, mitigating the impact of users manipulating policies.

Radio access technology is crucial in both 5G and 6G cellular networks, providing differentiated services that demand reliability, low latency, and high throughput. To meet these requirements, machine learning (ML) has demonstrated considerable progress by facilitating resource allocation. However, these ML techniques can be susceptible to attacks, and the jamming attack is one of the most considered attacks in the literature, disrupting network functionality by sending interference signals. This paper, to the best of our knowledge for the first time, examines the vulnerability of radio access networks (RANs) to jamming attacks on resource allocation of a transfer reinforcement learning (TRL) based system and provides a mitigation approach to such attacks. A system model is presented for RAN slicing, followed by an introduction of the TRL algorithm for resource allocation. Afterward, we investigate covert patterned jamming attack (CPJA) on the TRL algorithm in downlink communication which decreases system throughput by 17% and 38.14% in the expert and learner agents and increases latency by 7.36% and 9.37% respectively. In addition, we propose a neural network (NN) solution to mitigate the CPJA trained on the network side and provide the trained NN model to the users' equipment (UEs) to eliminate interference from the signal by the filter. The trained NN is applied to predict the future activity of the interference generated by the attacker. Attack mitigation reduces the impact of the attack while the system's throughput suffers a 6% and 1.8% degradation, and its latency increases by 6.5% and 3.83% compared to the original system for expert and learner agents, respectively.

Federated learning (FL) allows distributed partic-ipants to train machine learning models in a decentralized manner. It can be used for radio signal classification with multiple receivers due to its benefits in terms of privacy and scalability. However, the existing FL algorithms usually suffer from slow and unstable convergence and are vulnerable to poisoning attacks from malicious participants. In this work, we aim to design a versatile FL framework that simultaneously promotes the performance of the model both in a secure system and under attack. To this end, we leverage attention mechanisms as a defense against attacks in FL and propose a robust FL algorithm by integrating the attention mechanisms into the global model aggregation step. To be more specific, two attention models are combined to calculate the amount of attention cast on each participant. It will then be used to determine the weights of local models during the global aggregation. The proposed algorithm is verified on a real-world dataset and it outperforms existing algorithms, both in secure systems and in systems under data poisoning attacks.

Extended Reality (XR) services will revolutionize applications over $5^{\text {th }}$ and $\mathbf{6}^{\text {th }}$ generation wireless networks by providing seamless virtual and augmented reality experiences. These applications impose significant challenges on network infrastructure, which can be addressed by machine learning algorithms due to their adaptability. This paper presents a Multi-Agent Reinforcement Learning (MARL) solution for optimizing codec parameters of XR traffic, comparing it to the Adjust Packet Size (APS) algorithm. Our cooperative multi-agent system uses an Optimistic Mixture of Q-Values ($\mathbf{O Q M I X}$) approach for handling Cloud Gaming (CG), Augmented Reality (AR), and Virtual Reality (VR) traffic. Enhancements include an attention mechanism and slate-Markov Decision Process (MDP) for improved action selection. Simulations show our solution outperforms APS with average gains of $30.1 \%, 15.6 \%, 16.5 \% 50.3 \%$ in XR index, jitter, delay, and Packet Loss Ratio (PLR), respectively. APS tends to increase throughput but also packet losses, whereas oQMIX reduces PLR, delay, and jitter while maintaining goodput.

Proper and efficient beam selection is of great importance to unleash the full potential of mmWave communications. Traditionally, each candidate beam is evaluated using reference signals (beam sweeping), however, the exhaustive search method can be time-consuming with high signaling overhead. To avoid such problems, in B5G and 6G, sensing information is considered to be used, as in Integrated Sensing and Communication (ISAC) solutions, and Machine Learning (ML) methods can be applied to map sensing data inputs to an optimal beam index. When using sensing information sources external to the Radio Access Network (RAN) in a multi-vendor disaggregated environment, those methods need to account for issues such as privacy and data ownership. In this work, we apply multi-modal sensing information to the beam selection task. Specifically, we propose a multi-modal sensing-aided ML strategy based on Split Learning (SL) that can cope with deployment challenges in novel RAN architectures. Moreover, the method is applied to single and multi-level beam selection decisions, where the latter considers the case of hierarchical codebook structures. With the proposed approach, accuracy levels above 90% can be achieved while overhead diminishes by 85% or more. SL achieves comparable performance with centralized learning-based strategies, with the added value of accounting for privacy and data ownership issues. We also show that sensing-aided ML-based beam selection decisions in multi-level codebooks are more effective when applied to their first level.

As wireless networks continue to evolve, machine learning (ML) algorithms are used to address communication challenges and meet various service requirements. While ML methods are promising, they can be prone to malicious attacks, which may degrade user experience and network performance. Specifically, the security challenges of radio access networks (RANs) are highlighted due to frequent interactions with a large number of users, and evaluating these attacks is critical for securing wireless communications. In this paper, for the first time, we investigate the vulnerability of transfer reinforcement learning (TRL) algorithms for resource allocation in 5G RAN slicing. In particular, we first present the system model for RAN slicing, and then the TRL algorithm is introduced for resource allocation. Afterward, we investigate three types of attack methods on the TRL algorithm. The simulations indicate that the attack on an expert agent can affect the performance of a learner agent since the expert shares knowledge with the learner. We show that the effect of the black-box policy poisoning attack on the learner increases latency by 18.31% and reduces throughput by 13.80%, while white-box policy poisoning attacks result in a 48.96% increase in latency and an 87.02% reduction in throughput.

In 5G non-standalone mode, an intelligent traffic steering mechanism can vastly aid in ensuring a smooth user experience by selecting the best radio access technology (RAT) from a multi-RAT environment for a specific traffic flow. In this paper, we propose a novel load-aware traffic steering algorithm based on hierarchical reinforcement learning (HRL) while satisfying the diverse quality of service requirements of different traffic types. HRL can significantly increase system performance using a bi-level architecture having a meta-controller and a controller. In our proposed method, the meta-controller provides an appropriate threshold for load balancing, while the controller performs traffic admission to an appropriate RAT in the lower level. Simulation results show that HRL outperforms a Deep Q-Learning (DQN) and a threshold-based heuristic baseline with 8.49%, 12.52% higher average system throughput and 27.74%, 39.13% lower network delay, respectively.

This paper proposes a comprehensive study of uplink scheduling in multi-cell OFDMA networks. We first focus on two scenarios for the homogeneous case, one without and one with a Cloud-RAN (C-RAN), and explore how to design efficient practical uplink schedulers for those scenarios. To compute the best achievable performance (BAP) under complete information, we study a centralized multi-cell scheduler. To this end, we formulate an MINLP problem and show how to solve it quasi-optimally. Then, we study the performance of an existing practical local benchmark scheduler (LBM) in terms of goodput and losses. We compare its performance to BAP and show that LBM only yields 44 percent of BAP. To reduce this performance gap, we propose two practical enhancements for LBM, one per scenario. The enhanced scheduler for the first scenario yields 51 percent of BAP (70 percent for the second). To reduce the gap further, we propose a new scheduler inspired by soft-frequency reuse (SFR). Its performance is 69 percent (resp. 83 percent) of BAP. It outperforms LBM by 56 percent for the scenario without C-RAN (84 percent with C-RAN). We finally extend our SFR-based scheduler to heterogeneous networks and show that it outperforms LBM by 53 percent for the scenario without C-RAN (96 percent with C-RAN).

In this work, we propose a robust planning tool that allocates power statically in homogeneous and heterogeneous cellular networks with non-regular base station (BTS) placement, to mitigate interference and improve overall performance. Each BTS will use the total available spectrum, but it will divide it into multiple sub-bands, and each BTS will transmit with a specific pre-computed power on each sub-band. We refer to such a power allocation as a power map. Our offline planning tool computes a robust power map for a given topology, by solving a non-convex, non-linear optimization problem, through simple transformations, based on geometric programming. The power map is computed based solely on the network topology, and it is made available to all BTSs that use it throughout the network operation to perform scheduling using a fast quasi-optimal online algorithm that we propose. We evaluate our planning tool for different homogeneous and heterogeneous networks (HetNets), first in a static setting where scheduling is performed optimally and then in a dynamic setting when scheduling is performed with our online scheduler. Results show that our solution significantly outperforms a classical equal power/fixed frequency reuse scheme in terms of sum-rate, by up to 30% in homogeneous networks and by up to 70% in HetNets.

In cellular networks, a local uplink scheduler cannot compute good estimates of the inter-cell interference even with exact channel state information (CSI). Losses (due to decoding errors) will hence frequently occur. The maximum achievable performance (MAP) can only be achieved when all the cells are scheduled simultaneously centrally so as to manage interference and power optimally. In this paper, we focus on the performance of a practical system with local schedulers (i.e., each cell is scheduled independently) and compare it to MAP. We also study how to improve the performance of practical systems when a Cloud Radio Access Network (C-RAN) is present, with simple coordination schemes. To this end, to compute MAP, we first formulate an offline system-wide scheduling problem and transform it into a more tractable signomial problem that we solve quasi-optimally using an iterative algorithm. Since this problem requires all the channel information in the system and it has a high computational complexity, it is not suitable for a real-time system. Then, we show that a practical system using an efficient local scheduler (in each cell) yields a much lower performance than MAP. To decrease this performance gap, in a system with a C-RAN, we propose a very simple and fast scheme to coordinate the scheduling in all cells and show that it improves the performance significantly (in terms of throughput and losses) even when only partial CSI is available.

Uplink scheduling in cellular networks is challenging due to power and interference management. Typically, each cell performs local scheduling, which requires estimation of inter-cell interference (ICI) to compute the appropriate modulation and coding scheme (MCS) based on the Signal-to-Interference-plus-Noise-Ratio (SINR) for each allocated resource block. Since schedules of neighboring cells are unknown to schedulers, the SINR can be badly estimated, which causes resource losses or under-utilization. The benchmark uplink scheduler we study in this paper produces a high goodput at the cost of significant resource losses, because it does not take the possibility of losses into account. Resource losses imply retransmissions, hence, high variability in delay. Therefore, a scheduler should be evaluated in terms of its goodput/loss trade-off. We propose a novel uplink scheduler that is inspired by Soft Frequency Reuse (SFR) and uses an MCS selection that takes the probability of losses into account, i.e., it selects an MCS that maximizes the effective rates seen by users, while keeping the loss probability below a threshold e. We show that the proposed scheduler yields significantly better goodput/loss trade-off than the benchmark scheduler.

We introduce a new, simple, yet effective scheme for reducing the impact of receiver buffer blocking in Multipath TCP (MPTCP). This blocking primarily occurs when the paths have different characteristics. This phenomenon is due to the limited size of the MPTCP receiver buffer. In a nutshell, our scheme allows, under certain conditions, the retransmission of a segment by a different interface than the one used originally and the closure of the badly behaving TCP connection. Our scheme anticipates the problem by opening multiple TCP connections for each interface, but only uses one at a given time. It detects when a segment needs to be retransmitted from another interface, closes the ongoing connection on the original interface, and starts using one of the backup connections. We show through NS3 simulations that our scheme improves MPTCP goodput, with a gain of 19% compared to other MPTCP schemes, and provides a smooth data delivery to the application layer. This last feature is of utmost importance for streaming applications. Furthermore, while other MPTCP schemes fail to perform better than the best single path TCP for some scenarios, our proposed scheme always outperforms the best single path TCP.

Many new mobile applications create traffic among cellular users. We define intra-cellular traffic as the traffic from one cellular user to another user in the same cellular network. This type of traffic introduces new challenges for cellular network operators. Most work in the literature focuses on the possibility to utilize the direct links between those users, if they are close to each other (this is called device-to-device (D2D) communication), to by-pass the base station. However, implementing D2D is not easy, especially because detecting that a traffic is intracellular is difficult. In this paper, we assume that we know how to detect if a traffic is intra-cellular or not and focus on designing a type-aware scheduler (i.e., a scheduler which has the information on the type of traffic) in a case where direct communications between users is not enabled. This scheduler can be seen as the benchmark against the case where direct communications are allowed. We show that performance gain can be obtained by jointly scheduling the uplink and downlink with respect to the case where the scheduler is blind to the types. We show for a homogeneous network that when the traffic types are known to a scheduler, a significant performance gain (up to 28%) can be achieved compared to the case where the traffic types are not known. We also analyze heterogeneous networks that consist of macro cells and small cells and show that up to a 36% performance gain can be obtained by performing type-aware user association jointly with user scheduling.