Cognitive radio network with proactive resource allocation to identify unused spectrum bands and utilize them opportunistically is observed as an evolving technology to handle spectrum scarcity problem. However, it is a challenging problem to predict the accurate information about availability of unused resources due to randomness in licenced user appearance and high mobility at the cost of minimizing sensing time. To address this issue, we mathematically model the metrics like resource availability probability, resource allocation time, throughput, connection continuance probability, and the expected number of networks switching to propose a proactive optimal resource allocation (PORA) technique. Furthermore, the performance of proposed PORA technique is analysed under different traffic environments such as low, moderate, and high traffic. The results show that the proposed PORA technique addresses challenges related to providing resource allocation in proactive manner over the traditional techniques.

The primary obstacle for the wireless industry is meeting the growing demand for cellular services, which necessitates the deployment of numerous femto base stations (FBSs) in ultra-dense networks. Effective resource distribution among densely and randomly distributed FBSs in ultra-dense is difficult, mainly because of intensified interference problems. The K-means clustering is improved by employing the Davies Bouldin index, which separates the clusters to prevent overlapping and mitigate interference. The elbow approach is utilized to determine the optimal number of clusters. Afterward, attention is directed toward addressing efficient resource allocation through a distributive methodology. The proposed approach makes use of a replay buffer-based multi-agent framework and uses the generative adversarial networks deep distributional Q-network (GAN-DDQN) to efficiently model and learn state-action value distributions for intelligent resource allocation. To further improve control over the training error, the distributions are estimated by approximating a whole quantile function. The numerical results validate the effectiveness of both the proposed clustering method and the GAN-DDQN-based resource allocation scheme in optimizing throughput, fairness, energy efficiency, and spectrum efficiency, all while maintaining the QoS for all users.

It is certain that in the modern era the ultra-dense network (UDN) structure will play a major role for the evolution of 5G and beyond wireless communication system, particularly for blind wireless area and hotspot. In resource constraint environment, obtaining higher energy efficiency (EE), spectrum efficiency (SE), and greater fairness during resource allocation process are conflicting objectives. To obtain the balance among them a multi-objective optimization problem (MOOP) is designed and an enhanced version of non-dominated sorting genetic algorithm II (NSGA-II), which integrates the advantage of evolutionary method and machine learning framework is suggested. Firstly, the chromosome coding scheme is designed which is suitable for spectrum allocation. Afterward, a deep learning framework is designed to enable the self-tuning of crossover and mutation operators to improve the diversity of candidate solutions. Further, an elitist retention strategy is modified by designing variable fraction scheme. This intelligent approach enables micro-cell users to improve their downlink performance of SE, EE, and fairness by assigning resource blocks. The simulated results yield the effectiveness of the proposed scheme in perfect and imperfect channel state information (CSI) environment by analysing the obtained performance gains when compared with other existing allocation methods in terms of EE, SE, and fairness.

Driven by the vision of 6G, the need for diverse computation-intensive and delay-sensitive tasks continues to rise. The integration of mobile edge computing with the ultra-dense network is not only capable of handling traffic from a large number of smart devices but also delivers substantial processing capabilities to the users. This combined network is expected as an effective solution for meeting the latency-critical requirement and will enhance the quality of user experience. Nevertheless, when a massive number of devices offload tasks to edge servers, the problem of channel interference, network load and energy shortage of user devices (UDs) would increase. Therefore, we investigate the joint uplink and downlink resource allocation and task offloading optimization problem in terms of minimizing the overall task delay while sustaining the UD battery life. Thus, to achieve long-term gains while making quick decisions, we propose an improved double deep Q-network scheme named Prioritized double deep Q-network. In this, the prioritized experience replay has been improved by considering the experience freshness factor along with temporal difference error to achieve fast and efficient learning. Extensive numerical results prove the efficacy of the proposed scheme by analyzing delay and energy consumption. Especially, our scheme can considerably decrease the delay by 11.86%, 26.22%, 48.56%, and 61.04% compared to the OELO scheme, DQN scheme, LOS, and EOS, respectively, when the number of UDs varied from 30 to 180.

The ultra-dense network structure of 5G with dense femto-cells deployment, is identified as a prospective way out for the problem of growing demand for cellular services. However, optimum resource allocation among dense and random deployed femto-cell base stations (FBSs) is a challenging task since there are severe interferences. To overcome these interferences and to obtain efficient resource allocation, a two-stage cluster-based resource allocation scheme has been proposed. In the first step, an efficient dynamic clustering algorithm based on unsupervised learning is proposed which group the FBSs in an optimal number of clusters while balancing the traffic load in each cluster. Afterward, we focus on the green resource allocation problem by using a cooperative methodology. A cloud-based multi-agent reinforcement learning algorithm has been proposed in a decentralized way. The cloud server provides fast-flexible access, large storage space, and turns down operational expenditure. Further, the reinforcement model exploits a brief representation of Q-value to resolve the problem of large state/action space, diminish the computational complexity and hasten the convergence. The proposed scheme is evaluated in real time experimental setup. The experimental results and the numerical results verify that the proposed scheme significantly improves energy efficiency with a QoS guarantee.

Due to exponential growth in demand for radio spectrum for wireless communication networking, the radio spectrum has become over-crowded. The fixed spectrum allocation policy of the radio spectrum leads to inefficient utilisation of the available spectrum, which diverted the attention of researchers towards different intelligent techniques to access the spectrum dynamically and efficiently. The concept of Cognitive Radio (CR) has been considered as a promising technology to solve the problem of spectrum scarcity through the utilisation of various unutilised spectrum bands. In a future network deployment, multiple radio access networks may coexist having different characteristics. Hence, it becomes a challenge for CR networks to select the optimal network out of available networks. For efficient realisation, CRs requires intelligent spectrum management techniques for Dynamic Spectrum Management (DSM). Till now, there does not exist a literature survey that addresses the spectrum management with machine learning techniques in an intelligent manner. Hence, this paper presents the detailed classification and comprehensive survey of various machine learning techniques for intelligent spectrum management with their paradigms of optimisation for cognitive radio networks. The paper also provides new directions and open issues for the research community to work further in CR networks.

The rapid development of wireless traffic pushed the wireless community to research different solutions towards the efficient utilization of the available radio spectrum. However, a recent study shows that most of the dynamically allocated spectrum bands (radio frequency resources), experience significant underutilization as cognitive radio (CR) technology still lacks intelligence. An intelligence in CRs can be incorporated with machine learning algorithms. Further, the perfect channel state information (CSI) is hardly obtained and CSI imperfections play a crucial role in Dynamic Spectrum Management. Thus, for efficient utilization of available spectrum, a decentralized Multi-Agent Reinforcement Learning based resource allocation scheme has been proposed. A robust resource allocation scheme is proposed which integrates machine learning and CR technology into a sophisticated multi-agent system (MAS). Moreover, assisted with cloud computing which provides a huge amount of storage space, reduces operating expenditures, and provides wider flexibility of cooperation. Hence, to foster the performance of the proposed scheme, a cooperative framework in MAS is introduced which enhances the performance of the proposed scheme in terms of network capacity, outage probability, and convergence speed. Numerical results verify the effectiveness of the proposed scheme and show the non-negligible impact of imperfect CSI, thus highlighting the importance of robust designs that maintains users’ QoS in practical wireless networks.

The upcoming 5G and beyond wireless networking technologies are intended to coexist with different communication networks to solve the spectrum scarcity problem. The seamless roaming across integrated wireless networks and an optimal network’s resource allocation would be a doable challenge for designing futuristic networks. The cognitive radio (CR) networks, with the inclusion of game theory, can handle spectrum scarcity issues by permitting spectrum access to the secondary user (SU) in the absence of a primary user (PU). This work proposes an AFSOS (an Auction Framework and Stackelberg game oriented Optimal network’s resource Selection) technique to achieve a maximum payoff and acceptable interference power restraint in CR networks. The contest among candidate networks can be modeled as a Stackelberg game, and the contest between CR nodes and candidate networks can be modeled as an auction framework. As a schema for the realization and capabilities evaluations, the CR networks consisting of LTE cellular network interworking with Wi-Fi network for multiple access points (APs)/single vehicle (low traffic environment) is analyzed. This framework is further analyzed for single AP/multiple CR nodes (high traffic environment). The equilibrium for the Stackelberg game is achieved, and the proposed AFSOS technique is analyzed with numerical examples. The results show that the proposed AFSOS technique is more operative to obtain maximum payoff.

The explosive growth of heterogeneous mobile data traffic with stringent Quality of Service (QoS) requirements put significant pressure on the existing network infrastructure and also present an important challenge even to very anticipated Cognitive Radio (CR) networks. In this perspective, to meet the demand of upcoming QoS requirements tied to their distinctive needs for resources, an intelligent solution is required which offers higher flexibility and to handle both current and future upcoming QoS challenges. The conventional resource allocation strategies face significant problems in meeting the increasing demand for bandwidth-hungry services due to stochastic behavior of wireless networks. To solve this problem, an efficient resource allocation scheme is proposed which maximizes energy efficiency while maintaining QoS requirements for all users. The Reinforcement learning based Q-Learning (Q-L) scheme is found most suitable for green resource allocation based on current network conditions that manages the QoS provisioning for heterogeneous traffic even under dynamic environmental conditions. Further, to enhance the performance of such sophisticated scheme, cooperative framework is introduced to improve the convergence speed. Experimental and simulated results demonstrate the effectiveness of proposed cooperative scheme. The proposed scheme outperforms current benchmark schemes in terms of meeting the energy-efficiency and stringent heterogeneous QoS requirements.

The most prominent challenge to the wireless community is to meet the demand for radio resources. Cognitive Radio (CR) is envisioned as a potential solution that utilizes its cognition ability intended to enhance the proper utilization of available radio resources and improves energy efficiency. However, due to the co-existence of Primary Base Stations (PU-BSs) and Cognitive Base Stations (CR-BSs) in CR networks, the problem of aggregated interference occurs which poses a critical challenge for resource allocation in CR networks. Moreover, in practical scenarios, it is difficult to form the correct network model due to complex network dynamics beforehand. Therefore, this work presents Multi-Agent Model-Free Reinforcement Learning schemes namely Q-Learning (Q-L) and State-Action-Reward- (next) State- (next) Action (SARSA) for resource allocation which mitigates interference and eliminate the need of network model. The proposed schemes are implemented in a decentralized cooperative manner with CRs act as multi-agent, forms a stochastic dynamic team to obtain optimal energy-efficient resource allocation strategy. Numerical results reveal that: 1) proposed cooperative scheme 1 (Cooperative Q-L scheme) expedites the convergence; 2) proposed cooperative scheme 2 (Cooperative SARSA scheme) achieves significant improvement in network capacity. Both the proposed cooperative schemes demonstrate its effectiveness by providing significant improvement in energy efficiency and maintain users' QoS.

When a mobile network changes its point of attachments in Cognitive Radio (CR) vehicular networks, the Mobile Router (MR) requires spectrum handoff. Network Mobility (NEMO) in CR vehicular networks is concerned with the management of this movement. In future NEMO based CR vehicular networks deployment, multiple radio access networks may coexist in the overlapping areas having different characteristics in terms of multiple attributes. The CR vehicular node may have the capability to make call for two or more types of nonsafety services such as voice, video, and best effort simultaneously. Hence, it becomes difficult for MR to select optimal network for the spectrum handoff. This can be done by performing spectrum handoff using Multiple Attributes Decision Making (MADM) methods which is the objective of the paper. The MADM methods such as grey relational analysis and cost based methods are used. The application of MADM methods provides wider and optimum choice among the available networks with quality of service. Numerical results reveal that the proposed scheme is effective for spectrum handoff decision for optimal network selection with reduced complexity in NEMO based CR vehicular networks.