This paper studies beam bandwidth partitioning problem in mobile millimeter-wave (mmWave) and multiple antennas networks. The main novelty is to flexibly optimize the beamforming bandwidth with the aim to minimize the energy consumption of the system while guaranteeing the data requirements of all mobile users. We formulate the problem as an integer nonlinear programming problem. To efficiently solve the problem, we design a deep reinforcement learning using the proximal policy optimization approach and train a deep neural network in an on-policy manner. Then, for comparison purposes, we develop low-complexity online iterative accurate solutions. We show that our approach achieves better performance compared to the iterative solutions and is able to achieve at least 4% less energy consumption and more than 12% energy efficiency gains.

As 5G networks continue to be deployed and 6G networks begin to be envisioned, mobile network operators (MNOs) are embarking on a revolutionary transformation of the way they manage their networks. Various technology bricks are currently considered paramount in this transformation, including radio access network (RAN) slicing. The concept of an open radio access network (Open RAN) promises to provide more flexibility to support RAN slicing. However, RAN slicing in an O-RAN architecture raises a major challenge in achieving efficient resource sharing among slices, due to the diverse and permanent changes in RAN slices’ QoS requirements. To overcome this challenge in a RAN environment involving an MNO and multiple mobile virtual network operators (MVNOs), we propose a two-level RAN slicing mechanism. The first level is executed on a long time-scale to allocate radio resources from the MNO to MVNOs while the second level is executed on a shorter time-scale to allocate MVNO resources to users. This mechanism improves the performance of the RAN slicing operation by enabling users to obtain the required resources as quickly as possible and with a high level of granularity. We formulate the two-level problem as two mathematical optimization problems and we study their NP hardness. To efficiently solve the two-level problem, we first propose a game-theoretic solution to solve the first-level resource allocation problem using a matching algorithm. Next, we propose a deep reinforcement learning (DRL) algorithm that uses the double deep$Q$-network procedure to solve the second-level resource allocation problem. The two proposed algorithms are coupled such that the DRL algorithm uses the solution obtained using the game-theoretic matching algorithm. We show through extensive simulations that the proposed two-level solution outperforms the current state-of-the-art solutions and achieves efficient performance.

This paper considers the transmission scheduling problem in a single-node energy harvesting (EH) wireless communication system, where the monitoring application requires regular status updates. The objective is to minimize the number of inter-delivery violations events over a time horizon in a wireless sensor network consisting of an EH sensor node providing status updates to a non-EH sink. The offline scheduling problem is formulated as an integer linear program and is solved optimally in polynomial time using a dynamic programming approach. Next, an efficient and low complexity heuristic algorithm is proposed for the online setting. Simulation results show the effectiveness of our proposed algorithms compared to baseline methods.

This paper studies the problem of minimizing the age of information (AoI) in cellular vehicle-to-everything communications. To provide minimal AoI and high reliability for vehicles’ safety information, non-orthogonal multiple access is exploited. We reformulate a resource allocation problem that involves half-duplex transceiver selection, broadcast coverage optimization, power allocation, and resource block (RB) scheduling. First, to obtain the optimal solution, we formulate the problem as a mixed-integer nonlinear programming problem and then study its NP-hardness. The negative result of NP-hardness motivates us to design efficient sub-optimal solutions. Consequently, we model the problem as a single-agent Markov decision process (MDP). The MDP model helps in solving the problem efficiently using fingerprint deep reinforcement learning (DRL) techniques such as deep-Q-network (DQN) methods. Nevertheless, applying DQN is not straightforward due to the curse of dimensionality implied by the large and mixed action space that contains discrete RB scheduling decisions and continuous power and coverage optimization decisions. Therefore, to solve this mixed discrete/continuous problem efficiently simply and elegantly, we propose a decomposition technique that consists of first solving the discrete subproblem using a matching algorithm based on state-of-the-art stable roommate matching and then solving the continuous subproblem using DRL algorithm that is based on deep deterministic policy gradient (DDPG). We validate our proposed method through Monte Carlo simulations where we show that the decomposed matching and DRL algorithm successfully minimizes the AoI and achieves almost 66% performance gain compared to the best benchmarks for various vehicles’ speeds, transmission power, or packet sizes. Further, we prove the existence of an optimal value of broadcast coverage at which the learning algorithm provides the optimal AoI.

Federated Learning (FL) is expected to play a prominent role for privacy-preserving machine learning (ML) in autonomous vehicles. FL involves the collaborative training of a single ML model among edge devices on their distributed datasets while keeping data locally. While FL requires less communication compared to classical distributed learning, it remains hard to scale for large models. In vehicular networks, FL must be adapted to the limited communication resources, the mobility of the edge nodes, and the statistical heterogeneity of data distributions. Indeed, a judicious utilization of the communication resources alongside new perceptive learning-oriented methods are vital. To this end, we propose a new architecture for vehicular FL and corresponding learning and scheduling processes. The architecture utilizes vehicular-to-vehicular(V2V) resources to bypass the communication bottleneck where clusters of vehicles train models simultaneously and only the aggregate of each cluster is sent to the multi-access edge (MEC) server. The cluster formation is adapted for single and multi-task learning, and takes into account both communication and learning aspects. We show through simulations that the proposed process is capable of improving the learning accuracy in several non-independent and-identically-distributed (non-i.i.d) and unbalanced datasets distributions, under mobility constraints, in comparison to standard FL.

In this paper, we address the resource provisioning problem for service function chaining (SFC) in terms of the placement and chaining of virtual network functions (VNFs) within a multi-access edge computing (MEC) infrastructure to reduce service delay. We consider the VNFs as the main entities of the system and propose a mean-field game (MFG) framework to model their behavior for their placement and chaining. Then, to achieve the optimal resource provisioning policy without considering the system control parameters, we reduce the proposed MFG to a Markov decision process (MDP). In this way, we leverage reinforcement learning with an actor-critic approach for MEC nodes to learn complex placement and chaining policies. Simulation results show that our proposed approach outperforms benchmark state-of-the-art approaches.

This paper studies the problem of massive Internet of things (IoT) access in beyond fifth generation (B5G) networks using non-orthogonal multiple access (NOMA) technique. The problem involves massive IoT devices grouping and power allocation in order to respect the low latency as well as the limited operating energy of the IoT devices. The considered objective function, maximizing the number of successfully received IoT packets, is different from the classical sum-rate-related objective functions. The problem is first divided into multiple NOMA grouping subproblems. Then, using competitive analysis, an efficient online competitive algorithm (CA) is proposed to solve each subproblem. Next, to solve the power allocation problem, we propose a new reinforcement learning (RL) framework in which a RL agent learns to use the CA as a black box and combines the obtained solutions to each subproblem to determine the power allocation for each NOMA group. Our simulations results reveal that the proposed innovative RL framework outperforms deep-Q-learning methods and is close-to-optimal.

Multi-access edge computing (MEC) is a key enabler to reduce the latency of vehicular network. Due to the vehicles mobility, their requested services (e.g., infotainment services) should frequently be migrated across different MEC servers to guarantee their stringent quality of service requirements. In this paper, we study the problem of service migration in a MEC-enabled vehicular network in order to minimize the total service latency and migration cost. This problem is formulated as a nonlinear integer program and is linearized to help obtaining the optimal solution using off-the-shelf solvers. Then, to obtain an efficient solution, it is modeled as a multi-agent Markov decision process and solved by leveraging deep Q learning (DQL) algorithm. The proposed DQL scheme performs a proactive services migration while ensuring their continuity under high mobility constraints. Finally, simulations results show that the proposed DQL scheme achieves close-to-optimal performance.

Low-latency IoT applications, such as autonomous vehicles, augmented/virtual reality devices, and security applications, require high computation resources to make decisions on the fly. However, these kinds of applications cannot tolerate offloading their tasks to be processed on a cloud infrastructure due to the experienced latency. Therefore, edge computing (EC) is introduced to enable low latency by moving the tasks processing closer to the users at the edge of the network. The edge of the network is characterized by the heterogeneity of edge devices (EDs) forming it; thus, it is crucial to devise novel solutions that take into account the different physical resources of each ED. In this article, we propose a resource representation scheme, allowing each ED to expose its resource information to the supervisor of the edge node through the mobile EC application programming interfaces proposed by the European Telecommunications Standards Institute. The information about the ED resource is exposed to the supervisor of the edge node each time a resource allocation is required. To this end, we leverage a Lyapunov optimization framework to dynamically allocate resources at the EDs. To test our proposed model, we performed intensive theoretical and experimental simulations on a testbed to validate the proposed scheme and its impact on different system's parameters. The simulations have shown that our proposed approach outperforms other benchmark approaches and provides low latency and optimal resource consumption.

This article studies the problem of online user grouping, scheduling, and power allocation for massive Internet of Things (IoT) access in beyond 5G networks using nonorthogonal multiple access (NOMA). NOMA has been identified as a promising technology to accommodate a large number of devices using a limited number of radio resources. In this work, the objective is to maximize the number of served devices while allocating their transmission powers such that their real-time requirements as well as their limited operating energy are respected. First, we formulate the problem as a mixed-integer nonlinear program (MINLP) that can be transformed to MILP for some special cases. Second, we study its NP-hardness in different cases. Then, by dividing the problem into multiple NOMA grouping and scheduling subproblems, an efficient online competitive algorithm is proposed to solve each subproblem. Next, we show how to use the proposed online algorithm as a black box and how to combine the obtained solutions to each subproblem in a reinforcement learning setting to obtain the power allocation for each NOMA group. Our analyses are supplemented by simulation results to illustrate the performance of the proposed algorithms in comparison to optimal and state-of-the-art methods.

This paper studies the problem of minimizing the age of information (AoI) by optimally associating users to energy harvesting access points (EH-APs) and scheduling their packets that have stringent deadlines constraints. With a single EH-AP, this problem is already shown to be NP-hard. First, we consider the single EH-AP scenario and study the fairness between packets. We show the existence of fairness-AoI tradeoff. Further, we improve the previously proposed algorithms by reducing the average age of information. Finally, the general problem is considered. We reduce the problem to a knapsack problem and propose a dynamic programming approach to solve it. We present simulation results and show the efficiency of the proposed solutions compared to the optimal and the state-of-the-art ones.

This paper considers the problem of maximizing the number of scheduled users that request to download data with deadlines from an energy-harvesting base station. This problem is solved based on two frameworks: online computation and online learning. In the first framework, an optimal offline and a deterministic competitive algorithms are designed. In the online learning framework, the multi-armed bandit approach is used to design a learning algorithm based on the well-known exponential-weight algorithm for exploration and exploitation. We bound its regret and show that it grows sub-linearly with time. Finally, we supplement our theoretical results by simulations to illustrate the performance of the proposed algorithms.

This paper studies the problem of minimizing the age of information by scheduling packets with hard deadlines in wireless networks powered by energy harvesting base stations. The problem is shown to be NP-hard in its simplest form. We propose a general optimization framework to model the problem and further we provide an integer linear programming formulation. The integer program is useful to help solving the problem optimally using off-the-shelf solvers. To solve the problem in polynomial-time efficiently, we derive two greedy online (non-anticipative) algorithms-one being an improved version of the other. We present simulation results and show the efficiency of the proposed solutions compared to the optimal and the state-of-the-art ones.

The problem of user scheduling with an energy harvesting base station is considered. We study a wireless network where a set of users request to download data of certain sizes with hard deadline constraints from a base station powered exclusively by harvested energy. The objective is to maximize the number of scheduled users while respecting the deadline and energy constraints. To solve the problem, a polynomial-time algorithm is developed and proved to be optimal. In addition, when the users have common deadlines, a less complex and optimal algorithm is designed. Finally, we present simulation results to illustrate the impact of different parameters on the performance of the proposed algorithms.

As sub-6 GHz spectrum is becoming increasingly scarce, millimeter wave (mmWave) bands are considered as one of the key technologies for future cellular networks. Motivated by the rapid growth of the data rate demands and the number of wirelessly-connected devices, this paper considers a hybrid (sub-6 GHz and mmWave) heterogeneous network with a limited number of time-frequency resource blocks (RBs). To overcome the mmWave propagation problems and the need of frequent update of association due to mobility, a novel mobility-aware user-base station association strategy based on Markov chain is proposed. Simulation results validate the performance of the proposed strategy by reducing the need of frequent handovers between mmWave base stations in the network.

This paper studies the base station operation and association (BOA) problem for energy-efficient heterogeneous cellular networks. The objective is to find the set of base stations (BSs) to activate and to associate users to BSs under minimum rate requirements. BOA is formulated as a nonconvex programming problem. In order to solve it, we distinguish between two cases: 1) BOA with high-rate requirements (BOAH) and 2) BOA with low-rate requirements (BOAL). First, we show that finding feasible solutions for BOAH is NP-hard, and second, we reduce it to a BS operation problem (i.e., user association becomes straightforward). Based on this reduction, we develop a brute force algorithm and show that its complexity can be extremely reduced though it is still exponential. Hence, we propose a polynomial-time heuristic algorithm. As for BOAL, since BOA is extremely coupled, we relax the problem. Consequently, BOAL can be formulated as an integer linear program. Finding feasible solutions to it is shown to be NP-hard. To efficiently solve it, we propose a greedy-based algorithm. The proposed greedy algorithm is shown to admit a logarithmic approximation factor when it finds feasible solutions and a constant approximation factor otherwise. Finally, simulation results illustrate the performance of the proposed algorithms.

This paper studies the base station operation and association problem for energy-efficient heterogeneous cellular networks. The objective is to find a set of base stations to activate and to associate users to base stations under minimum rate requirements. The problem is formulated as a nonconvex program and is shown to be NP-hard. Then, the problem is reduced to a base station operation problem where the association of users becomes straightforward. Based on this reduction, a brute force algorithm is developed and its complexity is discussed. To solve the problem efficiently in polynomial-time, an heuristic algorithm is proposed. Simulation results illustrate the performance of the proposed algorithm and compares it to the brute force algorithm and a benchmark algorithm.

We consider the problem of associating users to base stations in heterogeneous cellular networks (HetNets). Given a set of users, base stations (BSs) and time-slots, the considered problem, called multi-slot user-BS association (MUBA), is to maximize the number of associated users to the BSs during the time-slots such that the signal to interference-plus-noise ratios (SINRs) of the users in each slot are above a certain threshold. First, we formulate MUBA as a 0-1 nonlinear optimization problem and then we transform it into a linear one. Next, MUBA is reduced to a link scheduling problem. Based on this reduction, an approximation algorithm for MUBA is designed and is shown to admit a constant approximation factor. Simulation results support our theoretical analysis and show that the proposed approximation algorithm gives tight-to-optimal performance.

Combining millimeter wave (mmWave) with sub-6 GHz communications is a promising solution for future heterogeneous cellular networks (HetNets) to improve coverage and capacity. This paper studies the user-base station association problem in HetNets with the existence of both sub-6 GHz and mmWave base stations (BSs) where each BS has a limited number of resource blocks (RBs). Motivated by the observation that traditional user-BS association methods may not be effective in such hybrid HetNet, an optimization problem is formulated in order to maximize the number of associated users and to ensure an efficient resource utilization by minimizing simultaneously the number of used RBs. Since the formulated problem is proved to be NP-hard, a heuristic algorithm is proposed. Simulation results show that the proposed algorithm approaches the optimal one with a significant reduction in computational complexity.

In this paper, the user association problem under quality of service (QoS) requirements in a heterogeneous and small cells network (HetSNet) is considered. We have shown in a previous work that this problem is NP-hard and thus cannot be solved optimally in polynomial time unless P = NP. Therefore, new suboptimal algorithms are needed in order to solve it efficiently. Even though, it is very hard to implement the suboptimal algorithm in a centralized fashion because it needs a high amount of information exchange between the base stations and the users and it suffers from a huge computational complexity. Thus, in this paper, we model the problem of user association in HetSNets as a non-cooperative game and we propose a completely distributed algorithm inspired by the theory of learning to solve it. Specifically, we propose a modified win-stay-lose-shift learning model in order to converge to a near optimal user association. We evaluate by simulations the performance of the proposed algorithm and and we show that it is close to the performance of the computationally complex optimal centralized algorithm which assumes complete channel information knowledge.

In this paper we consider the problem of user assignment and power allocation in a small cell environment which is one of the most important problems in present wireless cellular network research. We consider a two-tier cellular network where randomly dispersed overlay femtocell base stations (FBSs) coexist with a macrocell. Our objective is to maximize the total number of users served by the FBSs while satisfying their signal to noise and interference (SINR) requirements. This problem is known to be NP-Hard and hence there is no known optimal solution to solve it in polynomial time. First we formulate the problem of maximization of allocated users under SINR constraints with constant transmit power as an integer programming problem. We provide two heuristic polynomial time algorithms. Then we propose a third algorithm for joint power and user allocation. We evaluate the complexity of the proposed algorithms and furthermore compare the results against the brute force optimal solution and a basic random user assignment through simulations. The results demonstrate the performance and the efficiency of the proposed algorithms. We see in the simulation that the best proposed heuristic for maximizing the number of assigned users is only 3% less than the optimal while reducing the power consumption below that of the optimal user assignment algorithm.

In this paper, we investigate several relaying schemes for cooperative communications in Cognitive Radio Networks (CRNs) in order to improve the performances of secondary transmissions while respecting a certain Quality of Service (QoS) requirement at the primary transmissions. We propose relaying schemes where a number of relay nodes, randomly located, may help either the primary or the secondary transmission. By defining proper relay selection criteria and power allocation schemes, we illustrate the secondary outage probability performance while guaranteeing the primary QoS. Using simulations, we present the impact of different parameters, such as the QoS requirement, the chosen relay selection criteria, the number of available relays, the positions of the relays, etc., on the secondary transmission performance. The obtained results show the potential of the proposed relaying schemes, and provide guidelines about the expected secondary performance under the impact of several parameters.