As next generation cellular networks become denser, associating users with the optimal base stations at each time while ensuring no base station is overloaded becomes critical for achieving stable and high network performance. We propose multi-agent online Q-learning (QL) algorithms for performing real-time load balancing user association and handover in dense cellular networks. The load balancing constraints at all base stations couple the actions of user agents, and we propose two multi-agent action selection policies, one centralized and one distributed, to satisfy load balancing at every learning step. In the centralized policy, the actions of UEs are determined by a central load balancer (CLB) running an algorithm based on swapping the worst connection to maximize the total learning reward. In the distributed policy, each UE takes an action based on its local information by participating in a distributed matching game with the BSs to maximize the local reward. We then integrate these action selection policies into an online QL algorithm that adapts in real-time to network dynamics including channel variations and user mobility, using a reward function that considers a handover cost to reduce handover frequency. The proposed multi-agent QL algorithm features low-complexity and fast convergence, outperforming 3GPP max-SINR association. Both policies adapt well to network dynamics at various UE speed profiles from walking, running, to biking and suburban driving, illustrating their robustness and real-time adaptability.

We study the joint optimization of beamforming, RIS phase shift, and association for the links of BS-user and RIS-user communications in a multi-cell wireless network aided by multiple RISs. Consider a network setting with many RISs, we can optimize the reflection of each RIS for a single associated user, even though the RIS will reflect the signals of all users. We first design the optimal BS transmit beamforming together with the phase shift of RIS in a closed form to maximize the effective channel gain (ECG). Then, we design two different BS-RIS-user association algorithms satisfying the load balancing constraint. The first algorithm uses worst connection swapping on both the BS-user and RIS-user links, whereas the second algorithm uses a simpler max-ECG rule for the RIS-user link because of no load balancing at the RIS. A joint algorithm alternates between BS-RIS-user association and beamforming/reflection design until convergence. The proposed algorithms not only have substantially lower complexity than existing algorithms, but also outperforms the conventional max SINR association and effectively exploits multiple RISs to boost the network sum rate.

Using a multi-armed bandit technique, we propose centralized and semi-distributed online algorithms for load balancing user association and handover in mmWave-enabled networks. Load balancing at all base stations (BSs) imposes explicit constraints that makes the actions of all user equipment (UEs) co-dependent, a challenging twist to reinforcement learning. We propose a central load balancer to guarantee load balancing at all BSs for every learning step. We consider two association vectors: one for leaning update, and one best-to-date for data transmission, allowing UEs to engage in best-result data transmission while effectively participating in a background learning process indefinitely. For dynamic networks, we introduce a measurement model capturing rapid channel variations and user mobility. To minimize handover rate, we also differentiate between the handover cost for transmission and that for learning, and introduce a learning handover cost decreasing with sojourn time. The proposed algorithms can be implemented online as they require no offline training and can effectively adapt to network dynamics. Numerical results show that the proposed algorithms exhibit fast learning convergence and outperform 3GPP handover by achieving an order of magnitude lower handover rate at a significantly higher network sum-rate, reaching within 94-97% of the near-optimal worst connection swapping benchmark algorithm.

We consider a mobile network consisting of both the wireless access network and the backhaul network. All base stations in the access network and gateways in the backhaul network are equipped with caches, so that routing costs for serving content requests can be reduced by caching the requested content items closer to the users. In this case, user association in the wireless access network must be aware of both the quality of wireless channels and the content caching strategy. In this paper, we propose a framework that jointly optimizes wireless user association and content caching in both access and backhaul networks. The resulting problem is NP-hard. We propose a polynomial-time algorithm based on convex approximation and pipage rounding that produces a solution within a constant factor of 1 − 1/e from the optimal. Simulation results show that the proposed joint algorithm outperforms schemes that combine cache-independent user association methods with traditional caching strategies (e.g. LRU) in terms of minimizing the aggregate routing cost and backhaul traffic while achieving a high data sum rate in the access network.

In this paper, we introduce a new load balancing user association scheme for millimeter wave (mmWave) cellular networks in which intelligent reflecting surface (IRS) is applied in the cellular network to improve the coverage region of each cell and mitigate mmWave vulnerability to non-line of sight (N-LoS) paths. The user association scheme improves network performance significantly by adjusting the interference according to the association. We study the IRS-assisted mmWave cellular network where one IRS is deployed to assist in the communication from the base station (BS) to mobile users (MUs) in each cell. We balance BS loads and maximize a network utility by optimizing the user association with a matching game. Simulation results show that the proposed scheme significantly improves the throughput compared to conventional user association techniques.

We study distributed user association in 5G and beyond millimeter-wave enabled heterogeneous networks using matching theory. We propose a novel and efficient distributed matching game, calledearly acceptance(EA), which allows users to apply for association with their ranked-preference base station in a distributed fashion and get accepted as soon as they are in the base station’s preference list with available quota. Several variants of the EA matching game with preference list updating and reapplying are compared with the original and stability-optimal deferred acceptance (DA) matching game, which implements a waiting list at each base station and delays user association until the game finishes. We show that matching stability needs not lead to optimal performance in other metrics such as throughput. Analysis and simulations show that compared to DA, the proposed EA matching games achieve higher network throughput while exhibiting a significantly faster association process. Furthermore, the EA games either playing once or multiple times can reach closely the network utility of a centralized user association while having much lower complexity.

Using a reinforcement learning multi-armed bandit (MAB) technique, we design a centralized and a semi-distributed online algorithms, for performing load balancing user association in multi-tier heterogeneous cellular networks. The proposed algorithms guarantee user association solutions that satisfy load balancing constraints among the base stations (BSs) by employing a central load balancer (CLB). At each time step, these algorithms provide real-time associations which give the best-to-date network spectral efficiency. In the centralized approach, the CLB performs base station assignments which determine the action for each user equipment (UE) to update its reward. In the semi-distributed approach, each UE proposes an association action based on its local information and communicates with the BS for an associated reward. Numerical results show that the proposed MAB-based algorithms exhibit fast convergence and reach closely a near-optimal benchmark centralized solution.

We examine the use of matching theory for user association in millimeter wave (mmWave)-enabled cellular heterogeneous networks. In a mmWave system, the channel variations can be fast and unpredictable, rendering centralized user association potentially inefficient. We propose an efficient distributed matching algorithm, called early acceptance (EA), tailored for user association in 5G HetNets. The effectiveness of the proposed algorithm is assessed by comparing with the well-known deferred acceptance (DA) matching algorithm, in which user association is delayed until the algorithm terminates. Numerical results show that the proposed distributed EA matching algorithm reaches a near-optimal solution compared to a centralized algorithm, and leads to a more power-efficient and much faster user association process compared to the distributed DA algorithm.

User association is necessary in dense millimeter wave (mmWave) networks to determine which base station a user connects to in order to balance base station loads and maximize a network utility. Given that mmWave connections are highly directional and vulnerable to small channel variations, user association changes these connections and hence significantly affects the network interference and consequently the users' instantaneous rates. In this paper, we introduce a new load balancing user association scheme for mmWave MIMO cellular networks which consider these dependencies. We formulate the user association problem as mixed integer nonlinear programming and design a polynomial-time algorithm, called worst connection swapping (WCS), to find a near-optimal solution. Simulation results confirm that the proposed user association scheme improves network performance significantly by adjusting the interference according to the association, and under the max-min fairness, also enhances cell-edge users' transmission rates. We also show how the proposed algorithm can be applied under mobility. Furthermore, the proposed WCS algorithm outperforms other generic algorithms for combinatorial programming such as the genetic algorithm in both accuracy and speed at several orders of magnitude faster, and for small networks, where exhaustive search is possible, it reaches the optimal solution.

User association determines which base stations a user connects to, hence affecting the amount of network interference and consequently the network throughput. Conventional user association schemes, however, assume that user instantaneous rates are independent of user association. In this paper, we introduce a new load-aware user association scheme for millimeter wave (mmWave) MIMO networks which takes into account the dependency of network interference on user association. This consideration is well suited for mmWave communications, where the links are highly directional and vulnerable to small channel variations. We formulate our user association problem as a mixed integer nonlinear programming (MINLP) and solve it using the genetic algorithm. We show that the proposed method can improve network performance by moving the traffic of congested base stations to lightly-loaded base stations and adjusting the interference accordingly. Our simulations confirm that our scheme results in a higher network throughput compared to conventional user association techniques.