Controlling antenna tilts in cellular networks is critical to achieve a good trade-off between network coverage and capacity. We devise algorithms learning optimal tilt control policies from existing data (passive learning setting) or from data actively generated by the algorithms (active learning setting). We formalize the design of such algorithms as a Best Policy Identification problem in Contextual Linear Bandits (CLB). In CLB, an action represents an antenna tilt update; the context captures current network conditions; the reward corresponds to an improvement of performance, mixing coverage and capacity. The objective is to identify an approximately optimal policy (a function mapping the context to an action with maximal reward). For both active and passive learning, we derive information-theoretical lower bounds on the number of samples required by any algorithm returning an approximately optimal policy with a given level of certainty, and devise algorithms achieving these fundamental limits. We apply our algorithms to the Remote Electrical Tilt optimization problem in cellular networks, and show that they can produce optimal tilt update policy using much fewer data samples than naive or existing rule-based learning algorithms. This paper is an extension of work presented at IEEE International Conference on Computer Communications (INFOCOM) 2022 (Vannella et al. 2022).

We investigate the problem of best arm identification in Multi-Agent Multi-Armed Bandits (MAMABs) where the rewards are defined through a factor graph. The objective is to find an optimal global action with a prescribed level of confidence and minimal sample complexity. We derive a tight instance-specific lower bound of the sample complexity and characterize the corresponding optimal sampling strategy. Unfortunately, this bound is obtained by solving a combinatorial optimization problem with a number of variables and constraints exponentially growing with the number of agents. We leverage Mean Field (MF) techniques to obtain, in a computationally efficient manner, an approximation of the lower bound. The approximation scales at most as $\rho K^d$ (where $\rho$, $K$, and $d$ denote the number of factors in the graph, the number of possible actions per agent, and the maximal degree of the factor graph). We devise MF-TaS (Mean-Field-Track-and-Stop), an algorithm whose sample complexity provably matches our approximated lower bound. We illustrate the performance of MF-TaS numerically using both synthetic and real-world experiments (e.g., to solve the antenna tilt optimization problem in radio communication networks).

We study the problem of regret minimization in Multi-Agent Multi-Armed Bandits (MAMABs) where the rewards are defined through a factor graph. We derive an instance-specific regret lower bound and characterize the minimal expected number of times each global action should be explored. Unfortunately, this bound and the corresponding optimal exploration process are obtained by solving a combinatorial optimization problem with a set of variables and constraints exponentially growing with the number of agents. We approximate the regret lower bound problem via Mean Field techniques to reduce the number of variables and constraints. By tuning the latter, we explore the trade-off between achievable regret and complexity. We devise Efficient Sampling for MAMAB (ESM), an algorithm whose regret asymptotically matches the corresponding approximated lower bound. We assess the regret and computational complexity of ESM numerically, using both synthetic and real-world experiments in radio communications networks.

Future generations of mobile networks are expected to contain more and more antennas with growing complexity and more parameters. Optimizing these parameters is necessary for ensuring the good performance of the network. The scale of mobile networks makes it challenging to optimize antenna parameters using manual intervention or hand-engineered strategies. Reinforcement learning is a promising technique to address this challenge but existing methods often use local optimizations to scale to large network deployments. We propose a new multi-agent reinforcement learning algorithm to optimize mobile network configurations globally. By using a value decomposition approach, our algorithm can be trained from a global reward function instead of relying on an ad-hoc decomposition of the network performance across the different cells. The algorithm uses a graph neural network architecture which generalizes to different network topologies and learns coordination behaviors. We empirically demonstrate the performance of the algorithm on an antenna tilt tuning problem and a joint tilt and power control problem in a simulated environment.

The study of Boolean functions with low $c$-differential uniformity has become recently an important topic of research. However, in odd characteristic case, there are not many results on the ($c$-)differential uniformity of functions that are not power functions. In this paper, we investigate the $c$-differential uniformity of the swapped inverse functions in odd characteristic, and show that their $c$-differential uniformities are at most 6 except for some special case.

Controlling antenna tilts in cellular networks is imperative to reach an efficient trade-off between network coverage and capacity. In this paper, we devise algorithms learning optimal tilt control policies from existing data (in the so-called passive learning setting) or from data actively generated by the algorithms (the active learning setting). We formalize the design of such algorithms as a Best Policy Identification (BPI) problem in Contextual Linear Multi-Arm Bandits (CL-MAB). An arm represents an antenna tilt update; the context captures current network conditions; the reward corresponds to an improvement of performance, mixing coverage and capacity; and the objective is to identify, with a given level of confidence, an approximately optimal policy (a function mapping the context to an arm with maximal reward). For CL-MAB in both active and passive learning settings, we derive information-theoretical lower bounds on the number of samples required by any algorithm returning an approximately optimal policy with a given level of certainty, and devise algorithms achieving these fundamental limits. We apply our algorithms to the Remote Electrical Tilt (RET) optimization problem in cellular networks, and show that they can produce optimal tilt update policy using much fewer data samples than naive or existing rule-based learning algorithms.

A multi-agent deep reinforcement learning (MADRL) is a promising approach to challenging problems in wireless environments involving multiple decision-makers (or actors) with high-dimensional continuous action space. In this paper, we present a MADRL-based approach that can jointly optimize precoders to achieve the outer-boundary, called pareto-boundary, of the achievable rate region for a multiple-input single-output (MISO) interference channel (IFC). In order to address two main challenges, namely, multiple actors (or agents) with partial observability and multi-dimensional continuous action space in MISO IFC setup, we adopt a multi-agent deep deterministic policy gradient (MA-DDPG) framework in which decentralized actors with partial observability can learn a multi-dimensional continuous policy in a centralized manner with the aid of shared critic with global information. Meanwhile, we will also address a phase ambiguity issue with the conventional complex baseband representation of signals widely used in radio communications. In order to mitigate the impact of phase ambiguity on training performance, we propose a training method, called phase ambiguity elimination (PAE), that leads to faster learning and better performance of MA-DDPG in wireless communication systems. The simulation results exhibit that MA-DDPG is capable of learning a near-optimal precoding strategy in a MISO IFC environment.

In this paper, we propose a novel deep-learning based precoding framework, called deep MIMO autoprecoder, that can be used to learn an optimal precoding policy for multiple-input multiple-output (MIMO) precoding problems, for which optimal solutions are not known. We divide the proposed framework into two steps: action value estimation and optimal action prediction. In the first step, reinforcement learning (RL) is applied to learn and estimate the state-action value function (also called Q-value function) of each discrete precoder in a codebook through interactions with environments of MIMO systems, while for the second step, a supervised learning model is used to predict an optimal precoder in a continuous precoder space that corresponds to the output from the first step. The proposed framework can be implemented by means of two deep neural networks: a deep RL algorithm, called deep Q-network (DQN), and a new neural network, that we call deep Q-value-to-policy network (DQPN). More specifically, an environment-specific DQN is trained for a MIMO scenario in order to estimate the Q-values of each precoder in environmental states. Meanwhile, a single universal DQPN is pretrained offline on the most general MIMO channel model that allows it to achieve generalization over unseen channels. The numerical results verify the effectiveness of the deep MIMO autoprecoder and show that the proposed precoding framework can outperform the conventional approximation algorithm.

We address the problem of Remote Electrical Tilt (RET) optimization using off-policy Contextual Multi-Armed-Bandit (CMAB) techniques. The goal in RET optimization is to control the orientation of the vertical tilt angle of the antenna to optimize Key Performance Indicators (KPIs) representing the Quality of Service (QoS) perceived by the users in cellular networks. Learning an improved tilt update policy is hard. On the one hand, coming up with a new policy in an online manner in a real network requires exploring tilt updates that have never been used before, and is operationally too risky. On the other hand, devising this policy via simulations suffers from the simulation-to-reality gap. In this paper, we circumvent these issues by learning an improved policy in an offline manner using existing data collected on real networks. We formulate the problem of devising such a policy using the off-policy CMAB framework. We propose CMAB learning algorithms to extract optimal tilt update policies from the data. We train and evaluate these policies on real-world 4G Long Term Evolution (LTE) cellular network data. Our policies show consistent improvements over the rule-based logging policy used to collect the data.

Predicting the future location of users in wireless networks has numerous applications, and can help service providers to improve the quality of service perceived by their clients. The location predictors proposed so far estimate the next location of a specific user by inspecting the past individual trajectories of this user. As a consequence, when the training data collected for a given user is limited, the resulting prediction is inaccurate. In this paper, we develop cluster-aided predictors that exploit past trajectories collected from all users to predict the next location of a given user. These predictors rely on clustering techniques and extract from the training data similarities among the mobility patterns of the various users to improve the prediction accuracy. Specifically, we present CAMP (Cluster-Aided Mobility Predictor), a cluster-aided predictor whose design is based on recent non-parametric Bayesian statistical tools. CAMP is robust and adaptive in the sense that it exploits similarities in users' mobility only if such similarities are really present in the training data. We analytically prove the consistency of the predictions provided by CAMP, and investigate its performance using two large-scale datasets. CAMP significantly outperforms existing predictors, and in particular those that only exploit individual past trajectories.

Mobility predictions are becoming more valuable in various applications with the rise of mobile devices. Given that existing prediction techniques are composed of two key procedures: 1) profiling past mobility trajectories as sequences of discrete atomic states (e.g., grid locations, semantic locations) and capturing them with an appropriate statistical model, 2) making a prediction on the next state using the statistical model, TravelMiner tackles the former with paths utilized as the atomic states for the first time, where the paths are defined as sub-trajectories with no branches. Comparing to available location-based predictors, TravelMiner makes a fundamental difference in that it is able to predict the sequence of paths rather than locations, which is far more detailed in the perspective of knowing the exact route to follow. TravelMiner enables this benefit by extracting disjoint paths from GPS trajectories via a similarity metric for curves, called Frechet distance and keeping the sequences of such paths in a statistical model, called probabilistic radix tree. Our extensive simulations over the GPS trajectories of 124 users reveal that TravelMiner outperforms other predictors in diverse popular performance metrics including predictability, prediction accuracy and prediction resolution.

An essential condition precedent to the success of mobile applications based on Wi-Fi (e.g., iCloud) is an energy-efficient Wi-Fi sensing. Clearly, a good Wi-Fi sensing policy should factor in both inter-access point (AP) arrival time (IAT) and contact duration time (CDT) distributions of each individual. However, prior work focuses on limited cases of those two distributions (e.g., exponential) or proposes heuristic approaches such as Additive Increase (AI). In this paper, we first formulate a generalized functional optimization problem on Wi-Fi sensing under general inter-AP and contact duration distributions and investigate how each individual should sense Wi-Fi APs to strike a good balance between energy efficiency and performance, which is in turn intricately linked with users mobility patterns. We then derive a generic optimal condition that sheds insights into the aging property, underpinning energy-aware Wi-Fi sensing polices. In harnessing our analytical findings and the implications thereof, we develop a new sensing algorithm, called Wi-Fi Sensing with AGing (WiSAG), and demonstrate that WiSAG outperforms the existing sensing algorithms up to 37% through extensive trace-driven simulations for which real mobility traces gathered from hundreds of smartphones is used.

In this paper, a joint optimization of link scheduling, routing and replication for delay-tolerant networks (DTNs) has been studied. The optimization problems for resource allocation in DTNs are typically solved using dynamic programming which requires knowledge of future events such as meeting schedules and durations. This paper defines a new notion of approximation to the optimality for DTNs, called snapshot approximation where nodes are not clairvoyant, i.e., not looking ahead into future events, and thus decisions are made using only contemporarily available knowledges. Unfortunately, the snapshot approximation still requires solving an NP-hard problem of maximum weighted independent set (MWIS) and a global knowledge of who currently owns a copy and what their delivery probabilities are. This paper proposes an algorithm, Max-Contribution (MC) that approximates MWIS problem with a greedy method and its distributed online approximation algorithm, Distributed Max-Contribution (DMC) that performs scheduling, routing and replication based only on locally and contemporarily available information. Through extensive simulations based on real GPS traces tracking over 4,000 taxies and 500 taxies for about 30 days and 25 days in two different large cities, DMC is verified to perform closely to MC and outperform existing heuristically engineered resource allocation algorithms for DTNs.

An essential condition precedent to the success of mobile applications based on Wi-Fi (e.g., iCloud) is an energy-efficient Wi-Fi sensing. From a user's perspective, a good WiFi sensing policy should depend on both inter-AP arrival and contact duration time distributions. Prior work focuses on limited cases of those two distributions (e.g., exponential) or introduces heuristic approaches such as AI (Additive Increase). In this paper, we formulate a functional optimization problem on Wi-Fi sensing under general inter-AP and contact duration distributions, and propose how each user should sense Wi-Fi APs to strike a balance between energy efficiency and performance, depending on the users' mobility pattern. To that end, we derive an optimal condition which sheds insights into the aging property, the key feature required by efficient Wi-Fi sensing polices. Guided by the analytical studies and the implications, we develop a new sensing algorithm, called WiSAG (Wi-Fi Sensing with AGing), which is demonstrated to outperform the existing sensing algorithms up to 34% through extensive trace-driven simulations using the real mobility traces gathered from smartphones.

This is by far the first paper considering joint optimization of link scheduling, routing and replication for disruption-tolerant networks (DTNs). The optimization problems for resource allocation in DTNs are typically solved using dynamic programming which requires knowledge of future events such as meeting schedules and durations. This paper defines a new notion of optimality for DTNs, called snapshot optimality where nodes are not clairvoyant, i.e., cannot look ahead into future events, and thus decisions are made using only contemporarily available knowledge. Unfortunately, the optimal solution for snapshot optimality still requires solving an NP-hard problem of maximum weight independent set and a global knowledge of who currently owns a copy and what their delivery probabilities are. This paper presents a new efficient approximation algorithm, called Distributed Max-Contribution (DMC) that performs greedy scheduling, routing and replication based only on locally and contemporarily available information. Through a simulation study based on real GPS traces tracking over 4000 taxies for about 30 days in a large city, DMC outperforms existing heuristically engineered resource allocation algorithms for DTNs.