Due to their adaptability and mobility, Unmanned Aerial Vehicles (UAVs) are becoming increasingly essential for wireless network services, particularly for data harvesting tasks. In this context, Artificial Intelligence (AI)-based approaches have gained significant attention for addressing UAV path planning tasks in large and complex environments, bridging the gap with real-world deployments. However, many existing algorithms suffer from limited training diversity and limited generalization beyond the training distribution, which hampers their performance in highly dynamic environments. Moreover, they often overlook the inherently multi-objective nature of the task, treating it in an overly simplistic manner. To address these limitations, we propose an attention-based Multi-Objective Reinforcement Learning (MORL) architecture that explicitly handles the trade-off between data collection and energy consumption in urban environments, even without prior knowledge of wireless channel conditions. Our method learns a single model capable of adapting to varying trade-off preferences and dynamic scenario parameters without the need for fine-tuning or retraining. Extensive simulations show that our approach achieves substantial improvements in performance, model compactness, sample efficiency, and most importantly, generalization to previously unseen scenarios, outperforming existing RL solutions.

Network slicing has been proposed as a paradigm for 5G+ networks. The operators slice physical resources from the edge all the way to the datacenter, and are responsible to micro-manage the allocation of these resources among tenants bound by predefined Service Level Agreements (SLAs). A key task, for which recent works have advocated the use of Deep Neural Networks (DNNs), is tracking the tenant demand and scaling its resources. Nevertheless, for the edge resources (e.g., RAN), a question arises on whether operators can: (a) scale them fast enough (often in the order of ms) and (b) afford to transmit huge amounts of data towards a remote cloud where such a DNN model might operate. We propose a Distributed DNN (DDNN) architecture for a class of such problems: a small subset of the DNN layers at the edge attempt to act as fast, standalone resource allocator; this is complemented by a mechanism to intelligently offload a percentage of (harder) decisions to additional DNN layers running at a remote cloud. To implement the offloading, we propose: (i) a Bayes-inspired method, using dropout during inference, to estimate the confidence in the local prediction; (ii) a learnable function which automatically classifies samples as “remote” (to be offloaded) or “local”. Using the public Milano dataset, we investigate how such a DDNN should be trained and operated to address (a) and (b). In some cases, our offloading methods are near-optimal, resolving up to 50% of decisions locally with little or no penalty on the allocation cost.

This work addresses the challenge of Dynamic Multi-Channel Access (DMCA) in the context of Ultra Reliable Low Latency Communications (URLLC), a framework subjected to notably stringent constraints, required by numerous Internet of Things (IoT) applications across various sectors. We introduce a theoretically grounded approach, leveraging Deep Multi-Agent Reinforcement Learning (MARL) to tackle this problem. While prior research has not fully addressed the DMCA problem in URLLC networks under time-varying heterogeneous channels and traffic profiles, nor provided robust theoretical guarantees in the multi-agent context, this paper adapts the recent theoretical framework of Trust Region Policy Optimization (TRPO) in MARL to meet the specific challenges and requirements of the URLLC-DMCA problem. Specifically, we introduce Multi Channel Access - Proximal Policy Optimization (MCA-PPO), a MARL algorithm that benefits from theoretical guarantees and effectively handles the partial observability and the combinatorial nature of the DMCA challenge. We validate the superiority of our proposed method across a variety of heterogeneous scenarios, in terms of traffic models and system parameters, and show that we outperform the traditional multiple access benchmark and learning algorithms.

To accommodate low latency and computation-intensive services, such as the Internet-of-Things (IoT), 5G networks are expected to have cloud and edge computing capabilities. To this end, we consider a generic network setup where devices, performing analytics-related tasks, can partially process a task and offload its remainder to base stations, which can then reroute it to cloud and/or to edge servers. To account for the potentially unpredictable traffic demands and edge network dynamics, we formulate the resource allocation as an online convex optimization problem with service violation constraints and allow limited communication between neighboring nodes. To address the problem, we propose an online distributed (across the nodes) primal-dual algorithm and prove that it achieves sublinear regret and violation; in fact, the achieved bound is of the same order as the best known centralized alternative. Our results are further supported using the publicly available Milano dataset.

In this paper we investigate the problem of uplink scheduling in a multicell system in order to minimize the total queuing delay at the mobile devices. The proposed setting introduces an environment with multiple interacting decision makers: Base Stations are considered as agents who have partial view of the system and interact with each other through interference generated by their scheduled devices for uplink transmissions. In addition, since traffic and channel processes are unknown, finding the optimal global scheduling policy can be modeled as a multiagent reinforcement learning problem. In this work, we propose distributed learning algorithms based on policy gradient to tackle this problem and investigate the impact of information exchange between Base Stations. Our results illustrate that the proposed algorithm outperforms standard schedulers, such as Proportional Fair and MaxWeight and that information exchange is crucial for challenging problem instances, such as topologies with many devices on the cell edge and/or high traffic demands.

In this paper, we propose a novel algorithm for video bitrate adaptation in HTTP Adaptive Streaming (HAS), based on online learning. The proposed algorithm, named Learn2Adapt (L2A) , is shown to provide a robust bitrate adaptation strategy which, unlike most of the state-of-the-art techniques, does not require parameter tuning, channel model assumptions, or application-specific adjustments. These properties make it very suitable for mobile users, who typically experience fast variations in channel characteristics. Experimental results, over real 4G traffic traces, show that L2A improves on the overall Quality of Experience (QoE) and in particular the average streaming bitrate, a result obtained independently of the channel and application scenarios.

This paper considers the Multiple Access problem where$N$Internet of Things (IoT) devices share a common wireless medium towards a central Base Station (BS). We propose a Reinforcement Learning (RL) method where the BS is the agent and the devices are part of the environment. A device is allowed to transmit only when the BS decides to schedule it. Besides the information packets, devices send additional messages like the delay or the number of discarded packets since their last transmission. This information is used to design the RL reward function and constitutes the next observation that the agent can use to schedule the next device. Leveraging RL allows us to learn the sporadic and heterogeneous traffic patterns of the IoT devices and an optimal scheduling policy that maximizes the channel throughput. We adapt the Proximal Policy Optimization (PPO) algorithm with a Recurrent Neural Network (RNN) to handle the partial observability of our problem and exploit the temporal correlations of the users' traffic. We demonstrate the performance of our model through simulations on different number of heterogeneous devices with periodic traffic and individual latency constraints. We show that our RL algorithm outperforms traditional scheduling schemes and distributed medium access algorithms.

Today’s traffic projections speak of almost 58% video traffic across the Internet. Nearly all video traffic is encrypted, accounting for more than 50% encrypted traffic worldwide. To analyze video traffic today, or even estimate its quality in the network, a deep look into the traffic characteristics has to be done. But then, important quality metrics from the traffic behavior can be derived. Based on extensive measurements we show in this work how to measure and estimate video stalls for mobile adaptive streaming. The underlying dataset includes more than 900 hours of video footage from the native YouTube app, measured over 18 different videos in 56 network scenarios in two cities in Europe. We outline a possible approach to estimate the video playback buffer size based on uplink video chunk requests in real-time to break down the video stalls. This work is intended as a tool for network operators to receive further knowledge of the characteristics of video streaming traffic to quantify the most important QoE degradation factors of one of the most important applications today.

We consider the dynamic spectrum access (DSA) problem where K Internet of Things (IoT) devices compete for T time slots constituting a frame. Devices collectively monitor M events where each event could be monitored by multiple IoT devices. Each device, when at least one of its monitored events is active, picks an event and a time slot to transmit the corresponding active event information. In the case where multiple devices select the same time slot, a collision occurs and all transmitted packets are discarded. In order to capture the fact that devices observing the same event may transmit redundant information, we consider the maximization of the average sum event rate of the system instead of the classical frame throughput. We propose a multi-agent reinforcement learning approach based on a stochastic version of Multi-Agent Deep Deterministic Policy Gradient (MADDPG) to access the frame by exploiting device-level correlation and time correlation of events. Through numerical simulations, we show that the proposed approach is able to efficiently exploit the aforementioned correlations and outperforms benchmark solutions such as standard multiple access protocols and the widely used Independent Deep Q-Network (IDQN) algorithm.

Responding fluently to user requests is important to keep them immersed. In this paper, we are presenting an extensive dataset to study the response time of YouTube's mobile video streaming service on Android. We illustrate the application of our dataset by studying YouTube's initial delay for a subset of 9 videos in 75 network scenarios. We find that in 41% of the cases, YouTube exceeds the attention span of a typical user, while deep immersion is only reached in 15% of the cases. Our factor analysis implies that the allocation of the initial CDN node is the critical link in this delay chain. Since our dataset includes a large variety of factors, we are describing setup, methodology, and data structure in detail. Our dataset and measurement tools are publicly available at [8].

In this paper, we develop an analytical framework to compute the Quality-of-Experience (QoE) metrics of video streaming in wireless networks. Our framework takes into account the system dynamics that arises due to the arrival and departure of flows. We also consider the possibility of users abandoning the system on account of poor QoE. Considering the coexistence of multiple services such as video streaming and elastic flows, we use a Markov chain based analysis to compute the user QoE metrics: probability of starvation, prefetching delay, average video quality and bitrate switching. Our simulation results validate the accuracy of our model and describe the impact of scheduler at eNB on the QoE metrics.

Dynamic Adaptive Streaming over HTTP (DASH) has become the standard choice for live events and on-demand video services. In fact, by performing bitrate adaptation at the client side, DASH operates to deliver the highest possible Quality of Experience (QoE) under given network conditions. In cellular networks, in particular, video streaming services are affected by mobility and cell load variation. In this context, DASH video clients continually adapt the streaming quality to cope with channel variability. However, since they operate in a greedy manner, adaptive video clients can overload cellular network resources, degrading the QoE of other users and suffer persistent bitrate oscillations. In this paper, we tackle this problem using a new eNB scheduler, named Shadow-Enforcer, which ensures minimal number of quality switches as well as efficient and fair utilization of network resources. Our scheduler works well under dynamic scenarios and mobility, and requires minimal information, i.e., just the set of video bitrates supported by DASH video clients. It consists of the cascade of a virtual scheduler, Shadow, and the actual scheduler, Enforcer, piloted by the virtual one. Extensive simulations demonstrate the efficiency, fairness and the smooth response to channel variations of the proposed solution.

Video streaming over cellular network has become extremely popular in 4G and will be an integral part of future cellular networks. While most modern-day video clients continually adapt quality of video streams, they neither coordinate with network elements nor among each other. Consequently, a streaming client may quickly overload the cellular network, leading to poor Quality of Experience (QoE) for users in the network. Motivated by this problem, we present D-VIEWS - a scheduling paradigm that assures video bitrate stability of adaptive video streams while ensuring better system utilization. D-VIEWS only needs to be aware of the set of video bitrates and requires no changes to streaming clients and other network functions. Through simulations, we also study the performance of proportional fairness scheduler and D-VIEWS in the presence of user arrival and departure events.

Accurate cross-layer information is very useful to optimize mobile networks for specific applications. However, providing application-layer information to lower protocol layers has become very difficult due to the wide adoption of end-to-end encryption and due to the absence of cross-layer signaling standards. As an alternative, this paper presents a traffic profiling solution to passively estimate parameters of HTTP Adaptive Streaming (HAS) applications at the lower layers. By observing IP packet arrivals, our machine learning system identifies video flows and detects the state of an HAS client's play-back buffer in real time. Our experiments with YouTube's mobile client show that Random Forests achieve very high accuracy even with a strong variation of link quality. Since this high performance is achieved at IP level with a small, generic feature set, our approach requires no Deep Packet Inspection (DPI), comes at low complexity, and does not interfere with end-to-end encryption. Traffic profiling is, thus, a powerful new tool for monitoring and managing even encrypted HAS traffic in mobile networks.

This demo introduces a wrapper used for automated measurements of mobile video streaming in the Android YouTube app. The difference to traditional measurement techniques is that the measurement is done with the native YouTube app as it is provided in the Google Play Store. In addition to bandwidth or packet loss detection, the QoE of the video stream can be measured and quantified. For this, the amount of quality changes, the current playtime, the buffer level, and statistics like video and audio format are captured. Thus, detailed relationships between network parameters and streaming behavior based on many factors can be detected within the native app available in the Play Store.

Adaptive video streaming improves users' quality of experience (QoE), while using the network efficiently. In the last few years, adaptive video streaming has seen widespread adoption and has attracted significant research effort. We study a dynamic system of random arrivals and departures for different classes of users using the adaptive streaming industry standard DASH (Dynamic Adaptive Streaming over HTTP). Using a Markov chain based analysis, we compute the user QoE metrics: probability of starvation, prefetching delay, average video quality and switching rate. We validate our model by simulations, which show a very close match. Our study of the playout buffer is based on client adaptation scheme, which makes efficient use of the network while improving users' QoE. We prove that for buffer-based variants, the average video bit-rate matches the average channel rate. Hence, we would see quality switches whenever the average channel rate does not match the available video bit rates. We give a sufficient condition for setting the playout buffer threshold to ensure that quality switches only between adjacent quality levels.

This paper describes a novel system that provides key parameters of HTTP Adaptive Streaming (HAS) sessions to the lower layers of the protocol stack. A non-intrusive traffic profiling solution is proposed that observes packet flows at the transmit queue of base stations, edge-routers, or gateways. By analyzing IP flows in real time, the presented scheme identifies different phases of an HAS session and estimates important application-layer parameters, such as play-back buffer state and video encoding rate. The introduced estimators only use IP-layer information, do not require standardization and work even with traffic that is encrypted via Transport Layer Security (TLS). Experimental results for a popular video streaming service clearly verify the high accuracy of the proposed solution. Traffic profiling, thus, provides a valuable alternative to cross-layer signaling and Deep Packet Inspection (DPI) in order to perform efficient network optimization for video streaming.

HTTP Adaptive Streaming (HAS) techniques are now the dominant solution for video delivery in mobile networks. Over the past few years, several HAS algorithms have been introduced in order to improve user quality-of-experience (QoE) by bit-rate adaptation. Their difference is mainly the required input information, ranging from network characteristics to application-layer parameters such as the playback buffer. Interestingly, despite the recent outburst in scientific papers on the topic, a comprehensive comparative study of the main algorithm classes is still missing. In this paper we provide such comparison by evaluating the performance of the state-of-the-art HAS algorithms per class, based on data from field measurements. We provide a systematic study of the main QoE factors and the impact of the target buffer level We conclude that this target buffer level is a critical classifier for the studied HAS algorithms. While buffer-based algorithms show superior QoE in most of the cases, their performance may differ at the low target buffer levels of live streaming services. Overall, we believe that our findings provide valuable insight for the design and choice of HAS algorithms according to networks conditions and service requirements.

In anticipatory networking, channel prediction is used to improve communication performance. This paper describes a new approach for allocating resources to video streaming traffic while accounting for quality of service. The proposed method is based on integrating a model of the user's local play-out buffer into the radio access network. The linearity of this model allows to formulate a Linear Programming problem that optimizes the trade-off between the allocated resources and the stalling time of the media stream. Our simulation results demonstrate the full power of anticipatory optimization in a simple, yet representative, scenario. Compared to instantaneous adaptation, our anticipatory solution shows impressive gains in spectral efficiency and stalling duration at feasible computation time while being robust against prediction errors.

This paper presents a simple and effective method to study the spectral and energy efficiency (SE-EE) trade-off in cellular networks, an issue that has attracted significant recent interest in the wireless community. The proposed theoretical framework is based on an optimal radio resource allocation of transmit power and bandwidth for the downlink direction, applicable for an orthogonal cellular network. The analysis is initially focused on a single cell scenario, for which in addition to the solution of the main SE-EE optimization problem, it is proved that a traffic repartition scheme can also be adopted as a way to simplify this approach. By exploiting this interesting result along with properties of stochastic geometry, this work is extended to a more challenging multicell environment, where interference is shown to play an essential role and for this reason several interference reduction techniques are investigated. Special attention is also given to the case of low signal-to-noise ratio (SNR) and a way to evaluate the upper bound on EE in this regime is provided. This methodology leads to tractable analytical results under certain common channel properties, and thus allows the study of various models without the need for demanding system-level simulations.

The exponentially growing traffic in cellular networks induced standardization groups to focus on spectral efficiency (SE) and providers to densify their networks in crowded areas. The price to pay is a significant reduction of the energy efficiency (EE). As a result, more balanced EE-SE solutions have attracted significant interest lately in the wireless community. However, the Pareto optimal bound of this problem is so far not well established. This paper makes a step by defining precisely this bound in a typical cell where the interference-plus-noise distribution is known. An analytical EE-SE bound is derived considering an optimal superposition coding mode and also three sub-optimal time-frequency sharing approaches. The typical noise-plus-interference distribution used in this paper is obtained from Poisson distributed cellular network simulations validated by the Greentouch reference model, but the analytical results broadly apply to any other reference distribution.

The issue of energy efficiency (EE) in Orthogonal Frequency-Division Multiple Access (OFDMA) wireless networks is discussed in this paper. Our interest is focused on the promising concept of base station (BS) sleep mode, introduced recently as a key feature in order to dramatically reduce network energy consumption. The proposed technical approach fully exploits the properties of stochastic geometry, where the number of active cells is reduced in a way that the outage probability, or equivalently the signal to interference plus noise (SINR) distribution, remains the same. The optimal EE gains are then specified with the help of a simplified but yet realistic BS power consumption model. Furthermore, the authors extend their initial work by studying a non-singular path loss model in order to verify the validity of the analysis and finally, the impact on the achieved user capacity is investigated. In this context, the significant contribution of this paper is the evaluation of the theoretically optimal energy savings of sleep mode, with respect to the decisive role that the BS power profile plays.