The energy consumption of 5G base stations (BSs) is significantly higher than that of 4G BSs, creating challenges for operators due to increased costs and carbon emissions. Existing solutions address this issue by switching off BSs during specific periods or forming cooperation coalitions where some BSs deactivate while others serve users. However, these approaches often rely on fixed geographic configurations, making them unsuitable for urban areas with numerous BSs and mobile users. To tackle these challenges, we propose a hierarchical reinforcement learning (RL) framework for energy conservation in large-scale 5G networks. In the upper-layer, we propose a deep Q-network integrated with a graph convolutional network that dynamically groups BSs into coalitions from a macro perspective. This layer focuses on high-level coalition formation to optimize system-wide energy efficiency by considering the global state of the network. In the lower-layer, we combine attention mechanism with multi-agent RL and graph convolutional networks to design a scalable algorithm that maximizes local energy efficiency through optimizing the cooperation within each coalition. These two layers align global coalition dynamics with local intra-coalition cooperation to achieve system-wide energy optimization. Moreover, we accurately model large-scale urban 5G scenarios leveraging a high-fidelity network simulator, which enables our RL framework to learn from real-world feedback. Extensive experiments conducted with the simulator demonstrate that our proposed framework achieves remarkable energy savings of up to 75.6%, significantly outperforming baseline approaches. These findings highlight the effectiveness and superiority of our hierarchical RL optimization framework in addressing the energy consumption challenges faced by large-scale 5G networks.

Recently, artificial intelligence paves the way for the development of smart services for people anytime and anywhere, which poses great challenges on accessing computing resources. Multi-access edge computing complements existing cloud computing infrastructure at the edge of the network, where mobile users can offload computationally intensive tasks of smart applications to edge servers that are in proximity to the users themselves. Existing offloading schemes mainly focus on selecting edge servers for each offloading task with the goal of optimizing the overall average latency. However, the solutions with the optimal overall average latency may be not the most suitable for all offloading tasks. There is still a possibility that offloading leads to an extreme case of ultra-high latency, which is not acceptable for latency-sensitive applications. To address this problem, we therefore introduce modern portfolio theory (MPT) to jointly consider the overall average latency and potential risks in optimal edge server selection. The task offloading problem is regarded as an investment portfolio with the objective of maximizing the ‘return’ while minimizing the risk. Combining MPT with deep reinforcement learning (DRL), we design two proximal policy optimization (PPO)-based task offloading algorithms to jointly optimize these two objectives. The algorithm computes a portfolio for each mobile user that enables the diversification of edge server selection, thereby minimizing the risk and the average latency. Extensive simulation results based on three real-world trace datasets show that our algorithms significantly outperform the state-of-the-art solutions and can reduce the overall average latency and the risk by 59% and 85% at most, respectively.

Internet of Things (IoT) devices are increasingly growing in mobile networks with the ubiquity of various IoT services. They share the same infrastructure with smartphones while having different requirements for communication resources and security defense mechanisms. Distinguishing IoT devices from smartphones has far-reaching implications on effective network design, resource allocation scheme, pricing scheme, etc. In this article, we distinguish between 12 107 IoT devices and 12 693 smartphones in the real world via characterizing their network traffic. The IoT devices fall into five categories, namely, locating, monitoring, portable, point of sale (POS), and vehicle. We analyze the device behaviors from the network domain, physical domain, and time domain, make comparisons between each kind of IoT devices and smartphones, and design effective features based on the distinguishable network behavior characteristics at packet level, traffic level, and mobility level. Then, we train several classifiers based on our feature set to identify different kinds of mobile devices. Specifically, the accuracy of identifying IoT devices from smartphones achieves 95.86%, and the accuracies of distinguishing IoT devices in each category from smartphones are all over 95%. In the trained classifiers, feature importance verifies the discriminability of different network traffic characteristics observed in our multidomain measurement. Our study reveals the network traffic behavior characteristics for IoT devices, and successfully distinguishes them from smartphones, which paves the way for better network design, resource allocation, pricing scheme, and security defense mechanisms.

The last two decades have seen a clear trend toward crafting intelligent vehicles based on the significant advances in communication and computing paradigms, which provide a safer, stress-free, and more enjoyable driving experience. Moreover, emerging applications and services necessitate massive volumes of data, real-time data processing, and ultrareliable and low-latency communication (URLLC). However, the computing capability of current intelligent vehicles is minimal, making it challenging to meet the delay-sensitive and computation-intensive demand of such applications. In this situation, vehicular task/computation offloading toward the edge cloud (EC) and vehicular cloudlet (VC) seems to be a promising solution to improve the network’s performance and applications’ Quality of Service (QoS). At the same time, artificial intelligence (AI) has dramatically changed people’s lives. Especially for vehicular task offloading applications, AI achieves state-of-the-art performance in various vehicular environments. Motivated by the outstanding performance of integrating reinforcement learning (RL)/deep RL (DRL) to the vehicular task offloading systems, we present a survey on various RL/DRL techniques applied to vehicular task offloading. Precisely, we classify the vehicular task offloading works into two main categories: 1) RL/ DRL solutions leveraging EC and 2) RL/DRL solutions using VC computing. Moreover, the EC section-based RL/DRL solutions are further subcategorized into multiaccess edge computing (MEC) server, nearby vehicles, and hybrid MEC (HMEC). To the best of our knowledge, we are the first to cover RL/DRL-based vehicular task offloading. Also, we provide lessons learned and open research challenges in this field and discuss the possible trend for future research.

Internet of Thing (IoT) devices are rapidly becoming an indispensable part of our life with their increasing deployment in many promising areas, including tele-health, smart city, intelligent agriculture. Understanding the mobility of IoT devices is essential to improve quality of service in IoT applications, such as route planning in logistic management, infrastructure deployment, cellular network update and congestion detection in intelligent traffic. Despite its importance, there are not many results pertaining to the mobility of IoT devices. In this article, we aim to answer three research questions: (i) what are the mobility patterns of IoT device? (ii) what are the differences between IoT device and smartphone mobility patterns? (iii) how the IoT device mobility patterns differ among device types and usage scenarios? We present a comprehensive characterization of IoT device mobility patterns from the perspective of cellular data networks, using a 36-days long signal trace, including 1.5 million IoT devices and 0.425 million smartphones, collected from a nation-wide cellular network in China. We first investigate the basic patterns of IoT devices from two perspectives: temporal and spatial characteristics. Our study finds that IoT device mobility exhibits significantly different patterns compared with smartphones in multiple aspects. For instance, IoT devices move more frequently and have larger radius of gyration. Then we explore the essential mobility of IoT devices by utilizing two models that reveal the nature of human mobility, i.e., exploration and preferential return (EPR) model and entropy based predictability model. We find that IoT devices, with few exceptions, behave totally different from human, and we further derive a new formulation to describe their movement. We also find the gap mobility predictability and predictability limit between IoT and human is not as big as people expected.

Edge intelligence refers to a set of connected systems and devices for data collection, caching, processing, and analysis proximity to where data are captured based on artificial intelligence. Edge intelligence aims at enhancing data processing and protects the privacy and security of the data and users. Although recently emerged, spanning the period from 2011 to now, this field of research has shown explosive growth over the past five years. In this article, we present a thorough and comprehensive survey of the literature surrounding edge intelligence. We first identify four fundamental components of edge intelligence, i.e., edge caching, edge training, edge inference, and edge offloading based on theoretical and practical results pertaining to proposed and deployed systems. We then aim for a systematic classification of the state of the solutions by examining research results and observations for each of the four components and present a taxonomy that includes practical problems, adopted techniques, and application goals. For each category, we elaborate, compare, and analyze the literature from the perspectives of adopted techniques, objectives, performance, advantages and drawbacks, and so on. This article provides a comprehensive survey of edge intelligence and its application areas. In addition, we summarize the development of the emerging research fields and the current state of the art and discuss the important open issues and possible theoretical and technical directions.

Due to explosive growth of mobile data over the overburdened cellular network, opportunistic offloading is proposed to transmit the traffic originally transmitted through infrastructure network to delay-tolerant device-to-device networks at the edge of mobile networks. Existing offloading schemes mainly focus on selecting a subset of mobile nodes to work as offloadees, who first download the contents through the infrastructure, and then transmit them to subscriber nodes. However, when there is large amount of contents to be offloaded, and if all contents are disseminated through only one fixed set of offloadees every time, it may result in the risk of extreme cases that only extremely few nodes received the content, even if the expectation is large. This is like putting all of your eggs in one basket. To deal with this problem, we introduce an economic concept, conditional value at risk (CVaR) to measure the risk in offloadee set selection. CVaR is a widely used risk assessment measure for investments, which provides a unified and comprehensive risk evaluation framework for complex investment portfolio. We propose an algorithm to compute a portfolio over multiple offloadee sets with a guarantee on CVaR. Moreover, we prove the feasibility of our methodology from the mathematical point of view. Extensive evaluations with real-world traces show that our proposed offloading scheme can obtain a portfolio with significantly better CVaR, i.e., 91.53 percent performance gain at most, compared with the state-of-the-art solutions.