Federated Learning (FL) enables distributed model training across edge devices in a privacy-friendly manner. However, its efficiency heavily depends on effective device selection and high-dimensional resource allocation in dynamic and heterogeneous wireless environments. Conventional methods demand a confluence of domain-specific expertise, extensive hyperparameter tuning, and/or heavy interaction cost. This paper proposes a Tool-aided Evolutionary Large Language Model (T-ELLM) framework to generate a qualified policy for device selection in a wireless FL environment. Unlike conventional optimization methods, T-ELLM leverages natural language-based scenario prompts to enhance generalization across varying network conditions. The framework decouples the joint optimization problem mathematically, enabling tractable learning of device selection policies while delegating resource allocation to convex optimization tools. To facilitate the evolutionary process, T-ELLM interacts with a sample-efficient, model-based virtual learning environment that captures the relationship between device selection and learning performance. This developed virtual environment reduces reliance on real-world interactions, thus minimizing communication overhead while refining the LLM-based decision-making policy through group relative policy optimization. Theoretical analysis proves that the discrepancy between virtual and real environments is bounded, ensuring the advantage function learned in the virtual environment maintains a provably small deviation from real-world conditions. Experimental results demonstrate that T-ELLM outperforms benchmark methods in energy efficiency and exhibits robust adaptability to environmental changes.

Optimizing the deployment of large language models (LLMs) in edge computing environments is critical for enhancing privacy and computational efficiency. In the path toward efficient wireless LLM inference in edge computing, this study comprehensively analyzes the impact of different splitting points in mainstream open-source LLMs. Accordingly, this study introduces a framework taking inspiration from model-based reinforcement learning to determine the optimal splitting point across the edge and user equipment. By incorporating a reward surrogate model, our approach significantly reduces the computational cost of frequent performance evaluations. Extensive simulations demonstrate that this method effectively balances inference performance and computational load under varying network conditions, providing a robust solution for LLM deployment in decentralized settings.

Semantic communication (SemCom) has been deemed as a promising communication paradigm to break through the bottleneck of traditional communications. Nonetheless, most of the existing works focus more on point-to-point communication scenarios and its extension to multi-user scenarios is not that straightforward due to its cost-inefficiencies to directly scale the joint source-channel coding (JSCC) framework to the multi-user communication system. Meanwhile, previous methods optimize the system by differentiable bit-level supervision, easily leading to a “semantic gap”. Therefore, we delve into multi-user broadcast communication (BC) based on the universal transformer (UT) and propose a reinforcement learning (RL) based self-critical alternate learning (SCAL) algorithm, named SemanticBC-SCAL, to capably adapt to the different BC channels from one transmitter (TX) to multiple receivers (RXs) for sentence generation task. In particular, to enable stable optimization via a non-differentiable semantic metric, we regard sentence similarity as a reward and formulate this learning process as an RL problem. Considering the huge decision space, we adopt a lightweight but efficient self-critical supervision to guide the learning process. Meanwhile, an alternate learning mechanism is developed to provide cost-effective learning, in which the encoder and decoders are updated asynchronously as independent agents. Notably, the incorporation of RL makes SemanticBC-SCAL compliant with any user-defined semantic similarity metric and simultaneously addresses the channel non-differentiability issue by alternate learning. Besides, the convergence of SemanticBC-SCAL is also theoretically established. Extensive simulation results have been conducted to verify the effectiveness and superiorness of our approach, especially in low signal-to-noise ratio regions.

Vehicle-to-everything communications-assisted autonomous driving has witnessed remarkable advancements in recent years, with pragmatic communications (PragComm) emerging as a promising paradigm for real-time collaboration among vehicles and other agents. Simultaneously, extensive research has explored the interplay between collaborative perception and decision-making in end-to-end driving frameworks. In this work, we revisit the collaborative driving problem and propose the Select2Drive framework to optimize the utilization of limited computational and communication resources. Particularly, to mitigate cumulative latency in perception and decision-making, Select2Drive introduces distributed predictive perception by formulating an active prediction paradigm and simplifying high-dimensional semantic feature prediction into a computationally efficient, motion-aware reconstruction. Given the “less is more” principle that an over-broadened perceptual horizon possibly confuses the decision module rather than contributing to it, Select2Drive utilizes area-of-importance-based PragComm to prioritize the communication of critical regions, thus boosting both communication efficiency and decision-making efficacy. Empirical evaluations on the V2Xverse and real-world DAIR-V2X datasets demonstrate that Select2Drive achieves a 2.60% and 1.99% improvement in offline perception tasks under limited bandwidth (resp., pose error conditions). Moreover, it delivers at most 8.35% and 2.65% enhancement in closed-loop driving scores and route completion rates, particularly in scenarios characterized by dense traffic and high-speed dynamics.

Multiple quadrotor uncrewed aerial vehicles (UAVs) systems have garnered widespread research interest and fostered tremendous interesting applications, especially in multiconstrained pursuit-evasion games (MC-PEGs). The cooperative evasion and formation coverage (CEFC) task, where the UAV swarm aims to maximize formation coverage across multiple target zones while collaboratively evading predators, belongs to one of the most challenging issues in MC-PEGs, especially under communication-limited constraints. This multifaceted problem, which intertwines responses to obstacles, adversaries, target zones, and formation dynamics, brings up significant high-dimensional complications in locating a solution. In this article, we propose a novel two-level framework [i.e., consensus inference-based hierarchical reinforcement learning (CI-HRL)], which delegates target localization to a high-level policy, while adopting a low-level policy to manage obstacle avoidance, navigation, and formation. Specifically, in the high-level policy, we develop a novel multiagent reinforcement learning (RL) module, consensus-oriented multiagent communication (ConsMAC), to enable agents to perceive global information and establish consensus from local states by effectively aggregating neighbor messages. Meanwhile, we leverage an alternative training-based MAPPO (AT-M) and policy distillation to accomplish the low-level control. The experimental results, including the high-fidelity software-in-the-loop (SITL) simulations, validate that CI-HRL provides a superior solution with enhanced swarm's collaborative evasion and task completion capabilities.

Semantic Communication (SemCom) demonstrates strong superiority over conventional bit-level accurate transmission, by only attempting to recover the essential semantic information of data. Nevertheless, most SemCom works train the whole system in an End-to-End (E2E) way, with the assumption of a differentiable channel which is rare in reality applications. In this paper, to tackle the non-differentiability of channels, we propose an alternate learning-based sparse SemCom system with an SNR-adaptive capability for visual transmission, named SparseSBC-SADM. Specially, SparseSBC-SADM leverages two separate Deep Neural Network (DNN)-based models at the transmitter (TX) and receiver (RX), respectively. It alternates between learning the encoding and decoding processes, rather than the joint optimization commonly found in existing literature, to solve the non-differentiability in the channel. In particular, a “self-critic” training scheme is leveraged for stable training. Moreover, the DNN-based TX generates a sparse set of bits in deduced “semantic bases”, by further incorporating a binary quantization module by combining Compressive Sensing (CS) and DNN on the basis of minimal detrimental effect to the semantic accuracy. Furthermore, enlightened from the denoising steps in the Denoising Diffusion Model (DDM), a lightweight, SNR-Adaptive Denoising Module (SADM) is provisionally deployed at RX to improve data reconstruction with a gate mechanism to determine the activation under poor channel conditions. Extensive simulation results validate that SparseSBC-SADM shows efficient and effective transmission performance under various channel conditions, and outperforms typical SemCom solutions.

Semantic broadcast communications (SemanticBC) for image and text transmission have achieved significant performance gains for single tasks. Nevertheless, extending these methods to a multi-task scenario presents challenges, as different tasks often require distinct objective functions, and the shared encoder must handle potential conflicts effectively. In this paper, we propose a tri-level multiple gradient descent algorithm (MGDA) based reinforcement learning (RL) approach MagicRL for multi-task SemanticBC to effectively balance the multi-task learning conflicts and serve multiple different tasks simultaneously at the receiver sides, including classification and content-reconstruction tasks. In particular, we provide optimized decoders with given encoder parameters by a self-critical RL approach. Subsequently, MGDA-based weight assignment is applied to balance multiple decoders and a first-order Frank-Wolfe algorithm efficiently solves the underlying quadratic programming problem. On this basis, the encoder gets improved through a proper weighted summation of multi-task objective functions. Extensive simulation results have been conducted to verify the effectiveness, especially under low signal-to-noise ratio.

The paper considers independent reinforcement learning (IRL) for multi-agent collaborative decision-making in the paradigm of federated learning (FL). However, FL generates excessive communication overheads between agents and a remote central server, especially when it involves a large number of agents or iterations. Besides, due to the heterogeneity of independent learning environments, multiple agents may undergo asynchronous Markov decision processes (MDPs), which will affect the training samples and the model’s convergence performance. On top of the variation-aware periodic averaging (VPA) method and the policy-based deep reinforcement learning (DRL) algorithm (i.e., proximal policy optimization (PPO)), this paper proposes two advanced optimization schemes orienting to stochastic gradient descent (SGD): 1) A decay-based scheme gradually decays the weights of a model’s local gradients with the progress of successive local updates, and 2) By representing the agents as a graph, a consensus-based scheme studies the impact of exchanging a model’s local gradients among nearby agents from an algebraic connectivity perspective. This paper also provides novel convergence guarantees for both developed schemes, and demonstrates their superior effectiveness and efficiency in improving the system’s utility value through theoretical analyses and simulation results.

Multi-Agent Reinforcement Learning (MARL) has become a classic paradigm to solve diverse, intelligent control tasks like autonomous driving in Internet of Vehicles (IoV). However, the widely assumed existence of a central node to implement centralized federated learning-assisted MARL might be impractical in highly dynamic scenarios, and the excessive communication overheads possibly overwhelm the IoV system. Therefore, in this paper, we design a communication efficient cooperative MARL algorithm, named RSM-MAPPO, to reduce the communication overheads in a fully distributed architecture. In particular, RSM-MAPPO enhances the multi-agent Proximal Policy Optimization (PPO) by incorporating the idea of segment mixture and augmenting multiple model replicas from received neighboring policy segments. Afterwards, RSM-MAPPO adopts a theory-guided metric to regulate the selection of contributive replicas to guarantee the policy improvement. Finally, extensive simulations in a mixed-autonomy traffic control scenario verify the effectiveness of the RSM-MAPPO algorithm.

Semantic communication (SemCom) demonstrates strong superiority over conventional bit-level accurate transmission, by only attempting to recover the essential semantic information of data. In this paper, in order to tackle the non-differentiability of channels, we propose an alternate learning based SemCom system for visual transmission, named Spars-eSBC. Specially, SparseSBC leverages two separate Deep Neural Network (DNN)-based models at the transmitter and receiver, respectively, and learns the encoding and decoding in an alternate manner, rather than the joint optimization in existing literature, so as to solving the non-differentiability in the channel. In particular, a "self-critic" training scheme is leveraged for stable training. Moreover, the DNN-based transmitter generates a sparse set of bits in deduced "semantic bases", by further incorporating a binary quantization module on the basis of minimal detrimental effect to the semantic accuracy. Extensive simulation results validate that SparseSBC shows efficient and effective transmission performance under various channel conditions, and outperforms typical SemCom solutions.

Multi-Agent Reinforcement Learning (MARL) has reached astonishing achievements in various fields such as the traffic control of vehicles in a wireless connected environment. In MARL, how to effectively decompose a global feedback into the relative contributions of individual agents belongs to one of the most fundamental problems. However, the volatility of the environment (e.g., the vehicle movement and wireless disturbance) could significantly shape the time-varying topological relationships among agents, thus making the Value Decomposition (VD) challenging. Therefore, in order to cope with this annoying volatility, it becomes imperative to design a dynamic VD framework. Hence, in this paper, we propose a novel Stochastic VMIX (SVMIX) methodology by embedding the dynamic topological features into the VD and incorporating the corresponding components into a multi-agent actor-critic architecture. In particular, the Stochastic Graph Neural Network (SGNN) is leveraged to effectively extract underlying dynamics embedded in topological features and improve the flexibility of VD against the environment volatility. Finally, the superiority of SVMIX is verified through extensive simulations.

The paper considers independent reinforcement learning (IRL) for multi-agent decision-making process in the paradigm of federated learning (FL). We show that FL can clearly improve the policy performance of IRL in terms of training efficiency and stability. However, since the policy parameters are trained locally and aggregated iteratively through a central server in FL, frequent information exchange incurs a large amount of communication overheads. To reach a good balance between improving the model's convergence performance and reducing the required communication and computation over-heads, this paper proposes a system utility function and develops a consensus-based optimization scheme on top of the periodic averaging method, which introduces the consensus algorithm into FL for the exchange of a model's local gradients. This paper also provides novel convergence guarantees for the developed method, and demonstrates its superior effectiveness and efficiency in improving the system utility value through theoretical analyses and numerical simulation results.

With the development of network technology and the rapid growth of network equipment, the data throughput in the network is sharply increasing. To meet people’s requirements for low latency, the network architecture like Information-Centric Network (ICN) proposes to keep part of the content at the edge of network. In this paper, to maximize the cache hit rate, we propose a prediction model based on dynamic graph neural network (DGNN) to jointly learn the structural and temporal patterns embedded in the bipartite graph between users and visited content for predicting the content popularity. Furthermore, in order to strengthen the dynamic learning of graphs, we propose an age of information (AoI) based attention mechanism to extract useful historical information while avoiding the problem of message staleness. Extensive simulation results demonstrate that our model can obtain higher prediction accuracy, and generate a caching policy with boosted caching hits.

This paper investigates an air-ground integrated multi-access edge computing system, which is deployed by an infrastructure provider (InP). Under a business agreement with the InP, a third-party service provider provides computing services to the subscribed mobile users (MUs). MUs compete for the shared spectrum and computing resources over time to achieve their distinctive goals. From the perspective of an MU, we deliberately define the age of update to capture the staleness of information from refreshing computation outcomes. Given the system dynamics, we model the interactions among MUs as a stochastic game. In the Nash equilibrium without cooperation, each MU behaves in accordance with the local system states and conjectures. We can hence transform the stochastic game into a single-agent Markov decision process. As another major contribution, we develop an online deep reinforcement learning (RL) scheme that adopts two separate double deep Q-networks to approximate the Q-factor and the post-decision Q-factor, respectively. The deep RL scheme allows each MU to optimize the behaviours with unknown dynamic statistics. Numerical experiments show that our proposed scheme outperforms the baselines in terms of the average utility under various system conditions.

In this paper, we aim to propose a novel transmission control protocol (TCP) congestion control method from a cross-layer-based perspective and present a deep reinforcement learning (DRL)-driven method called DRL-3R (DRL for congestion control with Radio access network information and Reward Redistribution) so as to learn the TCP congestion control policy in a superior manner. In particular, we incorporate the RAN information to timely grasp the dynamics of RAN, and empower DRL to learn from the delayed RAN information feedback potentially induced by several consecutive actions. Meanwhile, we relax the implicit assumption (that the feedback to one specific action returns at a round-trip-time (RTT) after the action is applied) in previous researches, by redistributing the rewards and evaluating the merits of actions more accurately. Experiment results show that besides maintaining a reasonable fairness, DRL-3R significantly outperforms classical congestion control methods (e.g., TCP Reno, Westwood, Cubic, BBR and DRL-CC) on network utility by achieving a higher throughput while reducing delay in various network environments.

With the rapid evolution of wireless mobile devices, there emerges an increased need to design effective collaboration mechanisms between intelligent agents to gradually approach the final collective objective by continuously learning from the environment based on their individual observations. In this regard, independent reinforcement learning (IRL) is often deployed in multiagent collaboration to alleviate the problem of a nonstationary learning environment. However, behavioral strategies of intelligent agents in IRL can be formulated only upon their local individual observations of the global environment, and appropriate communication mechanisms must be introduced to reduce their behavioral localities. In this article, we address the problem of communication between intelligent agents in IRL by jointly adopting mechanisms with two different scales. For the large scale, we introduce the stigmergy mechanism as an indirect communication bridge between independent learning agents, and carefully design a mathematical method to indicate the impact of digital pheromone. For the small scale, we propose a conflict-avoidance mechanism between adjacent agents by implementing an additionally embedded neural network to provide more opportunities for participants with higher action priorities. In addition, we present a federal training method to effectively optimize the neural network of each agent in a decentralized manner. Finally, we establish a simulation scenario in which a number of mobile agents in a certain area move automatically to form a specified target shape. Extensive simulations demonstrate the effectiveness of our proposed method.

The neuropsychological characteristics inside the brain are still not sufficiently understood during the conventional image quality assessment procedure. In this paper, we extract the physiologically meaningful features of brain responses to different distortion levels images in conventional image assessment by combining persistent homology analysis with electroencephalogram (EEG). The experimental results show that more brain regions in the frontal lobe are involved when the subject perceives an unclear image compared to a clear one, which indicates that human perception of image quality might be related to the advanced cognitive processes to some extent. Meanwhile, a statistically significantly higher persistent entropy of EEG data evoked by a clear image compared to that of an unclear image is observed in several frequency bands. In general, this paper evaluates and quantifies quality-related neural correlates by persistent homology features of EEG signal, which provides an approach for a utility neurophysiological assessment of image quality.

Network slicing (NS) devotes to provisioning various services with distinct requirements over the same physical communication infrastructure. Considering a dense cellular network scenario that contains several NS over multiple base stations (BSs), it remains challenging to design a proper resource management strategy in real time, so as to cope with frequent BS handover and meet distinct service requirements. In this paper, we propose to formulate this challenge as a multiagent reinforcement learning (MARL) problem and leverage graph attention network (GAT) to strengthen the cooperation between agents. Furthermore, we incorporate GAT into deep Q network (DQN) and correspondingly design an intelligent resource management strategy for NS. Finally, we verify the superiority of the GAT-based DQN algorithm through extensive simulations.

The maximum capacity of fractal D2D (device-to-device) social networks with both direct and hierarchical communications is studied in this paper. Specifically, the fractal networks are characterized by the direct social connection and the self-similarity. First, for a fractal D2D social network with direct social communications, it is proved that the maximum capacity is$ \Theta (\frac{1}{\sqrt{n\,\log n}})$if a user communicates with one of his/her direct contacts randomly, where$ n$denotes the total number of users in the network, and it can reach up to$ \Theta (\frac{1}{\log n})$if any pair of social contacts with distance$ d$communicate according to the probability in proportion to$ d^{-\beta }$. Second, since users might get in touch with others without direct social connections through the inter-connected multiple users, the fractal D2D social network with these hierarchical communications is studied as well, and the related capacity is further derived. Our results show that this capacity is mainly affected by the correlation exponent$\epsilon$of the fractal structure. The capacity is reduced in proportional to$ \frac{1}{{\log n}}$if$ 2<\epsilon <3$, while the reduction coefficient is$ \frac{1}{n}$if$ \epsilon >3$.

Nowadays, deep neural networks (DNNs) have been rapidly deployed to realize a number of functionalities like sensing, imaging, classification, recognition, etc. However, the computational-intensive requirement of DNNs makes it difficult to be applicable for resource-limited Internet of Things IoT devices. In this paper, we propose a novel pruning-based paradigm that aims to reduce the computational cost of DNNs, by uncovering a more compact structure and learning the effective weights therein, on the basis of not compromising the expressive capability of DNNs. In particular, our algorithm can achieve efficient end-to-end training that transfers a redundant neural network to a compact one with a specifically targeted compression rate directly. We comprehensively evaluate our approach on various representative benchmark datasets and compared with typical advanced convolutional neural network (CNN) architectures. The experimental results verify the superior performance and robust effectiveness of our scheme. For example, when pruning VGG on CIFAR-10, our proposed scheme is able to significantly reduce its FLOPs (floating-point operations) and number of parameters with a proportion of 76.2% and 94.1%, respectively, while still maintaining a satisfactory accuracy. To sum up, our scheme could facilitate the integration of DNNs into the common machine-learning-based IoT framework, and establish distributed training of neural networks in both cloud and edge.

In this paper, we propose a novel network training mechanism called “dynamic channel propagation” to prune the neural networks during the training period. In particular, we pick up a specific group of channels in each convolutional layer to participate in the forward propagation in training time according to the significance level of channel, which is defined as channel utility. The utility values with respect to all selected channels are updated simultaneously with the error back-propagation process and will adaptively change. Furthermore, when the training ends, channels with high utility values are retained whereas those with low utility values are discarded. Hence, our proposed scheme trains and prunes neural networks simultaneously. We empirically evaluate our novel training scheme on various representative benchmark datasets and advanced convolutional neural network (CNN) architectures, including VGGNet and ResNet. The experiment results verify the superior performance and robust effectiveness of our approach.

Multi-agent system (MAS) needs to mobilize multiple simple agents to complete complex tasks. However, it is difficult to coherently coordinate distributed agents by means of limited local information. In this demo, we propose a decentralized collaboration method named as "stigmergy" in network-assisted MAS, by exploiting digital pheromones (DP) as an indirect medium of communication and utilizing deep reinforcement learning (DRL) on top. Correspondingly, we implement an experimental platform, where KHEPERA IV robots form targeted specific shapes in a decentralized manner. Experimental results demonstrate the effectiveness and efficiency of the proposed method. Our platform could be conveniently extended to investigate the impact of network factors (e.g., latency, data rate, etc).

This paper investigates an unmanned aerial vehicle (UAV)-assisted mobile-edge computing (MEC) system, in which the UAV provides complementary computation resource to the terrestrial MEC system. The UAV processes the received computation tasks from the mobile users (MUs) by creating the corresponding virtual machines. Due to finite shared I/O resource of the UAV in the MEC system, each MU competes to schedule local as well as remote task computations across the decision epochs, aiming to maximize the expected long-term computation performance. The non-cooperative interactions among the MUs are modeled as a stochastic game, in which the decision makings of a MU depend on the global state statistics and the task scheduling policies of all MUs are coupled. To approximate the Nash equilibrium solutions, we propose a proactive scheme based on the long short-term memory and deep reinforcement learning (DRL) techniques. A digital twin of the MEC system is established to train the proactive DRL scheme offline. Using the proposed scheme, each MU makes task scheduling decisions only with its own information. Numerical experiments show a significant performance gain from the scheme in terms of average utility per MU across the decision epochs.

Network slicing is a key technology in 5G communications system. Its purpose is to dynamically and efficiently allocate resources for diversified services with distinct requirements over a common underlying physical infrastructure. Therein, demand-aware resource allocation is of significant importance to network slicing. In this paper, we consider a scenario that contains several slices in a radio access network with base stations that share the same physical resources (e.g., bandwidth or slots). We leverage deep reinforcement learning (DRL) to solve this problem by considering the varying service demands as the environment state and the allocated resources as the environment action. In order to reduce the effects of the annoying randomness and noise embedded in the received service level agreement (SLA) satisfaction ratio (SSR) and spectrum efficiency (SE), we primarily propose generative adversarial network-powered deep distributional Q network (GAN-DDQN) to learn the action-value distribution driven by minimizing the discrepancy between the estimated action-value distribution and the target action-value distribution. We put forward a reward-clipping mechanism to stabilize GAN-DDQN training against the effects of widely-spanning utility values. Moreover, we further develop Dueling GAN-DDQN, which uses a specially designed dueling generator, to learn the action-value distribution by estimating the state-value distribution and the action advantage function. Finally, we verify the performance of the proposed GAN-DDQN and Dueling GAN-DDQN algorithms through extensive simulations.

In this paper, we investigate the problem of age of information (AoI)-aware radio resource management for expected long-term performance optimization in a Manhattan grid vehicle-to-vehicle network. With the observation of global network state at each scheduling slot, the roadside unit (RSU) allocates the frequency bands and schedules packet transmissions for all vehicle user equipment-pairs (VUE-pairs). We model the stochastic decision-making procedure as a discrete-time single-agent Markov decision process (MDP). The technical challenges in solving the optimal control policy originate from high spatial mobility and temporally varying traffic information arrivals of the VUE-pairs. To make the problem solving tractable, we first decompose the original MDP into a series of per-VUE-pair MDPs. Then we propose a proactive algorithm based on long short-term memory and deep reinforcement learning techniques to address the partial observability and the curse of high dimensionality in local network state space faced by each VUE-pair. With the proposed algorithm, the RSU makes the optimal frequency band allocation and packet scheduling decision at each scheduling slot in a decentralized way in accordance with the partial observations of the global network state at the VUE-pairs. Numerical experiments validate the theoretical analysis and demonstrate the significant performance improvements from the proposed algorithm.

Network slicing is a proposing technology to support diverse services from mobile users (MUs) over a common physical network infrastructure. In this paper, we consider radio access network (RAN)-only slicing, where the physical RAN is tailored to accommodate both computation and communication functionalities. Multiple service providers (SPs, i.e., multiple tenants) compete with each other to bid for a limited number of channels across the scheduling slots, aiming to provide their subscribed MUs the opportunities to access the RAN slices. An eavesdropper overhears data transmissions from the MUs. We model the interactions among the non-cooperative SPs as a stochastic game, in which the objective of a SP is to optimize its own expected long-term payoff performance. To approximate the Nash equilibrium solutions, we first construct an abstract stochastic game using the channel auction outcomes. Then we linearly decompose the per-SP Markov decision process to simplify the decision- makings and derive a deep reinforcement learning based scheme to approach the optimal abstract control policies. TensorFlow-based experiments verify that the proposed scheme outperforms the three baselines and yields the best performance in average utility per MU per scheduling slot.

Network slicing is a key technology in 5G communications system, which aims to dynamically and efficiently allocate resources for diversified services with distinct requirements over a common underlying physical infrastructure. Therein, demand-aware allocation is of significant importance to network slicing. In this paper, we consider a scenario that contains several slices in one base station on sharing the same bandwidth. Deep reinforcement learning (DRL) is leveraged to solve this problem by regarding the varying demands and the allocated bandwidth as the environment state and action, respectively. In order to obtain better quality of experience (QoE) satisfaction ratio and spectrum efficiency (SE), we propose generative adversarial network (GAN) based deep distributional Q network (GAN-DDQN) to learn the distribution of state-action values. Furthermore, we estimate the distributions by approximating a full quantile function, which can make the training error more controllable. In order to protect the stability of GAN-DDQN's training process from the widely-spanning utility values, we also put forward a reward-clipping mechanism. Finally, we verify the performance of the proposed GAN-DDQN algorithm through extensive simulations.

This paper targets at the problem of radio resource management for expected long-term delay-power tradeoff in vehicular communications. At each decision epoch, the road side unit observes the global network state, allocates channels and schedules data packets for all vehicle user equipment-pairs (VUE-pairs). The decision-making procedure is modelled as a discrete-time Markov decision process (MDP). The technical challenges in solving an optimal control policy originate from highly spatial mobility of vehicles and temporal variations in data traffic. To simplify the decision-making process, we first decompose the MDP into a series of per-VUE-pair MDPs. We then propose an online long short-term memory based deep reinforcement learning algorithm to break the curse of high dimensionality in state space faced by each per-VUE-pair MDP. With the proposed algorithm, the optimal channel allocation and packet scheduling decision at each epoch can be made in a decentralized way in accordance with the partial observations of the global network state at the VUE-pairs. Numerical simulations validate the theoretical analysis and show the effectiveness of the proposed online learning algorithm.

Software-defined security (SDS) overcomes the limitations of traditional security mechanisms, which brings significant merits for design, deployment and management. However, existing researches are usually limited to some independent algorithms, while not able to apply multiple algorithms to accommodate various types of attack in actual deployment. In this paper, we propose and implement a novel SDS framework, which aims to flexibly deploy a variety of security functions and artificial intelligence (AI) algorithms to automatically learn ongoing threats and proactively protect the network from attacks.

In this study, the authors consider the utilisation of multicast technology in cellular networks given different arrival patterns for the content requests of mobile users. Traditionally, the performance evaluation of multicast in the literature usually depends on the adoption of temporal Poisson processes for content requests, which is not accurate any more according to many real data measurements. Therefore, to make use of the bursty nature of content requests, they propose a hybrid unicast/multicast strategy where the base station (BS) can perform the unicast or multicast procedure according to its serving status. By modelling the complete process into a circular Markov chain, they derive the average latency of content requests and the average power consumption of BSs under different arrival patterns and serving configurations in theoretical and/or simulative ways. Moreover, the multicast threshold introduced in their strategy can be dynamically adjusted to achieve a joint optimisation between average latency and power consumption when confronted with varied demands. Numerous results show that the proposed strategy can not only reduce the average latency of content requests but also decrease the average power consumption of BSs, especially under the bursty request arrival patterns.

To improve the quality of computation experience for mobile devices, mobile-edge computing (MEC) is a promising paradigm by providing computing capabilities in close proximity within a sliced radio access network (RAN), which supports both traditional communication and MEC services. Nevertheless, the design of computation offloading policies for a virtual MEC system remains challenging. Specifically, whether to execute a computation task at the mobile device or to offload it for MEC server execution should adapt to the time-varying network dynamics. This paper considers MEC for a representative mobile user in an ultradense sliced RAN, where multiple base stations (BSs) are available to be selected for computation offloading. The problem of solving an optimal computation offloading policy is modeled as a Markov decision process, where our objective is to maximize the long-term utility performance whereby an offloading decision is made based on the task queue state, the energy queue state as well as the channel qualities between mobile user and BSs. To break the curse of high dimensionality in state space, we first propose a double deep Q-network (DQN)-based strategic computation offloading algorithm to learn the optimal policy without knowing a priori knowledge of network dynamics. Then motivated by the additive structure of the utility function, a Q-function decomposition technique is combined with the double DQN, which leads to a novel learning algorithm for the solving of stochastic computation offloading. Numerical experiments show that our proposed learning algorithms achieve a significant improvement in computation offloading performance compared with the baseline policies.

Software defined Internet of Things (SD-IoT) networks profit from centralized management and interactive resource sharing, which enhances the efficiency and scalability of Internet of Things applications. But with the rapid growth in services and applications, they are vulnerable to possible attacks and face severe security challenges. Intrusion detection has been widely used to ensure network security, but classical detection methods are usually signature-based or explicit-behavior-based and fail to detect unknown attacks intelligently, which makes it hard to satisfy the requirements of SD-IoT networks. In this paper, we propose an artificial intelligence-based two-stage intrusion detection empowered by software defined technology. It flexibly captures network flows with a global view and detects attacks intelligently. We first leverage Bat algorithm with swarm division and binary differential mutation to select typical features. Then, we exploit Random Forest through adaptively altering the weights of samples using the weighted voting mechanism to classify flows. Evaluation results prove that the modified intelligent algorithms select more important features and achieve superior performance in flow classification. It is also verified that our solution shows better accuracy with lower overhead compared with existing solutions.

With the cellular networks becoming increasingly agile, a major challenge lies in how to support diverse services for mobile users (MUs) over a common physical network infrastructure. Network slicing is a promising solution to tailor the network to match such service requests. This paper considers a system with radio access network (RAN)-only slicing, where the physical infrastructure is split into slices providing computation and communication functionalities. A limited number of channels are auctioned across scheduling slots to MUs of multiple service providers (SPs) (i.e., the tenants). Each SP behaves selfishly to maximize the expected long-term payoff from the competition with other SPs for the orchestration of channels, which provides its MUs with the opportunities to access the computation and communication slices. This problem is modelled as a stochastic game, in which the decision makings of a SP depend on the global network dynamics as well as the joint control policy of all SPs. To approximate the Nash equilibrium solutions, we first construct an abstract stochastic game with the local conjectures of channel auction among the SPs. We then linearly decompose the per-SP Markov decision process to simplify the decision makings at a SP and derive an online scheme based on deep reinforcement learning to approach the optimal abstract control policies. Numerical experiments show significant performance gains from our scheme.

To understand the spatial deployment of base stations (BSs) is the first step to analyze the performance of cellular networks and further design efficient networking protocols. Poisson point process (PPP), which has been widely adopted to characterize the deployment of BSs and established the reputation to give tractable results in the stochastic geometry analyses, usually assumes a static BS deployment density in homogeneous PPP (HPPP) models or delicately designed location-dependent density functions in in-homogeneous PPP models. However, the simultaneous existence of attractiveness and repulsiveness among BSs practically deployed in a large-scale area defies such an assumption, and the α-stable distribution, one kind of heavy-tailed distributions, has recently demonstrated superior accuracy to statistically model the varying BS density in different areas. In this paper, we start with these new findings and investigate the intrinsic feature (i.e., the spatial self-similarity) embedded in the BSs. Afterwards, we refer to a generalized PPP setup with α-stable distributed density and theoretically derive the related coverage probability. In particular, we give an upper bound of the derived coverage probability for high signal-to-interference-plus-noise ratio thresholds and show the monotonically decreasing property of this bound with respect to the variance of BS density. Besides, we prove that our model could reduce to the single-tier HPPP for some special cases and demonstrate the superior accuracy of the α-stable model to approach the real environment.

To improve the quality of computation experience for mobile devices, mobile-edge computing (MEC) is emerging as a promising paradigm by providing computing capabilities within radio access networks in close proximity. Nevertheless, the design of computation offloading policies for a MEC system remains challenging. Specifically, whether to execute an arriving computation task at local mobile device or to offload a task for cloud execution should adapt to the environmental dynamics in a smarter manner. In this paper, we consider MEC for a representative mobile user in an ultra dense network, where one of multiple base stations (BSs) can be selected for computation offloading. The problem of solving an optimal computation offloading policy is modelled as a Markov decision process, where our objective is to minimize the long-term cost and an offloading decision is made based on the channel qualities between the mobile user and the BSs, the energy queue state as well as the task queue state. To break the curse of high dimensionality in state space, we propose a deep Q-network-based strategic computation offloading algorithm to learn the optimal policy without having a priori knowledge of the dynamic statistics. Numerical experiments provided in this paper show that our proposed algorithm achieves a significant improvement in average cost compared with baseline policies.

Traffic prediction plays an important role in evaluating the performance of telecommunication networks and attracts intense research interests. A significant number of algorithms and models have been put forward to analyse traffic data and make prediction. In the recent big data era, deep learning has been exploited to mine the profound information hidden in the data. In particular, Long Short-Term Memory (LSTM), one kind of Recurrent Neural Network (RNN) schemes, has attracted a lot of attentions due to its capability of processing the long-range dependency embedded in the sequential traffic data. However, LSTM has considerable computational cost, which can not be tolerated in tasks with stringent latency requirement. In this paper, we propose a deep learning model based on LSTM, called Random Connectivity LSTM (RCLSTM). Compared to the conventional LSTM, RCLSTM makes a notable breakthrough in the formation of neural network, which is that the neurons are connected in a stochastic manner rather than full connected. We apply the RCLSTM to predict traffic and validate that the RCLSTM with even 35% neural connectivity still shows a satisfactory performance. When we gradually add training samples, the performance of RCLSTM becomes increasingly closer to the baseline LSTM. Moreover, for the input traffic sequences of enough length, the RCLSTM exhibits even superior prediction accuracy than the baseline LSTM.

Ultra-dense network deployment has been proposed as a key technique for achieving capacity goals in the fifth-generation (5G) mobile communication system. However, the deployment of smaller cells inevitably leads to more frequent handovers, thus making mobility management more challenging and reducing the capacity gains offered by the dense network deployment. In order to fully reap the gains for mobile users in such a network environment, we propose an intelligent dual connectivity mechanism for mobility management through deep learning-based mobility prediction. We first use LSTM (Long Short Term Memory) algorithm, one of deep learning algorithms, to learn every user equipment's (UE's) mobility pattern from its historical trajectories and predict its movement trends in the future. Based on the corresponding prediction results, the network will judge whether a handover is required for the UE. For the handover case, a dual connection will be established for the related UE. Thus, the UE can get the radio signal from two base stations in the handover process. Simulation results verify that the proposed intelligent dual connectivity mechanism can significantly improve the quality of service of mobile users in the handover process while guaranteeing the network energy efficiency.

The rapid urbanisation has transformed cities to the preferential human settlement and allowed cities to quietly witness all range of human activities. As the key enabler in the information and communications technology industry, cellular networks play a decisive role in delivering communication messages and entertainment content. In particular, cellular network operators respond to human initiated service requests by gradually deploying necessary infrastructure and calibrating transmission protocols. Hence, cellular network records encompass the interesting interaction between human‐initiated messages and network‐triggered responses. In this study, the authors collect the ‘big data’ in urban cellular networks and try to dig out the human and urban planning properties. Specifically, they focus on the statistical modelling of three representative scenarios like spatial deployment density of base stations, packet length or traffic volume of mobile services, as well as inter‐arrival time and dwell time of human mobility. Through extensive data mining, they validate the heavy‐tailed feature universally existing in these scenarios. Afterwards, they discuss the implications of this heavy‐tailed feature and talk about its fundamental contribution to intelligent resource adjustment, proactive content caching, and enhanced connection management in cellular networks. Finally, they highlight the applications of this feature towards smarter cellular networks and cities.

How to allocate the limited wireless resource in dense radio access networks (RANs) remains challenging. By leveraging a software-defined control plane, the independent base stations (BSs) are virtualized as a centralized network controller (CNC). Such virtualization decouples the CNC from the wireless service providers (WSPs). We investigate a virtualized RAN, where the CNC auctions channels at the beginning of scheduling slots to the mobile terminals (MTs) based on bids from their subscribing WSPs. Each WSP aims at maximizing the expected long-term payoff from bidding channels to satisfy the MTs for transmitting packets. We formulate the problem as a stochastic game, where the channel auction and packet scheduling decisions of a WSP depend on the state of network and the control policies of its competitors. To approach the equilibrium solution, an abstract stochastic game is proposed with bounded regret. The decision making process of each WSP is modeled as a Markov decision process (MDP). To address the signalling overhead and computational complexity issues, we decompose the MDP into a series of single-agent MDPs with reduced state spaces, and derive an online localized algorithm to learn the state value functions. Our results show significant performance improvements in terms of per-MT average utility.

When we talk about the queueing theory in cellular networks, M/M/1 always comes first, which supposes the inter-arrival time and service time are both exponentially distributed. However, with the prosperity of Mobile Instant Messaging (MIM), many researchers found that the inter-arrival times of users' MIM follow lognormal distribution and there are also many keep-alive(KA) messages produced by the keep-alive mechanism accompanied with the users' messages. Based on this discovery, we rethink the queueing model that people might apply to analyze the MIM traffic. Firstly, we focus on a queueing system with lognormally distributed inter-arrival time and exponentially distributed service time, and find that the time a customer spends for waiting in queue and the length of the queue are shorter and smaller than the situation that the inter-arrival time is exponential one as proposed by 3GPP. Secondly, we study the influence of the keep-alive mechanism on queueing process. After that, we carry out simulations for this model, and the numerical results show that there exists an optimal KA duration, which determines the minimal average waiting time.

The explosion of mobile data traffic has an adverse effect on communication delay and system performance in 5G networks. Mobile Edge Computing(MEC) is an emerging technology has become an essential solution, which provides services within the close proximity of users. In this paper, we present a edge caching structure for MEC and propose a CMAC(Cooperative Multicast-Aware Caching) strategy to reduce the average latency of delivering content. The strategy is based on the characteristics of multicast and cooperation between BSs. In this strategy, we pay more attention to the users' QoE and the content-access latency. The cooperative caching scheme for multicast is more important in improving the cache hit ratio and efficiency of content delivery. We also formulate the optimization problem in using CMAC strategy and introduce an algorithm with performance guarantees. The relevant trace-driven simulations reveal that CMAC yields up to 13% decrease in the average content- access latency compared to MAC(Multicast-Aware Caching) scheme with the same total cache capacity.

In order to ensure the transmit speed and quality, it is critical to adopt efficient path selection technique for load balancing in complex networks. However, traditional balancing technique is unable to possess global knowledge of the whole network and lacks the ability to find the optimal path. Software Defined Networks (SDN) provides an approach to obtain whole network status. But, the performance of controller limits the extension of SDN. This paper presents a novel intelligent SDN-based architecture and proposes a new scheme to optimize data transmission. The new scheme uses intelligent algorithms to complete three functions, including path selection, the important nodes and flow forecasting. Simulation results demonstrate that the proposed scheme achieves superior performance in terms of both latency and packet loss rate.

In conventional cellular networks, for base stations (BSs) that are deployed far away from each other, it is general to assume them to be mutually independent. Nevertheless, after long-term evolution of cellular networks in various generations, the assumption no longer holds. Instead, the BSs, which seem to be gradually deployed by operators in a casual manner, have embedded many fundamental features in their locations, coverage and traffic loading. Their features can be leveraged to analyze the intrinstic pattern in BSs and even human community. According to large-scale measurement datasets, we build spatial correlation model of BSs by utilizing one of the most important features, that is, traffic. Coupling with the theory of complex networks, we make further analysis on the structure and characteristics of this spatial correlation model. Simulation results show that its degree distribution follows scale-free property. Moreover, the datasets also unveil the characteristics of fractality and small-world. Furthermore, we study how to choose the appropriate metric to measure the importance of each BS with a combination of network efficiency and demonstrate that degree is relatively more important.

The proof-of-concept software-defined radio access network (RAN) is not flexible enough due to the inherent delay and the necessity of high-capacity fronthaul links. We are hence motivated to propose a hierarchical software-defined RAN architecture, over which the base stations (BSs) are abstracted into multiple virtual local controllers while these local controllers are administered by a high-level controller. Under such a hierarchical network architecture, we particularly investigate in this paper how to adapt the BS transmit power over a long term according to the network dynamics under the constraints of mobile user queue stability and limited fronthaul capacity. We first formulate an off-line stochastic power adaptation problem. Through developing the Lyapunov method, we transform the problem into an approximate on-line optimization task. However, the challenge arises from the introduced per-cluster fronthaul capacity constraint. To solve the task efficiently and avoid extensive information exchange between the high-level controller and the local controllers, we put forward a novel low-complexity algorithm by designing a non- cooperative power adaptation game among the local controllers. Simulations are provided to evaluate the efficacy of the proposed studies.

Traffic learning and prediction is at the heart of the evaluation of the performance of telecommunications networks and attracts a lot of attention in wired broadband networks. Now, benefiting from the big data in cellular networks, it becomes possible to make the analyses one step further into the application level. In this paper, we first collect a significant amount of application-level traffic data from cellular network operators. Afterward, with the aid of the traffic “big data,” we make a comprehensive study over the modeling and prediction framework of cellular network traffic. Our results solidly demonstrate that there universally exist some traffic statistical modeling characteristics at a service or application granularity, including α-stable modeled property in the temporal domain and the sparsity in the spatial domain. But, different service types of applications possess distinct parameter settings. Furthermore, we propose a new traffic prediction framework to encompass and explore these aforementioned characteristics and then develop a dictionary learning-based alternating direction method to solve it. Finally, we examine the effectiveness and robustness of the proposed framework for different types of application-level traffic. Our simulation results prove that the proposed framework could offer a unified solution for application-level traffic learning and prediction and significantly contribute to solve the modeling and forecasting issues.

Energy-efficient transmission is a hot topic in wireless communication due to the conflict between the booming traffic demand and the limited energy resource. In this paper, an algorithm for downlink power allocation combined with user association is studied to improve the energy efficiency of a software defined cellular network. The problem mentioned above is non- convex but can be solved with QPSO, a low-complexity and high-efficiency algorithm. Our new proposed algorithm optimizes the energy efficiency while maximum power constraints and minimum rate constrains are assured. Simulation result shows the system energy efficiency precede to that of previous algorithm, when the proposed scheme is applied.

Understanding traffic characteristics in cellular networks is of great significance for better network design and performance optimization. The rapid development of various social networking applications for smart devices makes it an imperative to carry out cellular data traffic analysis further into the application level. In this paper, based on a plenty of practical mobile data traffic records, we focus on three typical application types and draw conclusions in terms of statistical characteristics and appropriate distribution model for social mobile data traffic. Firstly, the universal existence of burstiness and self-similarity is demonstrated by testing traffic series at different time scales. Afterwards, α-stable distributions are used to model traffic series benefiting from their internal burstiness and self- similarity. The minor fitting errors verify the validity of α-stable model and a preliminary traffic prediction shows the usefulness of α-stable model for further traffic analysis.

As current cellular networks are becoming increasingly heterogeneous, traditional cell association rule based on the downlink reference signal receiving power (RSRP) no longer serves well. Under this circumstance, the concept of downlink (DL) and uplink (UL) decoupling (DUDe), in which user equipments choose the serving Base Station (BS) in DL and UL separately, has drawn great attention during the design of next generation cellular networks. In this paper, using the framework of stochastic geometry, we at first derive the theoretical energy efficiency (EE) distribution of a two-tier heterogeneous network both in DL and UL with DUDe. Then through numerical simulation, we verify that the DUDe rule can improve EE of a heterogeneous network comparing with the traditional RSRP rule. The improvement is shown to be significant through a further comparison with bias-based Cell Range Extension (CRE).

This paper addresses energy-aware traffic offloading in stochastic heterogeneous cellular networks (HCNs). The objective is to minimize energy consumption of the HCN while maintaining Quality-of-Service experienced by the mobile users. For each cell, the energy consumption depends on its associated system load, which is coupled with system loads in other cells due to the sharing over a common spectrum band. Such a traffic offloading problem is modeled by a discrete-time Markov decision process (DTMDP). Based on the traffic observations and the traffic offloading operations, the network controller learns to solve the optimal traffic offloading strategy with no prior knowledge of the DTMDP statistics. To deal with the curse of dimensionality, we design a centralized Q-learning with compact state representation algorithm, which is named as QC-learning. Moreover, a decentralized QC-learning algorithm is developed such that the macro-cell base stations (BSs) can independently manage the operations of small-cell BSs by making use of the network information obtained from the network controller. Simulations validate the proposed studies.

This paper first provides a brief survey on existing traffic offloading techniques in wireless networks. Particularly as a case study, we put forward an online reinforcement learning framework for the problem of traffic offloading in a stochastic heterogeneous cellular network (HCN), where the time-varying traffic in the network can be offloaded to nearby small cells. Our aim is to minimize the total discounted energy consumption of the HCN while maintaining the quality-of-service (QoS) experienced by mobile users. For each cell (i.e., a macro cell or a small cell), the energy consumption is determined by its system load, which is coupled with system loads in other cells due to the sharing over a common frequency band. We model the energy-aware traffic offloading problem in such HCNs as a discrete-time Markov decision process (DTMDP). Based on the traffic observations and the traffic offloading operations, the network controller gradually optimizes the traffic offloading strategy with no prior knowledge of the DTMDP statistics. Such a model-free learning framework is important, particularly when the state space is huge. In order to solve the curse of dimensionality, we design a centralized Q-learning with compact state representation algorithm, which is named QC-learning. Moreover, a decentralized version of the QC-learning is developed based on the fact the macro base stations (BSs) can independently manage the operations of local small-cell BSs through making use of the global network state information obtained from the network controller. Simulations are conducted to show the effectiveness of the derived centralized and decentralized QC-learning algorithms in balancing the tradeoff between energy saving and QoS satisfaction.

This paper presents a design of linear-phase, low-complexity, reconfigurable digital filter bank that offers independent and complete control over the bandwidth as well as the center frequency of all subbands. The proposed filter bank is designed by integrating spectral parameter approximation (SPA) technique with the modified coefficient decimation method (MCDM), referred to as SPA-MCDM-FB. The architectural details, design examples and complexity comparisons show that the SPA-MCDM-FB is easy to design and offers substantial savings in gate count, number of variable multipliers and group delay over other filter banks. Moreover, these savings increase further with the increase in the filter-bank resolution (i.e., number of subbands). The SPA-MCDM-FB is then combined with the upper confidence bound (UCB)-based decision-making algorithm to search the vacant band(s) of any desired bandwidth for spectrum-sensing application in cognitive radio (CR). The simulations results verify that the proposed scheme offers superior performance [i.e., improved utilization of vacant subband(s)] and needs fewer gate counts compared to uniform filter bank and UCB-algorithm-based schemes. Furthermore, the functionality and advantages of the SPA-MCDM-FB are also verified for the channelization operation in CR supporting multiple communication standards.

This paper proposes a distributed cooperative framework for improving the energy efficiency of green cellular networks. Based on the traffic load, neighboring base stations (BSs) cooperate to optimize the BS switching (sleeping) strategies so as to maximize the energy saving while guaranteeing users' minimal service requirements. The inter-BS cooperation is formulated following the principle of ecological self-organization. An interaction graph is defined to capture the network impact of the BS switching operation. Then, we formulate the problem of energy saving as a constrained graphical game, where each BS acts as a game player with the constraint of traffic load. The constrained graphical game is proved to be an exact constrained potential game. Furthermore, we prove the existence of a generalized Nash equilibrium (GNE), and the best GNE coincides with the optimal solution of total energy consumption minimization. Accordingly, we design a decentralized iterative algorithm to find the best GNE (i.e., the global optimum), where only local information exchange among the neighboring BSs is needed. Theoretical analysis and simulation results finally illustrate the convergence and optimality of the proposed algorithm.

Rapid growing demand for mobile traffic and severe service bursts generated by various mobile applications are challenging the capacities and management of future cellular networks. The measurement data from real cellular networks indicate that the mobile Internet traffic expresses long-range dependence (LRD) characteristics, which differs from the traditional exponential assumption as well as 3GPP reports, would greatly deteriorate the experience of the subscribers. On the other hand, the mobile cellular networks treat most services without differentiation, although different services have their own requirements on transmission rates and delay. After introducing a new metric called the Quality of Experience (QoE) index for different services and analyzing the influence of LRD traffic, two approaches are proposed to alleviate the experience deterioration caused by traffic bursts: cloud based baseband resource pool to improve the flexibility of networks and wireless context-aware service controller to manage the distinctive QoE requirements of different services. Finally, simulations validate that these approaches not only contribute to service-oriented cross-layer optimization methods to satisfy the network resource constraints, but also provide a software-controlled service management framework for cellular networks.

Poisson Point Process (PPP) has been widely adopted as an efficient model for the spatial distribution of base stations (BSs) in cellular networks. However, real BSs deployment are rarely completely random, due to environmental impact on actual site planning. Particularly, for multi-tier heterogeneous cellular networks, operators have to place different BSs according to local coverage and capacity requirement, and the diversity of BSs' functions may result in different spatial patterns on each networking tier. In this paper, we consider a two-tier scenario that consists of macrocell and microcell BSs in cellular networks. By analyzing these two tiers separately and applying both classical statistics and network performance as evaluation metrics, we obtain accurate spatial model of BSs deployment for each tier. Basically, we verify the inaccuracy of using PPP in BS locations modeling for either macrocells or microcells. Specifically, we find that the first tier with macrocell BSs is dispersed and can be precisely modelled by Strauss point process, while Matern cluster process captures the second tier's aggregation nature very well. These statistical models coincide with the inherent properties of macrocell and microcell BSs respectively, thus providing a new perspective in understanding the relationship between spatial structure and operational functions of BSs.

The spatially distributed sparsity of the mobile devices (MDs) in indoor wireless local area networks (WLANs) makes compressive sensing (CS) based localisation algorithms feasible and desirable. In this Letter, the authors exploit the most recent developments in CS to efficiently perform localisation in WLANs and design an accurate indoor localisation scheme by taking advantage of the theory of multi‐task Bayesian CS (MBCS). The proposed scheme assembles the strength measurements of signals from the MDs to distinct access points (APs) and jointly utilises them at a central unit or a specific AP to achieve localisation, thus being able to alleviate the burden of MDs while simultaneously giving a precise estimation of the locations. Afterwards, they give a deeper insight into the localisation problem in more practical scenarios with varying number of MDs and investigate two different adaptive algorithms to meet the satisfactory localisation error requirement. Compared with the conventional MBCS algorithms, simulation results validate that both adaptive algorithms could provide superior localisation accuracy and exhibit stronger resilience to the changes in the number of MDs.

Recent works have validated the possibility of improving energy efficiency in radio access networks (RANs), achieved by dynamically turning on/off some base stations (BSs). In this paper, we extend the research over BS switching operations, which should match up with traffic load variations. Instead of depending on the dynamic traffic loads which are still quite challenging to precisely forecast, we firstly formulate the traffic variations as a Markov decision process. Afterwards, in order to foresightedly minimize the energy consumption of RANs, we design a reinforcement learning framework based BS switching operation scheme. Furthermore, to speed up the ongoing learning process, a transfer actor-critic algorithm (TACT), which utilizes the transferred learning expertise in historical periods or neighboring regions, is proposed and provably converges. In the end, we evaluate our proposed scheme by extensive simulations under various practical configurations and show that the proposed TACT algorithm contributes to a performance jump start and demonstrates the feasibility of significant energy efficiency improvement at the expense of tolerable delay performance.

In traditional hard combing cooperative detection strategy, the fusion center (FC) determines whether the spectrum of interest is idle or occupied by primary user (PU) after collecting the decisions from all secondary users (SUs) involved. This paper introduces two kinds of simple and computationally efficient spectrum sensing schemes with which the final decision at the FC can be reached faster. Specifically, we proposed hard combing based sequential detection and ordered transmission schemes under different signal-to-noise ratio (SNR) distribution assumptions. In these schemes, the FC performs hypothesis test each time after it receiving one decision from a SU until the final decision can be made reliably. The simulation results show the proposed techniques can significantly reduce the number of data required in identification of the spectrum hole while achieving the same error probability compared with traditional methods.

This paper investigates energy efficiency for the two-tier femtocell networks through combining game theory and stochastic learning. With the Stackelberg game formulation, a hierarchical reinforcement learning framework is developed to study the joint expected utility maximization of macrocells and femtocells. The macrocells behave as the leaders and the femtocells are followers during the learning procedure. At each time step, the leaders commit to dynamic strategies based on the best responses of the followers, while the followers compete against each other with no further information but the leaders' strategy information. In this paper, two learning algorithms are proposed to schedule each cell's transmission power. Numerical results are presented to validate the proposed studies and show that the two learning algorithms substantially improve the energy efficiency of the femtocell networks.

The cross- and co-tier interference creates the challenges to facilitate the concept of heterogeneous cellular networks (HCNs) in practice. In this paper, we establish a combined learning framework to autonomously mitigate the destructive interference. The macrocell is modeled as the leader and protects itself through pricing the interference from small-cells, which are the followers in the stochastic learning process. During each epoch (an epoch consists of T time slots), the leader commits to a pricing policy by knowing the resource allocation policies of all followers, while the followers compete against each other in each time slot only with the leader's price information. In general, for any two consecutive epochs, the HCN states are highly correlated. The previous policy information can thus be leveraged to improve the learning performance. Numerical results support that the proposed study substantially protects the macrocell and at the same time, optimizes the energy efficiency in small-cells.

This paper considers designing efficient medium access strategies for secondary users (SUs) to select frequency channels to sense and access in cognitive radio networks. The interaction among the SUs is considered as a learning problem, in which every SU behaves as an intelligent agent. Each SU believes that its competitors alter their future medium access strategies in proportion to its own current strategy change. These beliefs adapt in accordance with limited information exchange. In this way, each SU can obtain the behavior feature of other users through conjecture, optimize the medium access strategy, and finally achieve the goal of reciprocity, based on which two learning algorithms are proposed. We show that the SUs' stochastic behaviors and beliefs converge to a steady state under some conditions. Numerical results are provided to evaluate the performance of the two algorithms, and show that the achieved system performance gain outperforms some existing protocols.

We propose a single framework for cooperative spectrum sensing of cognitive radio network as well as for the self-organization of femtocells. This constructs a large-scale spectrum database through cooperative spectrum sensing for cognitive radio network, which exploits the existing infrastructure of the two-tier (macrocell-femtocell) network and encourages a small cellular network, i.e., femtocell, to act as the rendezvous point between individual local spectrum monitors and the fusion center of spectrum database. Distributed sensing results reported from many cognitive radio users can be integrated into spectrum database via these rendezvous points. Thus the problem of the dependency of conventional cooperative spectrum sensing on common control channel is improved. Meanwhile, the interaction between cognitive radio users and femtocells can offer an intriguing possibility to improve the self-organization of femtocells by absorbing radio environment maps constructed by cognitive radio users. Game theory can be applied to formulate the mutual benefits in the cooperation between cognitive radio user and femtocell. We illustrate the numerical effects of the coalitions between cognitive radio users and femtocells.

Controlling the interference from secondary users utilizing TV white spaces (TVWS) to protect primary users (PUs) is of vital importance in multicarrier based cognitive radio (CR) systems. Due to the complexity in the real implementation scenarios, regulations relative to TV white spaces may not be fully implemented, and even meeting all the regulatory requirements can't guarantee that the primary users are not influenced completely. In this paper, we propose a power allocation algorithm based on location information of the secondary users to minimize the aggregate interference to the licensed TV receivers at the border of TV service contour while making the capacity of secondary system reach a certain level. In the meantime, we need to assure that the minimum aggregate interference meets the required interference protection ratios of incumbent users. Since the use of geolocation and database access is mandated by the regulatory authorities, it is quite easy for the white space database (WSDB) to obtain location information of the secondary users and allocate their power accordingly. Numerical simulations are carried out to validate the effectiveness of the proposed interference control algorithm.

As the scarce spectrum resource is becoming overcrowded, cognitive radio indicates great flexibility to improve the spectrum efficiency by opportunistically accessing the authorized frequency bands. One of the critical challenges for operating such radios in a network is how to efficiently allocate transmission powers and frequency resource among the secondary users (SUs) while satisfying the quality-of-service constraints of the primary users. In this paper, we focus on the noncooperative power allocation problem in cognitive wireless mesh networks formed by a number of clusters with the consideration of energy efficiency. Due to the SUs' dynamic and spontaneous properties, the problem is modeled as a stochastic learning process. We first extend the single-agent Q-learning to a multiuser context, and then propose a conjecture-based multiagent Q-learning algorithm to achieve the optimal transmission strategies with only private and incomplete information. An intelligent SU performs Q-function updates based on the conjecture over the other SUs' stochastic behaviors. This learning algorithm provably converges given certain restrictions that arise during the learning procedure. Simulation experiments are used to verify the performance of our algorithm and demonstrate its effectiveness of improving the energy efficiency.

Recent works have validated the possibility of energy efficiency improvement in radio access networks (RAN), depending on dynamically turn on/off some base stations (BSs). In this paper, we extend the research over BS switching operation, matching up with traffic load variations. However, instead of depending on the predicted traffic loads, which is still quite challenging to precisely forecast, we formulate the traffic variation as a Markov decision process (MDP). Afterwards, in order to foresightedly minimize the energy consumption of RAN, we adopt the actor-critic method and design a reinforcement learning framework based BS switching operation scheme. In the end, we evaluate our proposed scheme by extensive simulations under various practical configurations and prove the feasibility of significant energy efficiency improvement.

In cellular networks, the base station power consumption is not simply proportional to the traffic loads of its coverage. As the traffic load fluctuates spatially and temporally, the base stations consequently suffer from heavy energy wastage when the traffic loads of their coverage are low. In this paper, we propose a grid-based energy saving scheme over predicted traffic loads. We firstly take advantage of the spatial-temporal pattern of traffic loads and employ the compressed sensing method to predict the future traffic loads. Then, we propose a grid-based energy saving scheme to improve the energy efficiency through turning some base stations into sleeping mode while ensuring the quality of service. Results of the simulation with real traffic loads1finally show the accuracy of the traffic load prediction and large energy efficiency improvement.

As energy saving and environmental protection become an inevitable trend, researchers need to shift their focus to “green” oriented architecture design. Recent advances in the area of cognitive radio (CR) have significant potential towards “green” communications. One of the critical challenges for operating CRs in a wireless mesh network is how to efficiently allocate transmission powers and frequency resource among the secondary users (SUs) while satisfying the quality-of-service constraints of primary users. Due to the SUs' intelligent and selfish properties, this paper focuses on the non-cooperative spectrum sharing in cognitive wireless mesh networks formed by a number of clusters. In order to study the competition behaviors of SUs in a dynamic environment, the problem is modeled as a stochastic learning process. We first extend the single-agent reinforcement learning (RL) to a multi-user context, based on which a conjecture based multi-agent RL algorithm is proposed. A rational SU learns the optimal transmission strategy from the conjecture over the other SUs' responses.

The common control channel problem has become one of the main research challenges in a dynamic spectrum access (DSA) based ad hoc network without a global common channel. The basic question lies in how the control channels of the network are aligned in a reliable and distributed way with minimum management overhead. We propose in this paper a novel concept of the control channel cloud to handle this problem. A control channel cloud is a group of connected nodes using the same control channel. The idea is to have clouds evolved by merging so that few control channels are used in the network. In this paper the basic cloud operations are introduced to merge cloud in different situations. The cloud formation algorithms are proposed based on the cloud operations. We prove the convergence of the algorithms and study their performance by simulation.

IEEE 802.22 is the first worldwide Wireless Regional Area Networks (WRAN) standard based on cognitive radios (CR). In IEEE 802.22 WRAN, if a wireless microphone appears in any TV channel, the whole channel should be cleaned. 802.22 WRAN devices probably need to fractionally use the first adjacent channel for improving spectrum efficiency while ensuring no harmful interference is caused to the wireless microphone nearby. To determine the appropriate portion of the TV channel that WRAN can utilize, we firstly need to know the frequency location of wireless microphone. In this paper, we propose a two step DFT filter bank (TS-DFTFB) enhanced with an adaptive threshold method to detect the wireless microphone (WM). Comparing with conventional single step DFTFB, the TS-DFTFB method has low complexity while the detection precision is equal to single step DFTFB under the same condition. And with an adaptive method, the threshold can be kept very close to the noise power, which can increase the detection probability especially in the condition of low SNR.

Future wireless access networks, characterized by a large number of small cells densely distributed in metropolitan area, are likely to be the dominating form of future wireless communications. It is expected that the dynamic spectrum access (DSA) enabled by cognitive radio (CR) technologies will act as a key for their success. Considering the high cell density and the large size of network, the spectrum coexistence will be a prominent problem in future wireless access networks. In this paper we study the spectrum self-coexistence of DSA based wireless access networks. The objective is to improve the spectrum utilization among densely distributed cells. To achieve this, topology-aware distributed algorithms are proposed for mobile terminal (MT) association and access point (AP) channel selection. The proposed algorithms enable self-coordination of spectrum in studied networks. The simulation study shows the efficiency of distributed solutions to solve the spectrum coexistence problem in proposed networks.

Cognitive wireless mesh networks have great flexibility to improve the spectrum utilization by opportunistically accessing the authorized frequency bands, within which the secondary users (SUs) should not violate the quality of service (QoS) requirement of the primary users (PUs) while transmitting. In this paper, we consider inter-cluster connection among neighboring clusters under the framework of cognitive wireless mesh networks. Corresponding to the neighboring clusters, all nodes operate in half-duplex mode; hence exchanging control message usually needs four time slots by traditional scheme, which leads to a loss in networking and spectral efficiency especially at the gateway node. A novel scheme based on network coding is proposed, which needs only two time slots. Our simulation experiments reveal the following findings: the performances of traditional inter-cluster connection and network coding based inter-cluster connection are comparable. Next, how to choose optimal signal amplification factor at the gateway node according to the wireless environment is discussed. And we present an intelligent policy based on reinforcement learning to solve the problem. Theoretical analysis and numerical results both show the policy can achieve optimal throughput for the SUs in the long run.

Ultra Wideband (UWB) technology has been considered as one of the suitable transmission techniques for implementing a cognitive radio system, where spectrum-agile UWB waveforms achieve a number of spectral adaptation features. One approach for designing such waveforms relies on orthogonally combining PSWF (Prolate Spheroidal Wave Functions) based pulse wavelets. In a cognitive UWB radio environment when the transmit signal is M-ary pulse shape modulated, we can obtain M different eigenvalues corresponding to the M PSWF-based pulses while transmitting them through the multipath fading channel. Each eigenvalue represents the multipath channel gain for its corresponding pulse. We propose an optimal power allocation scheme depending on the eigenvalues to improve bit error rate (BER) performance in such a cognitive UWB radio environment. Numerical results show that the scheme outperforms the general equal power allocation scheme.

As the radio spectrum usage paradigm shifting from the traditional command and control allocation scheme to the open spectrum allocation scheme, wireless mesh networks meet new opportunities and challenges. The open spectrum allocation scheme has potential to provide those networks more capacity, and make them more flexible and reliable. However, the freedom brought by the new spectrum usage paradigm introduces spectrum management and network coordination challenges. In this paper, we study the network formation problem in cognitive radio based mesh networks. A cluster-based approach is proposed to form a mesh network in the context of cognitive radio scenario. Moreover, a topology management algorithm is developed to optimize the cluster configuration with regard to the network topology. The prominent feature of the proposed approach lies in the capability to adapt the cluster configuration to network and radio environment changes.

Cognitive ultra-wideband (UWB) radio is proposed to exploit the advantages and the unique features of enhancing UWB wireless technologies by utilizing cognitive radio functionality. In order to achieve the cognitive UWB radio, multiple pulse waveforms adaptation has been investigated for producing the expected spectral notches while matching with the Federal Communications Commission (FCC) spectral mask. Then the specifically designed pulse waveform with adaptive spectral notches is tested by high-speed digital processing unit for implementation estimation. To verify the pulse distortion effects of UWB antenna in the cognitive UWB radio, the generalized pulse waveform is further transferred through an actual UWB antenna. Furthermore, a pulse waveform predistortion scheme is considered before antenna transmission, so as to compensate the pulse distortion effects.

Cooperative transmission scheme utilizing Space-Frequency coding (SFC) is investigated for cognitive Ultra Wideband (UWB) communications and networks, in the cases where multiple relaying is necessary among distributed nodes. The new scheme avoids the issue of the lack of required perfect synchronization on propagation delay and timing among the distributed nodes while using virtual multi-input multi-output (MIMO) algorithms. By computer simulations, it is shown that in the modified IEEE 802.15.3a S-V channel, the proposed SFC scheme combined with selective Rake receiver achieves almost the same BER performance compared with Alamouti's Space-Time block coding (STBC) performance. The most promising perspective of this new scheme relies on its application to Cognitive UWB Radio and its networking in potentially providing full spatial and frequency diversity. Moreover, the proposed cooperative relaying scheme is capable of matching with the FCC emission mask while still achieving interference avoidance as well as efficient wireless transmission.