Considering the random access behaviour of mobile devices (MDs), heterogeneous service requirements, and dynamic completion or discarding of tasks, the statistical characteristics of tasks to be scheduled, transmitted, and computed vary over time in practical multi-access edge computing (MEC) systems. Therefore, it is necessary to design a dynamic task offloading and resource allocation (TORA) strategy to adaptively match these time-varying task characteristics. To achieve the goal, we first formulate the dynamic TORA problem as a Markov decision process (MDP) with time-varying extension of state space and action space, which cannot be effectively solved by conventional deep reinforcement learning (DRL) algorithms. To address this challenge, we propose a general state-action space adaptive (SASA) DRL framework by exploiting the advantages of the Transformer architecture and its multi-head attention (MHA) mechanism. This framework facilitates the integration of available actor-critic DRL algorithms to efficiently solve MDPs with time-varying state and action spaces. Based on the proposed SASA DRL framework, we further develop the SASA-based TORA algorithm, referred to as SASA-TORA, which is adaptable to not only dynamic network conditions but also time-varying statistical characteristics of tasks. Simulation results demonstrate the superiority of SASA-TORA over baseline algorithms and highlight the limitations of conventional DRL algorithms in handling MDPs with time-varying state and action spaces.

The emergence of new intelligent applications has fostered the development of a task-oriented communication paradigm, where a comprehensive, universal, and practical metric is crucial for unleashing the potential of this paradigm. To this end, we introduce an innovative metric, the Task-oriented Age of Information (TAoI), to measure whether the content of information is relevant to the system task, thereby assisting the system in efficiently completing designated tasks. We apply TAoI to a wireless monitoring system tasked with identifying targets and transmitting their images for subsequent analysis. To minimize TAoI and determine the optimal transmission policy, we formulate the dynamic transmission problem as a Semi-Markov Decision Process (SMDP) and transform it into an equivalent Markov Decision Process (MDP). Our analysis demonstrates that the optimal policy is threshold-based with respect to TAoI. Building on this, we propose a low-complexity relative value iteration algorithm tailored to this threshold structure to derive the optimal transmission policy. Additionally, we introduce a simpler single-threshold policy, which, despite a slight performance degradation, offers faster convergence. Comprehensive experiments and simulations validate the superior performance of our optimal transmission policy compared to two established baseline approaches.

In a practical Mobile Edge Computing (MEC) system, the stochastic arrival and departure of heterogeneous Mobile Devices (MDs) can cause fluctuations in the number of generated tasks to be scheduled over time, with the generated tasks differing in data size, complexity, and delay constraint. In this work, we consider a dynamic Task Offloading (TO) problem aiming at maximizing the long-term average system utility jointly defined by task completion and energy consumption in the MEC system, where the number of MDs to be served varies over time, and the processing of generated tasks may span multiple time slots. Particularly, we first transform the TO problem into a Markov Decision Process (MDP) with time-varying state and action spaces, which cannot be effectively solved by conventional Deep Reinforcement Learning (DRL) algorithms. Then, we propose a state and action spaces adaptive DRL algorithm to efficiently solve the formulated MDP by leveraging the Transformer model. Finally, simulation results demonstrate the superiority of our proposed algorithm over baseline algorithms and emphasize the limitation of the conventional DRL algorithm in handling time-varying state and action spaces.

The utilization of data buffer in unmanned aerial vehicle (UAV) introduces a dual role in the age of information (AoI) performance. Despite the enhancement of data delivery quality resulting from flexible and efficient data transmission, buffering may also risk increasing AoI by causing data aging. Against this background, this paper investigates the AoI minimization problem in buffer-aided UAV wireless communications while incorporating the effects of UAV buffering dynamics and limited buffer size. Specifically, we formulate the problem as a partially observable Markov decision process (POMDP) and propose a deep recurrent Q-network (DRQN) -based algorithm to jointly optimize UAV trajectory planning and buffering decisions. Simulation results exhibit the superiority of the proposed algorithm in terms of reducing the average AoI. Moreover, the study sheds light on the impact of UAV buffer size on AoI optimization, yielding essential design for buffer-aided UAV communications.

Joint sensing and communication (JSC) systems aim to leverage the same spectral resources for both communication and sensing tasks within a single system. These systems have the potential to enhance sensing capabilities through advanced communication techniques, while also utilizing precise localization and tracking information from sensing technologies to improve communication. However, the integration of information obtained from sensing and transmitted in communication is not yet fully understood. This paper investigates the challenge of guaranteeing timely delivery of sensing information within JSC systems. We introduce a novel metric, termed as the age of estimation information (AoEI), which integrates radar mutual information (MI) and age of information (AoI). This unified metric effectively captures both the passage of time and the accuracy of estimation information, making it well-suited for the JSC system. Further, we delve into the joint optimization of time and power allocation for a single JSC node with both sensing and communication capabilities. Our objective is to minimize the long-term average AoEI while adhering to a long-term average power constraint. To tackle this problem, we formulate it as an average-reward constrained Markov decision process (CMDP) and propose a model-free constrained deep reinforcement learning (CDRL) algorithm, namely the average policy optimization (APO)-Lagrangian based algorithm. Simulation results demonstrate that our proposed algorithm effectively meets the constraint in dynamic and uncertain environments while achieving a favorable balance between AoEI and power consumption. Additionally, our algorithm outperforms four baseline schemes, showcasing its superior performance.

Deep learning (DL) has been widely recognized for its strong feature representation capability, making it a promising technique to improve pansharpening methods. However, existing DL-based methods commonly extract the spectral information and spatial information from the high-resolution panchromatic (PAN) images and low-resolution multispectral (MS) images, respectively. This separation limits the effective extraction and integration of potential spectral-spatial information, ultimately reducing the quality of the generated high-resolution multispectral (HRMS) images. In this paper, we propose a novel spatialspectral dual guided network (SSDGN), aiming to fully capitalize on the spectral and spatial information contained in both the PAN and MS images. Firstly, to enhance feature extraction, we introduce two subnetworks: the progressive spectral feature extraction (PSpeFE) subnetwork and progressive spatial feature extraction (PSpaFE) for spatial features. These extract information from the PAN and MS images. Additionally, features from the frequency domain (FD) and intensity domain (ID) of both image types are leveraged to guide and enhance the efficiency of feature extraction. Then, a joint spatial-spectral attention feature fusion module and a multi-stage residual reconstruction module are devised to efficiently harness the extracted spatial and spectral information. Finally, extensive experiments are conducted to evaluate the performance and effectiveness of the proposed SSDGN. Compared to the second-best methods, our approach achieves an average reduction of 12.3% in ERGAS across three satellite datasets. The QNR metric improves by up to 2.1% and averages a 0.8% gain, demonstrating consistent advantages in spectral fidelity and fusion quality.

The emergence of intelligent applications has fostered the development of a task-oriented communication paradigm, where a comprehensive, universal, and practical metric is crucial for unleashing the potential of this paradigm. To this end, we introduce an innovative metric, the Task-oriented Age of Information (TAoI), to measure whether the content of information is relevant to the system task, thereby assisting the system in efficiently completing designated tasks. Also, we study the TAoI in a remote monitoring system, whose task is to identify target images and transmit them for subsequent analysis. We formulate the dynamic transmission problem as a Semi-Markov Decision Process (SMDP) and transform it into an equivalent Markov Decision Process (MDP) to minimize TAoI and find the optimal transmission policy. Furthermore, we demonstrate that the optimal strategy is a threshold-based policy regarding TAoI and propose a relative value iteration algorithm based on the threshold structure to obtain the optimal transmission policy. Finally, simulation results show the superior performance of the optimal transmission policy compared to the baseline policies.

Available deep learning (DL) based pansharpening methods primarily extract spectral information and spatial information from panchromatic (PAN) and multispectral (MS) images, respectively. This hinders the efficient derivation and exploitation of the potential spectral-spatial information, thereby degrading the quality of generated high-resolution multispectral (HRMS) images. In this paper, we propose a novel multi-stage dual-domain guided attention pansharpening network (MDGAPN), aiming to fully capitalize on the spectral and spatial information contained in both the PAN and MS images. Firstly, to enhance the feature extraction, we introduce two subnetworks: the progressive spectral feature extraction subnetwork (PSPeN) and progressive spatial feature extraction subnetwork (PSPaN), which are devised to extract both spatial and spectral information from the input PAN and MS images. Wherein, the frequency domain (FD) and intensity domain (ID) features of the PAN and MS image are leveraged as the guidance to improve the efficiency of feature extraction. Then, a joint spatial-spectral attention feature fusion module and a multi-stage residual reconstruction module are devised to efficiently harness the extracted spatial and spectral information. Finally, experiments are conducted to assess the effectiveness of our proposed MDGAPN.

In order to improve communication quality in some heavily blocking scenarios, the double-reconfigurable intelligent surface (RIS) is introduced, then there are more channel coefficients to be estimated. To reduce the pilot overhead, an efficient two-phase channel estimation scheme is proposed for passive double-RIS assisted MIMO systems to estimate the dual-reflection cascaded channels. Specifically, RIS 1 operates in a known fixed mode, and RIS 2 operates in a dynamically adjustable mode with the time slot, which can reduce the number of channel coefficients to be estimated. Then, the dual-reflection cascaded channel property is utilized to further reduce pilot overhead. Simulation results illustrate that the proposed scheme significantly reduces the pilot overhead and achieves superior estimation performance simultaneously.

Many real-time applications of the Internet of Things (IoT) need to deal with correlated information generated by multiple sensors. The design of efficient status update strategies that minimize the Age of Correlated Information (AoCI) is a key factor. In this paper, we consider an IoT network consisting of sensors equipped with the energy harvesting (EH) capability. We optimize the average AoCI at the data fusion center (DFC) by appropriately managing the energy harvested by sensors, whose true battery states are unobservable during the decision-making process. Particularly, we first formulate the dynamic status update procedure as a partially observable Markov decision process (POMDP), where the environmental dynamics are unknown to the DFC. In order to address the challenges arising from the causality of energy usage, unknown environmental dynamics, unobservability of sensors' true battery states, and large-scale discrete action space, we devise a deep reinforcement learning (DRL)-based dynamic status update algorithm. The algorithm leverages the advantages of the soft actor-critic and long short-term memory techniques. Meanwhile, it incorporates our proposed action decomposition and mapping mechanism. Extensive simulations are conducted to validate the effectiveness of our proposed algorithm by comparing it with available DRL algorithms for POMDPs.

This paper focuses on the timely delivery of sensing information in a joint sensing and communication (JSC) system to meet the requirements of emerging applications. Specifically, we investigate the time allocation of a single JSC node equipped with both sensing and communication functions to minimize the long-term average age of estimation information (AoEI) while satisfying the long-term average power constraint. The proposed metric, AoEI, combines radar mutual information (MI) and age of information (AoI) to capture both the passage of time and the accuracy of estimation information, making it more suitable for the JSC system. To solve this problem, we formulate the time allocation problem as a constrained Markov decision process (CMDP) and propose a model-free constrained deep reinforcement learning (CDRL) based algorithm. The simulation results demonstrate that the proposed algorithm can achieve a good trade-off between AoEI and power consumption and converge to a policy that satisfies the constraint in a highly dynamic and uncertain environment.

This paper proposes a client selection (CS) method to tackle the communication bottleneck of federated learning (FL) while concurrently coping with FL’s data heterogeneity issue. Specifically, we first analyze the effect of CS in FL and show that FL training can be accelerated by adequately choosing participants to diversify the training dataset in each round of training. Based on this, we lever-age data profiling and determinantal point process (DPP) sampling techniques to develop an algorithm termed Federated Learning with DPP-based Participant Selection (FL-DP3S). This algorithm effectively diversifies the participants’ datasets in each round of training while preserving their data privacy. We conduct extensive experiments to examine the efficacy of our proposed method. The results show that our scheme attains a faster convergence rate, as well as a smaller communication overhead than several baselines.

Mobile edge computing (MEC) is a promising paradigm to meet the quality of service (QoS) requirements of latency-sensitive IoT applications. However, attackers may eavesdrop on the offloading decisions to infer the edge server's (ES's) queue information and users' usage patterns, thereby incurring the pattern privacy (PP) issue. Therefore, we propose an offloading strategy which jointly minimizes the latency, ES's energy consumption, and task dropping rate, while preserving PP. Firstly, we formulate the dynamic computation offloading procedure as a Markov decision process (MDP). Next, we develop a Differential Privacy Deep Q-learning based Offloading (DP-DQO) algorithm to solve this-problem while addressing the PP issue by injecting noise into the generated offloading decisions. This is achieved by modifying the deep Q-network (DQN) with a Function-output Gaussian process mechanism. We provide a theoretical privacy guarantee and a utility guarantee (learning error bound) for the DP-DQO algorithm and finally, conduct simulations to evaluate the performance of our proposed algorithm by comparing it with greedy and DQN-based algorithms.

In this work, we study the age-optimal status update strategy for a large number of sensors in a wireless system with multiple heterogeneous unreliable channels. Particularly, we first formulate the status update procedure as a Restless Multi-Armed Bandit (RMAB) problem so as to minimize the long-term average Age of Information (AoI) cost. Then, a Deep Whittle index-based Q-Network (DWQN) algorithm is devised to solve it, in which an accurate approximation of the Whittle index can be learned. By applying this algorithm, the challenges from both the unknown of the environmental dynamics and large-scale state space can be addressed. Finally, simulations are conducted to validate the effectiveness of our proposed algorithm by comparing it with baseline strategies.

Due to the superior flexibility and extensive coverage, multiple unmanned aerial vehicles (UAVs) cooperation is a promising approach for data collection in improving the information freshness. In this paper, we consider a multi-UAV-assisted Internet of Things (IoT) network, where UAVs are deployed to collect data from sensor nodes (SNs) and transmit data back to the BS via wireless links so as to improve the information freshness, measured by the age of information (AoI). It is of great challenge to achieve effective cooperation under distributed decision-making because of the time-varying and stochasticity of the environment and the limited communication range of UAVs. To address this issue, we formulate the problem of joint trajectory plan, SN scheduling, and transmission scheduling as a decentralized partially observable markov decision process (Dec-POMDP), and develop an adaptive communication based multi-agent deep reinforcement learning (AC-MARL) algorithm to solve it. By applying our proposed AC-MARL algorithm, a more effective cooperation can be achieved by exploiting the benefits of semantic-aware communications among UAVs.

We consider a homogeneous Poisson bipolar network in which the bipoles represent source-destination pairs. The source nodes need to update their destinations about the new status perpetually, and the communications are taken place over a shared spectrum. The common goal of the source nodes is to minimize the network-wide age of information (AoI). We develop a policy by which every source node can adapt its frequency of generating status updates in a local and decentralized manner. At the same time, the network average AoI is minimized by reducing interference amongst transmitters located in geographical proximity. Following this policy, we also derive mathematical expressions to characterize the distribution of the optimal updating rate at each source node, the network average AoI, and the AoI violation probability, i.e., the probability that the AoI of a typical source node exceeds an age threshold. The analytical results are combined with discrete event simulations to provide a detailed evaluation of the performance of the proposed scheme. Particularly, it is shown that our policy is able to adaptively adjust the updating rate of each source node according to the variant of the network topology. In this manner, it is instrumental in decreasing both the network average AoI and AoI violation probability. Additionally, the scheme can maintain the AoI at a low level even when the network grows in size.

The massive-antenna is widely considered as one of key technologies in the 5th generation (5G) networks and beyond. To approach the system capacity with the massive-antenna, it is highly demanded for fundamental research on how to obtain the system capacity of massive-antenna based broadcast channel (BC) with simple closed-form expressions. To address this, our paper focuses on the massive-multiple-input single-output (mMISO) BC transmission, which is a common scenario in cellular networks. With regarding the massive-antenna, a precise approximation of the system capacity is derived with a simple closed-form expression. The simulations validate the correctness of the derived outcomes in mMISO-BC.

Age of Information (AoI), a notion that measures the information freshness, is an essential performance measure for time-critical applications in Internet of Things (IoT). With the surge of computing resources at the IoT devices, it is possible to preprocess the information packets that contain the status update before sending them to the destination so as to alleviate the transmission burden. However, the additional time and energy expenditure induced by computing also make the optimal updating a nontrivial problem. In this article, we consider a time-critical IoT system, where the IoT device is capable of preprocessing the status update before the transmission. Particularly, we aim to jointly design the preprocessing and transmission so that the weighted sum of the average AoI of the destination and the energy consumption of the IoT device is minimized. Due to the heterogeneity in transmission and computation capacities, the durations of distinct actions of the IoT device are nonuniform. Therefore, we formulate the status updating problem as an infinite horizon average cost semi-Markov decision process (SMDP) and then transform it into a discrete-time Markov decision process. We demonstrate that the optimal policy is of threshold type with respect to the AoI. Equipped with this, a structure-aware relative policy iteration algorithm is proposed to obtain the optimal policy of the SMDP. Our analysis shows that preprocessing is more beneficial in regimes of high AoIs, given it can reduce the time required for updates. We further prove the switching structure of the optimal policy in a special scenario, where the status updates are transmitted over a reliable channel and derive the optimal threshold. Finally, simulation results demonstrate the efficacy of preprocessing and show that the proposed policy outperforms two baseline policies.

In Internet of Things (IoT), the freshness of status updates is crucial for mission-critical applications. In this regard, it is suggested to quantify the freshness of updates by using Age of Information (AoI) from the receiver’s perspective. Specifically, the AoI measures the freshness over time. However, the freshness in the content is neglected. In this paper, we introduce an age-based utility, named asAge of Changed Information(AoCI), which captures both the passage of time and the change of information content. By modeling the underlying physical process as a discrete time Markov chain, we investigate the AoCI in a time-slotted status update system, where a sensor samples the physical process and transmits the update packets to the destination. With the aim of minimizing the weighted sum of the AoCI and the update cost, we formulate an infinite horizon average cost Markov Decision Process. We show that the optimal updating policy has a special structure with respect to the AoCI and identify the condition under which the special structure exists. By exploiting the special structure, we provide a low complexity relative policy iteration algorithm that finds the optimal updating policy. We further investigate the optimal policy for two special cases. In the first case where the state of the physical process transits with equiprobability, we show that optimal policy is of threshold type and derive the closed-form of the optimal threshold. We then study a more generalized periodic Markov model of the physical process in the second case. Lastly, simulation results are laid out to exhibit the performance of the optimal updating policy and its superiority over the zero-wait baseline policy.

In this paper, we consider an unmanned aerial vehicles (UAV)-assisted IoT network and study the trajectory planning problem to optimize the information freshness, in terms of age of information (AoI), where the update arrivals at IoT devices are stochastic and are not known to the UAV. To this end, we first formulate the dynamic UAV trajectory planning problem as a Partially Observable Markov Decision Process (POMDP) with non-uniform time steps, where the set of valid actions is coupled with the agent's observations. Then, a deep recurrent reinforcement learning (DRRL) algorithm is devised to find the policy minimizing the expectation of the weighted average AoI, in which a modified discount mechanism is utilized to deal with the challenge from non-uniform time steps and an action elimination mechanism is introduced to address the coupling between the valid actions and observations. Finally, simulations are conducted to validate the effectiveness of our proposed algorithm by comparing it with baseline strategies.

The dual connectivity is emerging as a promising solution to boost capacity in heterogeneous networks. However, it is challenging to obtain an optimal user association in heterogeneous networks with dual connectivity, due to its non-convex and combinatorial nature. In this paper, we propose a user association scheme based on deep reinforcement learning to maximize the overall network utility, which takes both throughput and user fairness into account, in the downlink of a heterogeneous network. Particularly, each user associates with the macro base station (BS) and a micro BS. We apply a deep Q-network (DQN) to obtain the nearly optimal policy to associate the users and micro BSs. Simulation results demonstrate that DQN-based user association performs better compared to the conventional user association schemes in heterogeneous networks with dual connectivity, and it behaves good scalability when the environment changes.

In the Internet of Things (IoT) networks, caching is a promising technique to alleviate energy consumption of sensors by responding to users’ data requests with the data packets cached in the edge caching node (ECN). However, without an efficient status update strategy, the information obtained by users may be stale, which in return would inevitably deteriorate the accuracy and reliability of derived decisions for real-time applications. In this paper, we focus on striking the balance between the information freshness, in terms of age of information (AoI), experienced by users and energy consumed by sensors, by appropriately activating sensors to update their current status. Particularly, we first depict the evolutions of the AoI with each sensor from different users’ perspective with time steps of non-uniform duration, which are determined by both the users’ data requests and the ECN’s status update decision. Then, we formulate a non-uniform time step based dynamic status update optimization problem to minimize the long-term average cost, jointly considering the average AoI and energy consumption. To this end, a Markov Decision Process is formulated and further, a dueling deep R-network based dynamic status update algorithm is devised by combining dueling deep Q-network and tabular R-learning, with which challenges from the curse of dimensionality and unknown of the environmental dynamics can be addressed. Finally, extensive simulations are conducted to validate the effectiveness of our proposed algorithm by comparing it with five baseline deep reinforcement learning algorithms and policies.

The notion of age-of-information (AoI) is investigated in the context of large-scale wireless networks, in which transmitters need to send a sequence of information packets, which are generated as independent Bernoulli processes, to their intended receivers over a shared spectrum. Due to interference, the rate of packet depletion at any given node is entangled with both the spatial configurations, which determine the path loss, and temporal dynamics, which influence the active states, of the other transmitters, resulting in the queues to interact with each other in both space and time over the entire network. To that end, variants in the packet update frequency affect not just the inter-arrival time but also the departure process, and the impact of such phenomena on the AoI is not well understood. In this paper, we establish a theoretical framework to characterize the AoI performance in the aforementioned setting. Particularly, tractable expressions are derived for both the peak and average AoI under two different transmission protocols, namely the first-come-first-serve (FCFS) and the last-come-first-serve with preemption (LCFS-PR). Additionally, our analysis also accounts for the effects of channel access controls such as ALOHA on the AoI. The accuracy of the analysis is verified via simulations, and based on the theoretical outcomes, we find that: i) networks operating under LCFS-PR are able to attain smaller values of peak and average AoI than that under FCFS, whereas the gain is more pronounced when the infrastructure is densely deployed, ii) in sparsely deployed networks, ALOHA with a universally designed channel access probability is not instrumental in reducing the AoI, thus calling for more advanced channel access approaches, and iii) when the infrastructure is densely rolled out, there exists a non-trivial ALOHA channel access probability that minimizes the peak and average AoI under both FCFS and LCFS-PR.

Small cell networks with dynamic time-division duplex (D-TDD) have emerged as a potential solution to address the asymmetric traffic demands in 5G wireless networks. By allowing the dynamic adjustment of cell-specific UL/DL configuration, D-TDD flexibly allocates percentage of subframes to UL and DL transmissions to accommodate the traffic within each cell. However, the unaligned transmissions bring in extra interference which degrades the potential gain achieved by D-TDD. In this work, we propose an analytical framework to study the performance of multi-antenna small cell networks with clustered D-TDD, where cell clustering is employed to mitigate the interference from opposite transmission direction in neighboring cells. With tools from stochastic geometry, we derive explicit expressions and tractable tight upper bounds for success probability and network throughput. The proposed analytical framework allows to quantify the effect of key system parameters, such as UL/DL configuration, cluster size, antenna number, and SINR threshold. Our results show the superiority of the clustered D-TDD over the traditional D-TDD, and reveal the fact that there exists an optimal cluster size for DL performance, while UL performance always benefits from a larger cluster.

Drone small cells (DSCs) are served as aerial base stations to complement the terrestrial cellular networks, in order to provide seamless wireless coverage and increased network capacity. In this paper, we study a drone-assisted downlink heterogeneous network (HetNet) consisting of a first tier of DSCs overlaid with a second tier of terrestrial small cells (TSCs), both of which operate on the same frequency band. By considering a flexible biased association policy, we develop an analytical framework to evaluate the network performance. After deriving the association probabilities, and the probability distribution functions (PDFs) of typical link length, we derive exact expressions for the per-tier and overall coverage probabilities. The proposed framework allows to quantify the impact on network performance of most important system parameters, such as the height of DSCs, the bias factor, and the base station density. In absence of interference management, our results show that very limited gains can be obtained in the dense network scenario and the improper deployment of DSCs can only degrade the coverage probability achieved by the single-tier TSC network. What's more, the unbiased cell association is shown to be optimal for the overall coverage probability in the interference-limited network regime.

The freshness of status updates is imperative in mission-critical Internet of things (IoT) applications. Recently, Age of Information (AoI) has been proposed to measure the freshness of updates at the receiver. However, AoI only characterizes the freshness over time, but ignores the freshness in the content. In this paper, we introduce a new performance metric, Age of Changed Information (AoCI), which captures both the passage of time and the change of information content. Also, we examine the AoCI in a time-slotted status update system, where a sensor samples the physical process and transmits the update packets with a cost. We formulate a Markov Decision Process (MDP) to find the optimal updating policy that minimizes the weighted sum of the AoCI and the update cost. Particularly, in a special case that the physical process is modeled by a two-state discrete time Markov chain with equal transition probability, we show that the optimal policy is of threshold type with respect to the AoCI and derive the closed-form of the threshold. Finally, simulations are conducted to exhibit the performance of the threshold policy and its superiority over the zero-wait baseline policy.

Internet of Things (IoT) has emerged as one of the key features of the next-generation wireless networks, where timely delivery of status update packets is essential for many real-time IoT applications. To provide users with context-aware services and lighten the transmission burden, the raw data usually need to be preprocessed before being transmitted to the destination. However, the effect of computing on the overall information freshness is not well understood. In this article, we first develop an analytical framework to investigate the information freshness, in terms of peak age of information (PAoI), of a computing-enabled IoT system with multiple sensors. Specifically, we model the procedure of computing and transmission as a tandem queue and derive the analytical expressions of the average PAoI for different sensors. Based on the theoretical results, we formulate a min-max optimization problem to minimize the maximum average PAoI of different sensors. We further design a derivative-free algorithm to find the optimal updating frequency, with which the complexity for checking the convexity of the formulated problem or obtaining the derivatives of the object function can be largely reduced. The accuracy of our analysis and the effectiveness of the proposed algorithm are verified with extensive simulation results.

Internet of Things (IoT) has been emerging as one of the use cases permeating our daily lives in 5th Generation wireless networks, where status update packages are usually required to be timely delivered for many IoT based intelligent applications. Enabling the collected raw data to be preprocessed before transmitted to the destination can provide users with better context-aware services and lighten the transmission burden. However, the effect from data preprocessing on the overall information freshness is an essential yet unrevealed issue. In this work we study the joint effect of data preprocessing and transmission procedures on information freshness measured by peak age of information (PAoI). Particularity, we formulate the considered multi-source preprocessing and transmission enabled IoT system as a tandem queue where a priority M/G/1 queue is followed by a G/G/1 queue. Then, we respectively derive the closed-form and an information theoretic approximation of the expectations of waiting time for the formulated processing queue and transmission queue, and further get the analytical expression of the average PAoI for packages from different sources. Finally, the accuracy of our analysis is verified with simulation results.

Caching content in devices and sharing content via device-to- device (D2D) communications can help reduce cellular data traffic. However, the content caching in devices will reshape the way the conventional D2D communications work, which has not been fully understood. Therefore, we explore the coupling relationship of content caching and sharing in this paper. Particularly, we first propose a concept of content-topology and give its corresponding graph model, which reveals the relationship among devices, content caching, and D2D links. Subsequently, a heuristic algorithm is proposed to find a content-topology, where the content caching among devices and the link activations for content sharing are well matched. Simulation results show that the scheme based on content- topology outperforms the existing algorithms in terms of the amount of offloaded traffic, the number of link activation, and link efficiency.

Topology control is an important approach to maintain the connectivity of cognitive radio networks (CRNs). Most existing works assumed that secondary users (SUs) must vacate the spectrum reclaimed by primary users (PUs), resulting the inefficient spectrum utilization. In this paper, we consider the simultaneous transmissions of SUs with PUs; meanwhile, SUs are equipped with successive interference cancellation (SIC) to mitigate the interference from PUs, thereby enabling SUs to access the spectrum more aggressively than previous works. Although SIC has been studied in the information theory and signal processing, it is not well investigated in guaranteeing the connectivity of wireless networks, especially the CRNs. On this account, we propose both centralized and distributed SIC-based topology control algorithm to alleviate the impact of the unpredictable activities of PUs and the potential interference between SUs. In particular, we integrate power control with channel assignment to construct a bi-channel-connected and conflict-free CRN with the fewest required channels. Theoretical analysis reveals that the bi-channel-connectivity and conflict-free properties can be ensured by our proposed algorithms. Then, simulation results demonstrate the effectiveness of proposed algorithms in terms of reducing the number of required channels and improving the robustness of topologies, as compared with the prevailing topology control algorithms.

Content caching at base stations (BSs) is a promising technique for future wireless networks by reducing network traffic and alleviating server bottleneck. However, in practice, the content popularity distribution may change with spatio-temporal variation but be unknown for BSs, which is an intractable obstacle for efficient caching strategy design. In this paper, considering unknown popularity distribution, we explore the content caching problem by jointly optimizing the content caching in cooperative BSs, content sharing among BSs, and cost of content retrieving. We tackle the problem from a multi-armed bandit learning perspective, where the learning of the popularity distribution is incorporated with the content caching and sharing process. Specifically, we first propose a centralized algorithm by employing a semidefinite relaxation approach, and we prove that this centralized algorithm learns efficient caching by deriving a sub-linear learning regret bound. To further reduce computational complexity, we propose a distributed algorithm based on alternating direction method of multipliers, where each BS only solves their own problems by exchanging local information with neighbor BSs. Extensive simulation results show the effectiveness of the proposed algorithms in terms of learning content popularity distributions of individual BSs, offloading traffic from the content server, and reducing cost of content retrieving.

Small satellite networks (SSNs) are playing an increasing role in nowadays earth observation due to their less development cost and energy consumption. In SSNs, it is pivotal to transmit a huge amount of data for differentiated missions to ground stations. Nevertheless, due to limited transponders and energy budget, not all contacts, i.e., potential available communication links, are feasible in data delivery. Besides, satellite downlink channel conditions are indeed time-varying due to atmospheric precipitation. Therefore, one daunting challenge is searching for feasible contacts termed as contact plan design with consideration of the differentiation for missions. In this paper, we exploit an extended time-evolving graph to characterize network resources. Based on the graph, we formulate the design of mission-aware contact plan, aiming at maximizing network profit in terms of sum weighted data volume as a mixed-integer linear programming. Due to its NP-hardness, we propose a primal decomposition method to efficiently solve the formulated problem by exploiting its special structure. To further reduce the complexity, we propose a link metric considering the issues of residual energy of satellites, time-varying satellite downlink contact capacity, and the differentiation for missions in the conflict graph. Based on the conflict graph, we devise a heuristic algorithm to design contact plan. Simulation results demonstrate the efficiency of the proposed algorithms and necessitate the consideration of the time-varying downlinks and the differentiation of missions for contact plan design.

In this paper, we investigate the energy-saving resource management problem for a new device-to-device (D2D) and cellular coexisting network, where D2D users employ orthogonal frequency division multiple access (OFDMA) and cellular users employ sparse code multiple access (SCMA). This hybrid network can support massive connectivity by exploiting the degrees of freedom in code and space domains, however, the complicated spectrum sharing pattern also leads to serious interference, which further boosts the power consumption of mobile devices (MDs). To tackle this problem, we propose a unified resource management scheme to minimize the total transmit power of all MDs by jointly optimizing mode selection, resource allocation, and power control. First, we analytically get the optimal resource-sharing mode (dedicated mode or reuse mode) for cellular users and D2D users based on the mapping rule between SCMA codebooks and OFDMA resource blocks. For each resource-sharing mode, we reformulate the resource management problems as classical problems in graph theory, and then devise efficient algorithms leveraging the special structure of the constructed graphs. Finally, simulation studies indicate that the network capacity is upgraded with the hybrid multiple access technologies, and the energy efficiency performance is also enhanced through the unified resource management.

Small satellite networks, with the advantage of remarkably less development cost and energy consumption with respect to geostationary relay platforms, are playing an increased role in nowadays earth observation. However, small satellites have limited transponders and energy budget, which makes it necessary to design efficient contact plans to improve the network throughput. This paper addresses such an issue of joint management of the energy and transponder resource to well match the mission demand and network resources. We adopt an extended time-evolving graph to characterize network resources and then, formulate the contact plan design problem with the goal of maximizing the throughput as a mixed-integer linear programming. Since the computational complexity of this problem coupling multiple time slots is prohibitive, we further propose two heuristic algorithms which operate on a slot-by-slot basis to achieve high throughput. Simulation results present the impact of different factors on the network performance and moreover, demonstrate that both our contact plan approaches can achieve high throughput with low complexity.

In energy-constrained Low Earth Orbit (LEO) satellite constellations, in order to prolong the network lifetime, more traffic should be carried by the satellites with high battery level, which, in turn, may result in congestion in such satellites. To strike a balance, we study the multi-path routing problem which aims at Maximizing network Lifetime while maintaining a Guaranteed network Congestion level (MLGC). Particularly, we formulate such a problem as a linear programming. However, it is time-consuming that solving the proposed MLGC needs to joint multiple time intervals. Therefore, we further design an Energy Aware Multi-path Routing (EAMR) strategy without solving the optimization problem. Simulation results show that the performance of EAMR is comparable with MLGC and moreover, compared with available routing strategies, the network lifetime can be effectively improved while the required congestion level being guaranteed by implementing our proposed schemes.

In this paper, we address the problem of power consumption minimization in Heterogeneous networks (HetNets) of China Mobile by implementing dynamic base stations (BSs) switching operation. Particularly, considering the minimal rate requirements of users as well as the realistic power consumption model of BSs, we formulate the problem as an integer linear programming which is NP-complete. Then, to efficiently solve this problem, the Power Efficient Base Station Operation with User Association scheme (PEBUA) and Adaptive Multi-cell Coordination Algorithm for Energy Saving (AMCES) have been developed, which can be efficiently adopted in different network conditions, respectively. Simulation results verify the validity of our analysis and additionally, compared with the existing method in practice network operation, show the effectiveness of the proposed schemes. It should be noted that both of the proposed algorithms can be applied in real HetNets to save the overall power consumption without decreasing users' traffic rates, which makes our study more feasible.

The advent of simultaneous wireless information and power transfer (SWIPT) offers a promising approach to providing cost-effective and perpetual power supplies for energy-constrained mobile devices in heterogeneous cellular networks (HCNs). As energy efficiency (EE) has been envisioned as a key performance metric in 5G wireless networks, we consider a multiple-input single-output (MISO) femtocell cochannel overlaid with a Macrocell to exploit the advantages of SWIPT while promoting the EE. The femto base station sends information to information decoding (ID) femto users (FUs) and transfers energy to energy harvesting (EH) FUs simultaneously, and also suppresses its interference to Macro users. We maximize the information transmission efficiency (ITE) of ID FUs and energy harvesting efficiency (EHE) of EH FUs, respectively, with the QoS of all users, and investigate their relationship. We formulate these problems as fractional programming, which are nontrivial to solve due to the nonconvexity of ITE and EHE. To tackle these problems, we devise two beamformers namely zero-forcing (ZF) and mixed beamforming (MBF), and then propose an efficient algorithm to obtain the optimal power under both beamformers. Simulation results demonstrate that MBF provides better ITE and EHE than ZF, and there exists a tradeoff between ITE and EHE in general.

In this paper, we study the inter-linked problems of wireless service provider (WSP) selection of users and bandwidth allocation of WSPs in multi-tier heterogeneous cellular networks employing the approach combining stochastic geometry and game theory. In particular, the expected average user achievable rate is calculated by modeling the distributions of users and base stations (BSs) as independent homogeneous Poisson point processes. Moreover, a hierarchical game framework is presented to model the complicated interactions among users and WSPs. Wherein, the evolutionary game, non-cooperative game, and multi-leader multi-follower Stackelberg game models are, respectively, adopted to formulate the competition among users, competition among WSPs, and cyclic dependence between users and WSPs. According to backward induction, the formulated Stackelberg game would be solved after the formulated evolutionary game and non-cooperative game are sequentially studied. For the evolutionary game, both the closed-form expression and the asymptotically stability of its evolutionary equilibrium (EE) were analyzed. Then, conditioned on the obtained EE, the existence of Nash equilibrium (NE) for the non-cooperative bandwidth allocation game is established; furthermore, a sufficient condition for the uniqueness of the NE is derived. Finally, extensive simulation results verify both the validity of our analysis and the effectiveness of the proposed scheme.

In this work, the inter-linked problems of bandwidth allocation for wireless service providers (WSPs) and WSP selection for users in heterogeneous cellular networks (HCNs) are addressed by employing a multi-hierarchical game framework. Wherein, while the interaction between users are modelled using evolutionary game theory, the interactions between competing WSP's are modelled as a non-cooperative spectrum bandwidth allocation game (N-BAG). Moreover, the interaction between the WSPs and users is modelled as a multi- leader multi-follower Stackelberg game. After that, for the formulated evolutionary game, the existence and uniqueness of the evolutionary equilibrium (EE) was investigated. Conditioned on the obtained EE, the existence of a Nash equilibrium (NE) for the proposed N-BAG has been further proven and an offline algorithm to achieve the equilibrium state was proposed. Finally, simulation results verify the validity of our analysis and demonstrate that a unique NE would be achieved by the HCNs adopting the developed algorithm.

Energy efficiency (EE) is very crucial for future wireless communication systems, especially for cognitive radio networks (CRNs). The EE performance relies on channel state information (CSI) of channels. Besides, the interference from secondary users (SUs) to primary users (PUs) also closely depends on CSI in underlay CRNs. However, available works on EE usually assume that CSI is perfect, which is often inaccurate in practical systems. Thus, in this paper we investigate the robust EE maximization problem in underlay CRNs with multiple SUs and PUs. Assuming CSI error to be bounded, we consider that all channels lie in some bounded uncertainty regions. From the perspective of worst-case optimization, we formulate it as the max-min problem with infinite constraint, which is nontrivial even without this constraint. This is because that the outer-maximization problem is non-convex and the inner-minimization problem is a concave minimization problem known as NP-hard in general. We propose a scheme to handle this problem via the fractional programming and global optimization techniques. Particularly, we efficiently solve this problem in two special cases. Simulation results validate that our proposed scheme can improve the worst-case EE of SUs distinctly and strictly guarantee the quality-of-service (QoS) of PUs under all parameters' uncertainties.

The architecture of decentralization makes future wireless networks more flexible and scalable. However, due to the lack of the central authority (e.g., BS or AP), the limitation of spectrum resource, and the coupling among different users, designing efficient resource allocation strategies for decentralized networks faces a great challenge. In this paper, we address the distributed channel selection and power control problem for a decentralized network consisting of multiple users, i.e., transmit-receiver pairs. Particularly, we first take the users' interactions into account and formulate the distributed resource allocation problem as a non-cooperative transmission control game (NTCG). Then, a utility-based transmission control algorithm (UTC) is developed based on the formulated game. Our proposed algorithm is completely distributed as there is no information exchange among different users and hence, is especially appropriate for this decentralized network. Furthermore, we prove that the global optimal solution can be asymptotically obtained with the devised algorithm, and more importantly, in contrast to existing utility-based algorithms, our method does not require that the converging point is one Nash equilibrium (NE) of the formulated game. In this light, our algorithm can be adopted to achieve efficient resource allocation in more general use cases.

We investigate the interactive behavior and strategic decision-making between multiple secondary users (SUs) and primary users (PUs), both of which are end-to-end performance aware in cognitive networks. A Stackelberg game is utilized to formulate the spectrum utilization maximization problem after the complex interference situation is analyzed. Especially, an interference power cap (IPC) function predefined by PUs as a pricing function is introduced into the utility function design of SUs to guarantee QoS of PUs, as well as to decouple constraints. Further, an asymmetric information situation can be formed by considering PUs as leaders who employ the optimal water-filling algorithm, and the closed-form power strategy of SUs can be derived. And accordingly, SUs as followers can observe the available information to do more foresighted decision by learning. What is more, we prove the optimality and existence of the deceived solutions. Numerical results demonstrate that the proposed distributed algorithm provides more spectrum revenue and better QoS guarantees to PUs with limited iterations.