In this paper, a novel joint energy and age of information (AoI) optimization framework for IoT devices in a non-stationary environment is presented. In particular, IoT devices that are distributed in the real-world are required to efficiently utilize their computing resources so as to balance the freshness of their data and their energy consumption. To optimize the performance of IoT devices in such a dynamic setting, a novel lifelong reinforcement learning (RL) solution that enables IoT devices to continuously adapt their policies to each newly encountered environment is proposed. Given that IoT devices have limited energy and computing resources, an unmanned aerial vehicle (UAV) is leveraged to visit the IoT devices and update the policy of each device sequentially. As such, the UAV is exploited as a mobile learning agent that can learn a shared knowledge base with a feature base in its training phase, and feature sets of a zero-shot learning method in its testing phase, to generalize between the environments. To optimize the trajectory and flying velocity of the UAV, an actor-critic network is leveraged so as to minimize the UAV energy consumption. Simulation results show that the proposed lifelong RL solution can outperform the state-of-art benchmarks by enhancing the balanced cost of IoT devices by 8.3% when incorporating warm-start policies for unseen environments. In addition, our solution achieves up to 49.38% reduction in terms of energy consumption by the UAV in comparison to the random flying strategy.

Timely delivery of fresh data/information is fundamental and critical in the Industrial Internet of Things (1IoT) powered various applications, which face the challenges of how to meet the dynamically changing requirement of heterogeneous information urgency among different moving devices under time-varying channels. In this paper, we investigate a UAV-enabled mobile edge computing IloT system. To combat the above challenge, we firstly design a new metric, namely alterable Urgency of Information (aUol) to quantify the changeable heterogeneous urgency of information across various devices since the conventional Age of Information (Aol) is incapable. Further, we exploit a deep reinforcement learning algorithm to optimize the aU 01 of the system by adaptively determining the optimal user scheduling and UAV trajectory planning strategy. Extensive simulation results demonstrate that the proposed method can effectively reduce the aUol of system and improve heterogeneous information urgency service satisfaction rate by 42.5 % to 87.5 % as compared to other benchmark approaches. Additionally, the proposed approach is more effective than the conventional Aol-oriented method.

Proactive Mobile Network (PMN) has been proposed to support extremely low latency communications with multi-tier computing architectures, machine-centricity, and data-driven operation features. Nevertheless, the communication reliability of the PMN introduces new substantial technological challenges. As PMN employs unique proactive open-loop communication, any feedback-based control is avoided to enhance end-to-end latency. This paper focuses on machine-initiated uplink transmission in PMN and proposes a reliability-guaranteed resource management scheme. Without requiring feedback control information, our scheme uniquely decomposes conventional resource management into two collaborative decision processes: predictive resource allocation suggested by network anchor nodes (ANs) and proactive smart resource utilization by smart equipment (SE). These two decision-making processes are constructed as independent reinforcement learning (RL) problems, but implicitly share the states of radio resources according to operating environments. Different algorithms for different operating scenarios have been investigated for this dual-decision solution. Simulation results in various scenarios show that our scheme enables PMN’s reliability close to the theoretical optimal value and successfully serves radio resource utilization in the PMNs.

In the emerging Internet-of-Things (IoT) paradigm, the freshness of sensory information plays a crucial role in online data-analyzing and application-level decision-making. As the tailor-made performance metric of information freshness, the age of information (AoI) depends on the data transmission efficiency and data update frequency, which are energy demanding for IoT devices with limited battery capacity. To alleviate the energy constraints of low-power IoT devices, we propose an AoI-oriented unmanned aerial vehicle (UAVs)-enabled wireless power transmission scheme, where UAVs are deployed to wirelessly charge IoT devices. With the harvested energy, the devices will upload their fresh information to UAVs. The proposed system aims for sustainable IoT networks with practical device-specific energy limitation, which has been long neglected by existing AoI optimization works. In addition, to explore the influence of dynamic time-varying channels on AoI, a practical line-of-sight (LoS)/NLoS channel model is established to accurately depict the dynamic channel characteristics and precisely capture the efficiency of both data transmission and energy harvesting. To achieve the optimal system-level AoI under dynamic channel conditions, a novel deep reinforcement learning-based proactive UAV trajectory planning (PUTP) algorithm is proposed to automatically adjust the UAV fight policy according to the channel variations and the trade-off between the energy transmission and data collection. Extensive simulation results demonstrate that the proposed PUPT algorithm can significantly reduce the AoI by approximately 20% to 65% compared to three other existing trajectory planning algorithms.

In this paper, a lifelong learning problem is studied for an Internet of Things (IoT) system. In the considered model, each IoT device aims to balance its information freshness and energy consumption tradeoff by controlling its computational resource allocation at each time slot under dynamic environments. An unmanned aerial vehicle (UAV) is deployed as a flying base station so as to enable the IoT devices to adapt to novel environments. To this end, a new lifelong reinforcement learning algorithm, used by the UAV, is proposed in order to adapt the operation of the devices at each visit by the UAV. By using the experience from previously visited devices and environments, the UAV can help devices adapt faster to future states of their environment. To do so, a knowledge base shared by all devices is maintained at the UAV. Simulation results show that the proposed algorithm can converge 25% to 50% faster than a policy gradient baseline algorithm that optimizes each device’s decision making problem in isolation.