Cellular-connected unmanned aerial vehicles (UAVs) play essential roles across various smart-city applications in low-altitude domains, such as logistics delivery, environmental monitoring, etc. To ensure their reliable deployment, it significantly depends on efficient and intelligent UAV coverage-aware navigation, where deep reinforcement learning (DRL) has emerged as a promising method. However, current DRL-based methods typically simplify UAV flight dynamics by action discretization, which neglects realistic continuous flight dynamic and limits their practical implementation. Moreover, they suffer from the sample inefficiency, since learning the navigation policy requires extensive and costly real-time interactions with the environment. To overcome these two challenges, we propose a novel offline-online radio map-enhanced soft actor-critic (OORM-SAC) framework for the energy-efficient UAV coverage-aware navigation under continuous dynamics constraints. Specifically, our proposed OORM-SAC leverages the classic SAC algorithm to handle large continuous action spaces. It aims to learn continuous steering control to navigate toward the destination while minimizing energy consumption and communication outage. Moreover, to enhance learning efficiency, OORM-SAC adopts a hybrid offline-online learning approach, where the energy-efficient flight pattern is first pre-trained offline and the policy is then refined through online environmental interactions. Furthermore, it incorporates the radio map construction during the online phase to generate diverse virtual training samples, which further accelerates the policy learning. Experimental results demonstrate the effectiveness of OORM-SAC in navigation tasks under continuous dynamic constraints. The OORM-SAC method exhibits superior learning efficiency and generates intelligent trajectories that effectively balance energy consumption and communication requirements.

Traditional navigation systems often overlook dynamic social contexts, affecting users' sense of safety and comfort in unfamiliar settings. This challenge requires integrating personalized real-time social data into route planning on resource-constrained mobile and wearable platforms. This paper introduces AuraNav, enhancing navigation by incorporating real-time recognition and individualized social patterns. AuraNav includes two main modules: (i) Social Topology Enhanced Navigation uses real-time familiar individual detection to create dynamic safety corridors by adjusting path costs with an influence field-based metric in A* search; and (ii) Personalized Path Memory mines long-term familiarity data to build heatmaps, offering route recommendations aligned with user habits and comfort zones. This system uses a modular edge-cloud architecture for low-latency sensor data processing and scalable analysis of social-spatial information. The evaluation shows that AuraNav improves routing safety by 17-20% and reduces travel time by 5%, maintaining immediate responsiveness with negligible latency. AuraNav offers a framework for socially aware navigation systems that evolve into personalized and context-aware systems.

Cellular-connected autonomous aerial vehicles (AAVs) act as aerial users to accomplish navigation missions while maintaining a cellular connection, which are of great importance for developing low-altitude Internet of Things (IoT) systems. However, the full aerial cellular coverage is an extreme challenge. To tackle this issue, existing coverage-aware navigation studies focus on designing AAV flight trajectories to avoid weak coverage areas while completing the mission, where considerable efforts of deep-reinforcement-learning-(DRL)-based methods have been conducted. However, they lack adaptability to various targets, which limits their practical applicability in real-world scenarios. In this article, we propose a novel DRL-based multitarget coverage-aware navigation (MTCN) framework, where the AAV learns a navigation policy to minimize the weighted sum of navigation time and the expected communication outage duration. MTCN allows the AAV to efficiently navigate to arbitrary targets without the need of the separate policy training for each target. Furthermore, to improve the sample efficiency of the MTCN, we introduce the radio map-enhanced MTCN (RM-MTCN) framework, which leverages a virtual radio map to generate supplementary navigation training data. RM-MTCN can significantly reduce the need for extensive real-world interactions. Extensive experimental results demonstrate that both MTCN and RM-MTCN achieve generalized navigation behaviors across different targets while avoiding weak communication coverage areas. It is also shown that RM-MTCN exhibits faster learning speed and better performance compared to MTCN, which indicates the effectiveness of leveraging the virtual radio map during the policy training.

The rapid development of the autonomous aerial vehicle (AAV) technology has significantly facilitated its application in the burgeoning Internet of Things (IoT) ecosystem. AAV visual navigation has emerged as a particularly vibrant and crucial research area, holding the potential to greatly enhance automated IoT services. deep reinforcement learning (DRL), recognized as an effective method for visual navigation, encounters two significant challenges in the complex outdoor environments, i.e., the partial observability and the local optima trapping. In this article, to address these two challenges, we propose a novel semantic map-based cognitive escape reinforcement learning (SM-CERL) navigation method, which consists of two innovatively designed modules, namely the semantic mapping module (SMM) and the cognitive escape module (CEM). By deeply exploring the similarity structure of raw images and mapping them to the advanced semantic representations, the SMM constructs a semantic map with rich implications, which provides the AAV with the memory to enhance its understanding of the environment. Meanwhile, the CEM can proactively identify local optima and leverage the rule knowledge to establish efficient escape strategies. The holistic fusion of semantic mapping and cognitive escape mechanisms efficiently enhances the environmental comprehension and prevents AAVs from being trapped in local optima. Extensive experimental results demonstrate that our SM-CERL outperforms the existing classical and state-of-the-art methods in terms of the navigation accuracy and efficiency.