Cutting-edge Large Language Models (LLMs) play a crucial role in improving autonomous navigation by offering efficient solutions. While LLMs require powerful computers to operate, security concerns and maintaining a stable connection with the cloud can be challenging due to various factors. To address this issue, we propose a metareasoning approach for edge-cloud collaborative LLM planning which leads to an efficient autonomous navigation. The proposed approach allows the system to seamlessly switch between cloud and edge devices to fulfill the mission even in the event of a lost connection or entering a GPS-denied environment. Moreover, we deploy state-of-the-art LLM models on resource-constrained systems like the NVIDIA Jetson Orin Nano 8GB, integrated with ROSMASTER X3. These LLMs have demonstrated exceptional utility in dynamic planning for multi-room or maze environments. A comprehensive LLM profiling of TinyLLM models was performed for five different LLMs. The LLM profiling result shows that while certain models with smaller sizes and lower power consumption were available, their accuracy was insufficient for our application requirements. As a result, LLaMa2-7B is considered the edge LLM model due to its optimal balance of performance and accuracy. The experimental results show that under weak signal conditions (<-50 dB), the metareasoning approach improves energy consumption by up to 4x while the cloud-based implementation exceeds the energy consumption of the onboard LLM implementation. Moreover, with delays of 10-20 seconds, cloud implementation becomes impractical for real-time applications in weak signal environments. This underscores the need for metareasoning, which optimizes energy consumption and response time, providing a balanced solution by adapting to signal strength. A real-world implementation of the proposed approach on ROSMASTER X3 with NVIDIA Jetson Orin Nano board can be found in this video which shows that the mission was completed despite losing the connection with cloud-based LLM.

The safety, energy efficiency, and small size of smart drones have led to the broad use of autonomous Unmanned Aerial Vehicles (UAVs) across various applications, creating opportunities for human-machine collaboration. Machine Learning (ML) algorithms like Neural Networks (NNs) offer promising solutions for vision-based navigation and autonomous systems. However, these algorithms are computationally intensive, making it challenging to deploy them on robots expanded by resource-constrained edge devices with limited computational power and low energy consumption requirements. In this paper, we propose an Energy-Efficient Framework for Video Streaming and Augmented Reality called E2AR to enable ML-based multi-edge device video streaming to a AR device while applying augmented reality to enhance human-machine teaming. For this aim, a YOLO is deployed on the edge device for energy-efficient computation and higher performance. Moreover, video streaming to HoloLens is optimized to improve communication latency and power consumption. To evaluate the proposed method, we implemented it on edge devices such as Crazyflie drone while streaming video to the HoloLens. Crazyflie drones with LiDAR sensors and the GAP8 processor has consisted of an octa-core RISC-V. We measured the power consumption, latency, and core usage of the GAP8 processor while implementing the proposed approach. View a video demonstration of the E2AR concept at: Video.

Safety, low-cost, small size, and Artificial Intelli-gence (AI) capabilities of drones have led to the proliferation of autonomous tiny Unmanned Aerial Vehicles (UAVs) in many applications which are dangerous, unknown, or time-consuming for humans. Deep Neural Networks (DNNs) have enabled au-tonomous navigation while using captured data by drone sensors as input to the model. Due to the extreme complexity of DNNs, cloud-based approaches have been highly addressed in which a drone is connected to the cloud and sends the data to the cloud, and takes the result. On the other hand, emerging tiny machine learning models and edge computing brings significant improvement in energy efficiency and latency with respect to cloud-based approaches. However, there is a trade-off in these two implementations for model accuracy, latency, and energy efficiency. For instance, applying tiny machine learning models leads to lower latency but it sacrifices model accuracy in comparison to cloud-based computing. To address these challenges, we consider multiple models and introduce a new approach named MLAE2 which applies Metareasoning approach for Latency-Aware Energy-Efficient autonomous drones. Metareasoning mon-itors parameters such as latency and energy consumption for different algorithms and chooses the appropriate algorithm due to the environmental situation changes. To Evaluate our approach we extract the power consumption and latency for both cloud-based computing and edge computing while deploying multiple models on a tiny drone named Crazyflie. The experimental results show that MLAE2 successfully meets the latency constraint while maximizing model accuracy and improving energy efficiency.

Artificial Intelligence (AI) and Deep Neural Networks (DNNs) have attracted attention as a solution within autonomous systems fields as they enable applications such as visual perception and navigation. Although cloud-based approaches have already been highly addressed, there is a growing interest in using both AI and DNNs on the edge as this allows for lower latency and avoids the potential security concerns of transmitting data to a remote server. However, deploying DNNs on edge devices is challenging due to the limited computational power available, as well as energy efficiency being of the utmost importance. In this work, we introduce an approach named E2EdgeAI for Energy-Efficient Edge computing that takes advantage of AI for autonomous tiny drones. This approach optimizes the energy efficiency of DNNs by considering the effects of memory access and core utilization on the energy consumption of tiny UAVs. To perform the experiment, we used a tiny drone named Crazyflie with the AI -deck expansion, which includes an octa-core RISC-V processor. The experimental results show the proposed approach reduces the model size by up to 14.4x, improves energy per inference by 78%, and increases energy efficiency by 5.6x. A recorded video for the proposed approach can be found here: Video.