Nano-sized unmanned aerial vehicles (UAVs) are well-fit for indoor applications and for close proximity to humans. To enable autonomy, the nano-UAV must be able to self-localize in its operating environment. This is a particularly-challenging task due to the limited sensing and compute resources on board. This work presents an online and onboard approach for localization in floor plans annotated with semantic information. Unlike sensor-based maps, floor plans are readily-available, and do not increase the cost and time of deployment. To overcome the difficulty of localizing in sparse maps, the proposed approach fuses geometric information from miniaturized time-of-flight sensors and semantic cues. The semantic information is extracted from images by deploying a state-of-the-art object detection model on a high-performance multi-core microcontroller onboard the drone, consuming only 2.5mJ per frame and executing in 38ms. In our evaluation, we globally localize in a real-world office environment, achieving 90% success rate. We also release an open-source implementation of our work1.

Autonomously navigating robots need to perceive and interpret their surroundings. Currently, cameras are among the most used sensors due to their high resolution and frame rates at relatively low energy consumption and cost. In recent years, cutting-edge sensors, such as miniaturized depth cameras, have demonstrated strong potential, specifically for nano-size unmanned aerial vehicles (UAVs), where low power consumption, lightweight hardware, and low computational demand are essential. However, cameras are limited to working under good lighting conditions, while depth cameras have a limited range. To maximize robustness, we propose to fuse a millimeter form factor 64 pixel depth sensor and a low-resolution grayscale camera. In this work, a nano-UAV learns to detect and fly through a gate with a lightweight autonomous navigation system based on two tinyML convolutional neural network models trained in simulation, running entirely onboard in 7.6 ms and with an accuracy above 91%. Field tests are based on the Crazyflie 2.1, featuring a total mass of 39 g. We demonstrate the robustness and potential of our navigation policy in multiple application scenarios, with a failure probability down to 1.2 ˙ 10-3 crash/meter, experiencing only two crashes on a cumulative flight distance of 1.7 km.

Nano-size unmanned aerial vehicles (UAVs) hold enormous potential to perform autonomous operations in complex environments, such as inspection, monitoring or data collection. Moreover, their small size allows safe operation close to humans and agile flight. An important part of autonomous flight is localization, a computationally intensive task, especially on a nano-UAV that usually has strong constraints in sensing, processing and memory. This work presents a real-time localization approach with low-element-count multizone range sensors for resource-constrained nano-UAVs. The proposed approach is based on a novel miniature 64-zone time-of-flight sensor from STMicroelectronics and a RISC-V-based parallel ultra-low-power processor to enable accurate and low latency Monte Carlo Localization on-board. Experimental evaluation using a nano-UAV open platform demonstrated that the proposed solution is capable of localizing on a 31.2 m2map with 0.15 m accuracy and an above 95% success rate. The achieved accuracy is sufficient for localization in common indoor environments. We analyze trade-offs in using full and half-precision floating point numbers as well as a quantized map and evaluate the accuracy and memory footprint across the design space. Experimental evaluation shows that parallelizing the execution for 8 RISC-V cores brings a 7x speedup and allows us to execute the algorithm onboard in real-time with a latency of 0.2-30 ms (depending on the number of particles), while only increasing the overall drone power consumption by 3–7%. Finally, we provide an open-source implementation of our approach.

Compression algorithms have side channels due to their data-dependent operations. So far, only the compression-ratio side channel was exploited, e.g., the compressed data size.In this paper, we present Decomp+Time, the first memory-compression attack exploiting a timing side channel in compression algorithms. While Decomp+Time affects a much broader set of applications than prior work. A key challenge is precisely crafting attacker-controlled compression payloads to enable the attack with sufficient resolution. Our evolutionary fuzzer, Comprezzor, finds effective Decomp+Time payloads that optimize latency differences such that decompression timing can even be exploited in remote attacks. Decomp+Time has a capacity of 9.73 kB/s locally, and 10.72 bit/min across the internet (14 hops). Using Comprezzor, we develop attacks that leak data bytewise in four different case studies: First, we leak 1.50 bit/min from Memcached on a remote PHP script. Second, we leak database records with 2.69 bit/min, from PostgreSQL in a Python-Flask application, over the internet. Third, we leak secrets with 49.14 bit/min locally from ZRAM-compressed pages on Linux. Fourth, we leak internal heap pointers from the V8 engine within the Google Chrome browser on a system using ZRAM. Thus, it is important to re-evaluate the use of compression on sensitive data even if the application is only reachable via a remote interface.

Nanosize drones hold enormous potential to explore unknown and complex environments. Their small size makes them agile and safe for operation close to humans and allows them to navigate through narrow spaces. However, their tiny size and payload restrict the possibilities for onboard computation and sensing, making fully autonomous flight extremely challenging. The first step toward full autonomy is reliable obstacle avoidance, which has proven to be challenging by itself in a generic indoor environment. Current approaches utilize vision-based or 1-D sensors to support nanodrone perception algorithms. This article presents a lightweight obstacle avoidance system based on a novel millimeter form factor 64 pixels multizone time-of-flight (ToF) sensor and a generalized model-free control policy. In-field tests are based on the Crazyflie 2.1, extended by a custom multizone ToF deck, featuring a total flight mass of 35 g. The algorithm only uses 0.3% of the onboard processing power (${210}\,{\mu }\mathrm{{s}}$execution time) with a frame rate of 15 f/s. The presented autonomous nanosize drone reaches 100% reliability at 0.5 m/s in a generic and previously unexplored indoor environment.

Unmanned aerial vehicles (UAVs) are a very active research topic, and especially the nano and micro subclass, characterized by cen-timeter size and minimal on-board computational capabilities, have gained popularity in recent years. These lightweight platforms provide good agility and movement freedom in indoor environments, but it is still a significant challenge to enable autonomous navigation or basic obstacle avoidance capabilities using standard image sensors, due to the limited computational capabilities that can be hosted on-board. This work demonstrates the possibility of using a new multi-zone Time of Flight (ToF) sensor to enhance autonomous navigation with a significantly lower computational load than most common visual-based solutions. Our system proved reliable (>95%) in-field obstacle avoidance capabilities when flying in indoor environments with dynamic obstacles.

Many emerging applications of nano-sized unmanned aerial vehicles (UAVs), with a few cm2form-factor, revolve around safely interacting with humans in complex scenarios, for example, monitoring their activities or looking after people needing care. Such sophisticated autonomous functionality must be achieved while dealing with severe constraints in payload, battery, and power budget (~100mW). In this work, we attack a complex task going from perception to control: to estimate and maintain the nano-UAV’s relative 3-D pose with respect to a person while they freely move in the environment—a task that, to the best of our knowledge, has never previously been targeted with fully onboard computation on a nano-sized UAV. Our approach is centered around a novel vision-based deep neural network (DNN), called Frontnet, designed for deployment on top of a parallel ultra-low power (PULP) processor aboard a nano-UAV. We present a vertically integrated approach starting from the DNN model design, training, and dataset augmentation down to 8-bit quantization and deployment in-field. PULP-Frontnet can operate in real-time (up to135 frame/s), consuming less than87 mWfor processing at peak throughput and down to0.43 mJ/framein the most energy-efficient operating point. Field experiments demonstrate a closed-loop top-notch autonomous navigation capability, with a tiny 27-g Crazyflie 2.1 nano-UAV. Compared against an ideal sensing setup, onboard pose inference yields excellent drone behavior in terms of median absolute errors, such as positional (onboard:41cm, ideal:26 cm) and angular (onboard:3.7°, ideal:4.1°). We publicly release videos and the source code of our work.

Miniaturizing an autonomous robot is a challenging task - not only the mechanical but also the electrical components have to operate within limited space, payload, and power. Furthermore, the algorithms for autonomous navigation, such as state-of-the-art (SoA) visual navigation deep neural networks (DNNs), are becoming increasingly complex, striving for more flexibility and agility. In this work, we present a sensor-rich, modular, nano-sized Unmanned Aerial Vehicle (UAV), almost as small as a five Swiss Franc coin - called Fünfliber - with a total weight of 18g and 7.2cm in diameter. We conceived our UAV as an open-source hardware robotic platform, controlled by a parallel ultra-low power (PULP) system-on-chip (SoC) with a wide set of onboard sensors, including three cameras (i.e., infrared, optical flow, and standard QVGA), multiple Time-of-Flight (ToF) sensors, a barometer, and an inertial measurement unit. Our system runs the tasks necessary for a flight controller (sensor acquisition, state estimation, and low-level control), requiring only 10% of the computational resources available aboard, consuming only 9mW - 13x less than an equivalent Cortex M4-based system. Pushing our system at its limit, we can use the remaining onboard computational power for sophisticated autonomous navigation workloads, as we showcase with an SoA DNN running at up to 18Hz, with a total electronics' power consumption of 271mW.