Abstract Detecting malicious drones using aerial surveillance cameras is challenging when the distance is large, because the drone then occupies only a few pixels. Current optical detection methods rely mostly on visual appearance features. Hence, they struggle to differentiate drones from other flying objects, especially birds, when the apparent object size is small. Fortunately, the observed trajectory over time can help improve the differentiation accuracy. Here, we propose to combine classification neural networks of the object’s trajectory features and visual appearance features. We train and test the networks using our dataset containing infrared videos of drones and birds, where the variation of drone configurations and flight patterns is relatively larger than other publicly available datasets. We show that, particularly for small objects with high motion, the inclusion of trajectory features for visual classification achieves up to 22% higher frame-wise classification accuracy compared to when only visual appearance features are used. We further demonstrate that integrating both feature types provides improved accuracy over all of the considered trajectories, with 4% more of the trajectories being classified correctly. Consistent results are also shown on an open dataset, confirming the generalizability. Our study demonstrates the crucial role of information beyond the frame-wise visual appearance features in extending the operational range of aerial surveillance cameras.

Event cameras provide low-latency perception for only milliwatts of power. This makes them highly suitable for resource-restricted, agile robots such as small flying drones. Self-supervised learning based on contrast maximization holds great potential for event-based robot vision, as it foregoes the need for high-frequency ground truth and allows for online learning in the robot’s operational environment. However, online, on-board learning raises the major challenge of achieving sufficient computational efficiency for real-time learning, while maintaining competitive visual perception performance. In this work, we improve the time and memory efficiency of the contrast maximization pipeline, making on-device learning of low-latency monocular depth possible. We demonstrate that online learning on board a small drone yields more accurate depth estimates and more successful obstacle avoidance behavior compared to only pre-training. Benchmarking experiments show that the proposed pipeline is not only efficient, but also achieves state-of-the-art depth estimation performance among self-supervised approaches. Our work taps into the unused potential of online, on-device robot learning, promising smaller reality gaps and better performance.

In high-speed quadcopter racing, finding a single controller that works well across different platforms remains challenging. This work presents the first neural network controller for drone racing that generalizes across physically distinct quadcopters. We demonstrate that a single network, trained with domain randomization, can robustly control various types of quadcopters. The network relies solely on the current state to directly compute motor commands. The effectiveness of this generalized controller is validated through real-world tests on two substantially different crafts (3-inch and 5-inch race quadcopters). We further compare the performance of this generalized controller with controllers specifically trained for the 3-inch and 5-inch drone, using their identified model parameters with varying levels of domain randomization (0%, 10%, 20%, 30%). While the generalized controller shows slightly slower speeds compared to the fine-tuned models, it excels in adaptability across different platforms. Our results show that no randomization fails sim-to-real transfer while increasing randomization improves robustness but reduces speed. Despite this trade-off, our findings highlight the potential of domain randomization for generalizing controllers, paving the way for universal AI controllers that can adapt to any platform.

Ego-Motion estimation is vital for drones when flying in GPS-denied environments. Vision-Based methods struggle when flight speed increases and close-by objects lead to difficult visual conditions with considerable motion blur and large occlusions. To tackle this, vision is typically complemented by state estimation filters that combine a drone model with inertial measurements. However, these drone models are currently learned in a supervised manner with ground-truth data from external motion capture systems, limiting scalability to different environments and drones. In this work, we propose a self-supervised learning scheme to train a neural-network-based drone model using only onboard monocular video and flight controller data (IMU and motor feedback). We achieve this by first training a self-supervised relative pose estimation model, which then serves as a teacher for the drone model. To allow this to work at high speed close to obstacles, we propose an improved occlusion handling method for training self-supervised pose estimation models. Due to this method, the root mean squared error of resulting odometry estimates is reduced by an average of 15%. Moreover, the student neural drone model can be successfully obtained from the onboard data. It even becomes more accurate at higher speeds compared to its teacher, the self-supervised vision-based model. We demonstrate the value of the neural drone model by integrating it into a traditional filter-based VIO system (ROVIO), resulting in superior odometry accuracy on aggressive 3D racing trajectories near obstacles. Self-Supervised learning of ego-motion estimation represents a significant step toward bridging the gap between flying in controlled, expensive lab environments and real-world drone applications. The fusion of vision and drone models will enable higher-speed flight and improve state estimation, on any drone in any environment.

Neuromorphic computing systems are set to revolutionize energy-constrained robotics by achieving orders-of-magnitude efficiency gains, while enabling native temporal processing. Spiking Neural Networks (SNNs) represent a promising algorithmic approach for these systems, yet their application to complex control tasks faces two critical challenges: (1) the non-differentiable nature of spiking neurons necessitates surrogate gradients with unclear optimization properties, and (2) the stateful dynamics of SNNs require training on sequences, which in reinforcement learning (RL) is hindered by limited sequence lengths during early training, preventing the network from bridging its warm-up period. We address these challenges by systematically analyzing surrogate gradient slope settings, showing that shallower slopes increase gradient magnitude in deeper layers but reduce alignment with true gradients. In supervised learning, we find no clear preference for fixed or scheduled slopes. The effect is much more pronounced in RL settings, where shallower slopes or scheduled slopes lead to a $\times2.1$ improvement in both training and final deployed performance. Next, we propose a novel training approach that leverages a privileged guiding policy to bootstrap the learning process, while still exploiting online environment interactions with the spiking policy. Combining our method with an adaptive slope schedule for a real-world drone position control task, we achieve an average return of 400 points, substantially outperforming prior techniques, including Behavioral Cloning and TD3BC, which achieve at most –200 points under the same conditions. This work advances both the theoretical understanding of surrogate gradient learning in SNNs and practical training methodologies for neuromorphic controllers demonstrated in real-world robotic systems.

Hybrid overactuated tilt rotor uncrewed aerial vehicles (TRUAVs) are a category of versatile UAVs known for their exceptional wind resistance capabilities. However, their extensive operational range, combined with thrust vectoring capabilities, presents complex control challenges due to nonaffine dynamics and the necessity to coordinate lift and thrust for controlling accelerations at varying airspeeds. Traditionally, these vehicles rely on switched logic controllers with two or more intermediate states to control transitions. In this study, we introduce an innovative, unified incremental nonlinear controller designed to seamlessly control an overactuated dual-axis tilting rotor quad-plane throughout its entire flight envelope. Our controller is based on an incremental nonlinear control allocation algorithm to simultaneously generate pitch and roll commands, along with physical actuator commands. The control allocation problem is solved using a sequential quadratic programming (SQP) iterative optimization algorithm making it well-suited for the nonlinear actuator effectiveness typical of thrust vectoring vehicles. The controller's design integrates desired roll and pitch angle inputs. These desired attitude angles are managed by the controller and then conveyed to the vehicle during slow airspeed phases, when the vehicle maintains its 6-degrees of freedom (6-DOF). As the airspeed increases, the controller seamlessly shifts its focus to generating attitude commands for lift production, consequently smoothly disregarding the desired roll and pitch angles. Furthermore, our controller integrates an angle of attack (AoA) protection logic to mitigate wing stalling risks during transitions. It also features a yaw rate reference model to enable coordinated turns and minimize side-slip. The effectiveness of our proposed control technique has been confirmed through comprehensive flight tests. These tests demonstrated the successful transition from hovering flight to forward flight, the attainment of vertical and lateral accelerations, and the ability to revert to hovering.

Gas Source Localization (GSL) is a challenging field of research within the robotics community, with high-stakes search-and-rescue applications. Existing methods vary widely and each has its strengths and weaknesses. Comparisons of different methods are limited due to the lack of a broadly adopted and standardized testing methodology. Existing GSL evaluations vary in environment size, wind conditions, and gas simulation fidelity. They also lack photo-realistic rendering for the integration of obstacle avoidance. In this paper, we propose GSL-Bench, a benchmarking tool that can evaluate the performance of existing GSL algorithms. GSL-Bench features high-fidelity graphics and gas simulation, featuring NVIDIA’s®Isaac Sim and OpenFOAM computational fluid dynamics software (CFD). Realism is further increased by simulating relevant gas and wind sensors. Scene generation is simplified with the introduction of AutoGDM+, capable of procedural environment generation, CFD and particle-based gas dispersion simulation. To illustrate GSL-Bench’s capabilities, three algorithms are compared in six warehouse settings of increasing complexity: E. Coli, dung beetle, and a random walker. Our results demonstrate GSL-Bench’s ability to provide valuable insights into algorithm performance.Site: https://sites.google.com/view/gslbench/

Aggressive time-optimal control of quadcopters poses a significant challenge in the field of robotics. The state-of-the-art approach leverages reinforcement learning (RL) to train optimal neural policies. However, a critical hurdle is the sim-to-real gap, often addressed by employing a robust inner loop controller —an abstraction that, in theory, constrains the optimality of the trained controller, necessitating margins to counter potential disturbances. In contrast, our novel approach introduces high-speed quadcopter control using end-to-end RL (E2E) that gives direct motor commands. To bridge the reality gap, we incorporate a learned residual model and an adaptive method that can compensate for modeling errors in thrust and moments. We compare our E2E approach against a state-of-the-art network that commands thrust and body rates to an INDI inner loop controller, both in simulated and real-world flight. E2E showcases a significant 1.39-second advantage in simulation and a 0.17-second edge in real-world testing, highlighting end-to-end reinforcement learning’s potential. The performance drop observed from simulation to reality shows potential for further improvement, including refining strategies to address the reality gap or exploring offline reinforcement learning with real flight data.

Insects have long been recognized for their ability to navigate and return home using visual cues from their nest’s environment. However, the precise mechanism underlying this remarkable homing skill remains a subject of ongoing investigation. Drawing inspiration from the learning flights of honey bees and wasps, we propose a robot navigation method that directly learns the home vector direction from visual percepts during a learning flight in the vicinity of the nest. After learning, the robot will travel away from the nest, come back by means of odometry, and eliminate the resultant drift by inferring the home vector orientation from the currently experienced view. Using a compact convolutional neural network, we demonstrate successful learning in both simulated and real forest environments, as well as successful homing control of a simulated quadrotor. The average errors of the inferred home vectors in general stay well below the 90° required for successful homing, and below 24° if all images contain sufficient texture and illumination. Moreover, we show that the trajectory followed during the initial learning flight has a pronounced impact on the network’s performance. A higher density of sample points in proximity to the nest results in a more consistent return. Code and data are available at https://mavlab.tudelft.nl/learning_to_home.

Inspired by frame-based methods, state-of-the-art event-based optical flow networks rely on the explicit construction of correlation volumes, which are expensive to compute and store, rendering them unsuitable for robotic applications with limited compute and energy budget. Moreover, correlation volumes scale poorly with resolution, prohibiting them from estimating high-resolution flow. We observe that the spatiotemporally continuous traces of events provide a natural search direction for seeking pixel correspondences, obviating the need to rely on gradients of explicit correlation volumes as such search directions. We introduce IDNet (Iterative Deblurring Network), a lightweight yet high-performing event-based optical flow network directly estimating flow from event traces without using correlation volumes. We further propose two iterative update schemes: "ID" which iterates over the same batch of events, and "TID" which iterates over time with streaming events in an online fashion. Our top-performing model (ID) sets a new state of the art on DSEC benchmark. Meanwhile, the base model (TID) is competitive with prior arts while using 80% fewer parameters, consuming 20x less memory footprint and running 40% faster on the NVidia Jetson Xavier NX. Furthermore, the TID scheme is even more efficient offering an additional 5x faster inference speed and 8 ms ultra-low latency at the cost of only a 9% performance drop, making it the only model among current literature capable of real-time operation while maintaining decent performance.Code: https://github.com/tudelft/idnet.

Unmanned air vehicles (UAVs) have traditionally been considered as "eyes in the sky", that can move in three dimensions and need to avoid any contact with their environment. On the contrary, contact should not be considered as a problem, but as an opportunity to expand the range of UAVs applications. In this paper, we designed, fabricated, and characterized a whisker sensor unit based on MEMS barometers suitable for tactile localization on UAVs, featuring lightweight, low stiffness, high sensitivity, a broad sensing range, and scalability. Then, for the challenging task of contact point localization, we propose a Recurrent Multi-output Network (RMN) for predicting 3D contact points under continuous contact conditions to address the problems of non-linearity, hysteresis, and non-injective mapping between signals and contact points by considering time series. In addition, we propose an azimuth prediction loss function which reduces the RMSE by 3.24◦compared to L1loss. Finally, we conduct experiments on a linear stage to validate the 3D contact point localization capability of the proposed whisker system and model. The results show that our localization can achieve excellent performance, with an inference time of 1.4 ms and a mean error of only 9.18 mm in Euclidean distance within 3D space, laying a robust foundation for future implementation of tactile localization on UAVs. The design files, dataset, and source code are available on: https://github.com/BioMorphic-Intelligence-Lab/Whisker-3D-Localization.

Event cameras have recently gained significant traction since they open up new avenues for low-latency and low-power solutions to complex computer vision problems. To unlock these solutions, it is necessary to develop algorithms that can leverage the unique nature of event data. However, the current state-of-the-art is still highly influenced by the frame-based literature, and usually fails to deliver on these promises. In this work, we take this into consideration and propose a novel self-supervised learning pipeline for the sequential estimation of event-based optical flow that allows for the scaling of the models to high inference frequencies. At its core, we have a continuously-running stateful neural model that is trained using a novel formulation of contrast maximization that makes it robust to nonlinearities and varying statistics in the input events. Results across multiple datasets confirm the effectiveness of our method, which establishes a new state of the art in terms of accuracy for approaches trained or optimized without ground truth.

Nano quadcopters are small, agile, and cheap platforms that are well suited for deployment in narrow, cluttered environments. Due to their limited payload, these vehicles are highly constrained in processing power, rendering conventional vision-based methods for safe and autonomous navigation incompatible. Recent machine learning developments promise high-performance perception at low latency, while dedicated edge computing hardware has the potential to augment the processing capabilities of these limited devices. In this work, we present NanoFlowNet, a lightweight convolutional neural network for real-time dense optical flow estimation on edge computing hardware. We draw inspiration from recent advances in semantic segmentation for the design of this network. Additionally, we guide the learning of optical flow using motion boundary ground truth data, which improves performance with no impact on latency. Validation results on the MPI-Sintel dataset show the high performance of the proposed network given its constrained architecture. Additionally, we successfully demonstrate the capabilities of NanoFlowNet by deploying it on the ultra-low power GAP8 microprocessor and by applying it to vision-based obstacle avoidance on board a Bitcraze Crazyflie, a 34 g nano quadcopter.

Enabling the capability of assessing risk and making risk-aware decisions is essential to applying reinforcement learning to safety-critical robots like drones. In this paper, we investigate a specific case where a nano quadcopter robot learns to navigate an apriori-unknown cluttered environment under partial observability. We present a distributional reinforcement learning framework to generate adaptive risk-tendency policies. Specifically, we propose to use lower tail conditional variance of the learnt return distribution as intrinsic uncertainty estimation, and use exponentially weighted average forecasting (EWAF) to adapt the risk-tendency in accordance with the estimated uncertainty. In simulation and real-world empirical results, we show that (1) the most effective risk-tendency varies across states, (2) the agent with adaptive risk-tendency achieves superior performance compared to risk-neutral policy or risk-averse policy baselines. Code and video can be found in this repository: https://github.com/tudelft/risk-sensitive-rl.git

We present a novel controller for fixed-wing UAVs that enables autonomous soaring in an orographic wind field, extending flight endurance. Our method identifies soaring regions and addresses position control challenges by introducing a target gradient line (TGL) on which the UAV achieves an equilibrium soaring position, where sink rate and updraft are balanced. Experimental testing validates the controller's effectiveness in maintaining autonomous soaring flight without using any thrust in a non-static wind field. We also demonstrate a single degree of control freedom in a soaring position through manipulation of the TGL.

Obstacle avoidance is an essential topic in the field of autonomous drone research. When choosing an avoidance algorithm, many different options are available, each with their advantages and disadvantages. As there is currently no consensus on testing methods, it is quite challenging to compare the performance between algorithms. In this paper, we propose AvoidBench, a benchmarking suite which can evaluate the performance of vision-based obstacle avoidance algorithms by subjecting them to a series of tasks. Thanks to the high fidelity of multi-rotors dynamics from RotorS and virtual scenes of Unity3D, AvoidBench can realize realistic simulated flight experiments. Compared to current drone simulators, we propose and implement both performance and environment metrics to reveal the suitability of obstacle avoidance algorithms for environments of different complexity. To illustrate AvoidBench's usage, we compare three algorithms: Ego-planner, MBPlanner, and Agile-autonomy. The trends observed are validated with real-world obstacle avoidance experiments. Code is available at: https://github.com/tudelft/AvoidBench

Utilizing wind hovering techniques of soaring birds can save energy expenditure and improve the flight endurance of micro air vehicles (MAVs). Here, we present a novel method for fully autonomous orographic soaring without a priori knowledge of the wind field. Specifically, we devise an Incremental Nonlinear Dynamic Inversion (INDI) controller with control allocation, adapting it for autonomous soaring. This allows for both soaring and the use of the throttle if necessary, without changing any gain or parameter during the flight. Furthermore, we propose a simulated-annealing-based optimization method to search for soaring positions. This enables for the first time an MAV to autonomously find a feasible soaring position while minimizing throttle usage and other control efforts. Autonomous orographic soaring was performed in the wind tunnel. The wind speed and incline of a ramp were changed during the soaring flight. The MAV was able to perform autonomous orographic soaring for flight times of up to 30 minutes. The mean throttle usage was only 0.25% for the entire soaring flight, whereas normal powered flight requires 38%. Also, it was shown that the MAV can find a new soaring spot when the wind field changes during the flight.

Robotic airships offer significant advantages in terms of safety, mobility, and extended flight times. However, their highly restrictive weight constraints pose a major challenge regarding the available computational resources to perform the required control tasks. Neuromorphic computing stands for a promising research direction for addressing such problem. By mimicking the biological process for transferring information between neurons using spikes or impulses, spiking neural networks (SNNs) allow for low power consumption and asynchronous event-driven processing. In this paper, we propose an evolved altitude controller based on an SNN for a robotic airship which relies solely on the sensory feedback provided by an airborne radar. Starting from the design of a lightweight, low-cost, open-source airship, we also present an SNN-based controller architecture, an evolutionary framework for training the network in a simulated environment, and a control strategy for ameliorating the gap with reality. The system's performance is evaluated through real-world experiments, demonstrating the advantages of our approach by comparing it with an artificial neural network and a linear controller. The results show an accurate tracking of the altitude command with an efficient control effort.

Relative localization is an important ability for multiple robots to perform cooperative tasks in GPS-denied environments. This paper presents a novel autonomous positioning framework for monocular relative localization of multiple tiny flying robots. This approach does not require any groundtruth data from external systems or manual labeling. Instead, the proposed framework is able to label real-world images with 3D relative positions between robots based on another onboard relative estimation technology, using ultra-wideband (UWB). After training in this self-supervised manner, the proposed deep neural network (DNN) can predict relative positions of peer robots by purely using a monocular camera. This deep learning-based visual relative localization is scalable, distributed, and autonomous. We also built an open-source and lightweight simulation pipeline by using Blender for 3D rendering, which allows synthetic image generation of other robots, and generalized training of the neural network. The proposed localization framework is tested on two real-world Crazyflie2 quadrotors by running the DNN on the onboard AIdeck (a tiny AI chip and monocular camera). All results demonstrate the effectiveness of the self-supervised multi-robot localization method. Video: https://youtu.be/7arkaIblPps

This paper discusses a low-cost, open-source and open-hardware design and performance evaluation of a low-speed, multi-fan wind system dedicated to micro air vehicle (MAV) testing. In addition, a set of experiments with a flapping wing MAV and rotorcraft is presented, demonstrating the capabilities of the system and the properties of these different types of drones in response to various types of wind. We performed two sets of experiments where a MAV is flying into the wake of the fan system, gathering data about states, battery voltage and current. Firstly, we focus on steady wind conditions with wind speeds ranging from 0.5 m S-1 to 3.4 m S-1. During the second set of experiments, we introduce wind gusts, by periodically modulating the wind speed from 1.3 m S−1to 3.4 m S−1with wind gust oscillations of 0.5 Hz, 0.25 Hz and 0.125 Hz. The “Flapper” flapping wing MAV requires much larger pitch angles to counter wind than the “CrazyFlie” quadrotor. This is due to the Flapper's larger wing surface. In forward flight, its wings do provide extra lift, considerably reducing the power consumption. In contrast, the CrazyFlie's power consumption stays more constant for different wind speeds. The experiments with the varying wind show a quicker gust response by the CrazyFlie compared with the Flapper drone, but both their responses could be further improved. We expect that the proposed wind gust system will provide a useful tool to the community to achieve such improvements.

End-to-end trained convolutional neural networks have led to a breakthrough in optical flow estimation. The most recent advances focus on improving the optical flow estimation by improving the architecture and setting a new benchmark on the publicly available MPI-Sintel dataset. Instead, in this article, we investigate how deep neural networks estimate optical flow. A better understanding of how these networks function is important for (i) assessing their generalization capabilities to unseen inputs, and (ii) suggesting changes to improve their performance. For our investigation, we focus on FlowNetS, as it is the prototype of an encoder-decoder neural network for optical flow estimation. Furthermore, we use a filter identification method that has played a major role in uncovering the motion filters present in animal brains in neuropsychological research. The method shows that the filters in the deepest layer of FlowNetS are sensitive to a variety of motion patterns. Not only do we find translation filters, as demonstrated in animal brains, but thanks to the easier measurements in artificial neural networks, we even unveil dilation, rotation, and occlusion filters. Furthermore, we find similarities in the refinement part of the network and the perceptual filling-in process which occurs in the mammal primary visual cortex.

Event cameras are novel vision sensors that sample, in an asynchronous fashion, brightness increments with low latency and high temporal resolution. The resulting streams of events are of high value by themselves, especially for high speed motion estimation. However, a growing body of work has also focused on the reconstruction of intensity frames from the events, as this allows bridging the gap with the existing literature on appearance- and frame-based computer vision. Recent work has mostly approached this problem using neural networks trained with synthetic, ground-truth data. In this work we approach, for the first time, the intensity reconstruction problem from a self-supervised learning perspective. Our method, which leverages the knowledge of the inner workings of event cameras, combines estimated optical flow and the event-based photometric constancy to train neural networks without the need for any ground-truth or synthetic data. Results across multiple datasets show that the performance of the proposed self-supervised approach is in line with the state-of-the-art. Additionally, we propose a novel, lightweight neural network for optical flow estimation that achieves high speed inference with only a minor drop in performance.

We present fully autonomous source seeking onboard a highly constrained nano quadcopter, by contributing application-specific system and observation feature design to enable inference of a deep-RL policy onboard a nano quadcopter. Our deep-RL algorithm finds a high-performance solution to a challenging problem, even in presence of high noise levels and generalizes across real and simulation environments with different obstacle configurations. We verify our approach with simulation and in-field testing on a Bitcraze CrazyFlie using only the cheap and ubiquitous Cortex-M4 microcontroller unit. The results show that by end-to-end application-specific system design, our contribution consumes almost three times less additional power, as compared to a competitive learning-based navigation approach onboard a nano quadcopter. Thanks to our observation space, which we carefully design within the resource constraints, our solution achieves a 94% success rate in cluttered and randomized test environments, as compared to the previously achieved 80%. We also compare our strategy to a simple finite state machine (FSM), geared towards efficient exploration, and demonstrate that our policy is more robust and resilient at obstacle avoidance as well as up to 70% more efficient in source seeking. To this end, we contribute a cheap and lightweight end- to-end tiny robot learning (tinyRL) solution, running onboard a nano quadcopter, that proves to be robust and efficient in a challenging task.

Neuromorphic processors like Loihi offer a promising alternative to conventional computing modules for endowing constrained systems like micro air vehicles (MAVs) with robust, efficient and autonomous skills such as take-off and landing, obstacle avoidance, and pursuit. However, a major challenge for using such processors on robotic platforms is the reality gap between simulation and the real world. In this study, we present for the very first time a fully embedded application of the Loihi neuromorphic chip prototype in a flying robot. A spiking neural network (SNN) was evolved to compute the thrust command based on the divergence of the ventral optic flow field to perform autonomous landing. Evolution was performed in a Python-based simulator using the PySNN library. The resulting network architecture consists of only 35 neurons distributed among 3 layers. Quantitative analysis between simulation and Loihi reveals a root-mean-square error of the thrust setpoint as low as 0.005 g, along with a 99.8% matching of the spike sequences in the hidden layer, and 99.7% in the output layer. The proposed approach successfully bridges the reality gap, offering important insights for future neuromorphic applications in robotics. Supplementary material is available at https://mavlab.tudelft.nl/loihi/.

In the field of visual ego-motion estimation for Micro Air Vehicles (MAVs), fast maneuvers stay challenging mainly because of the big visual disparity and motion blur. In the pursuit of higher robustness, we study convolutional neural networks (CNNs) that predict the relative pose between subsequent images from a fast-moving monocular camera facing a planar scene. Aided by the Inertial Measurement Unit (IMU), we mainly focus on translational motion. The networks we study have similar small model sizes (around 1.35MB) and high inference speeds (around 10 milliseconds on a mobile GPU). Images for training and testing have realistic motion blur. Departing from a network framework that iteratively warps the first image to match the second with cascaded network blocks, we study different network architectures and training strategies. Simulated datasets and a self-collected MAV flight dataset are used for evaluation. The proposed setup shows better accuracy over existing networks and traditional feature-point-based methods during fast maneuvers. Moreover, self-supervised learning outperforms supervised learning. Videos and open-sourced code are available at https://github. com/tudelft/PoseNet_Planar

Current existing stereo visual odometry algorithms are computationally too expensive for robots with restricted resources. Executing these algorithms on such robots leads to a low frame rate and unacceptable decay in accuracy. We modify S-MSCKF, one of the most computationally efficient stereo Visual Inertial Odometry (VIO) algorithm, to improve its speed and accuracy when tracking low numbers of features. Specifically, we implement the Inverse Lucas-Kanade (ILK) algorithm for feature tracking and stereo matching. An outlier detector based on the average sum square difference of the template and matching warp in the ILK ensures higher robustness, e.g., in the presence of brightness changes. We restrict stereo matching to slide the window only in the x-direction to further decrease the computational costs. Moreover, we limit detection of new features to the regions of interest that have too few features. The modified S-MSCKF uses half of the processing time while obtaining competitive accuracy. This allows the algorithm to run in real-time on the extremely limited Raspberry Pi Zero single-board computer.

Nano quadcopters are ideal for gas source localization (GSL) as they are safe, agile and inexpensive. However, their extremely restricted sensors and computational resources make GSL a daunting challenge. We propose a novel bug algorithm named ‘Sniffy Bug', which allows a fully autonomous swarm of gas-seeking nano quadcopters to localize a gas source in unknown, cluttered, and GPS-denied environments. The computationally efficient, mapless algorithm foresees in the avoidance of obstacles and other swarm members, while pursuing desired waypoints. The waypoints are first set for exploration, and, when a single swarm member has sensed the gas, by a particle swarm optimization-based (PSO) procedure. We evolve all the parameters of the bug (and PSO) algorithm using our novel simulation pipeline, ‘AutoGDM'. It builds on and expands open source tools in order to enable fully automated end-to-end environment generation and gas dispersion modeling, allowing for learning in simulation. Flight tests show that Sniffy Bug with evolved parameters outperforms manually selected parameters in cluttered, real-world environments. Videos: https://bit.ly/37MmtdL

Micro Air Vehicles (MAVs) are increasingly being used for complex or hazardous tasks in enclosed and cluttered environments such as surveillance or search and rescue. With this comes the necessity for sensors that can operate in poor visibility conditions to facilitate with navigation and avoidance of objects or people. Radar sensors in particular can provide more robust sensing of the environment when traditional sensors such as cameras fail in the presence of dust, fog or smoke. While extensively used in autonomous driving, miniature FMCW radars on MAVs have been relatively unexplored. This study aims to investigate to what extent this sensor is of use in these environments by employing traditional signal processing such as multi-target tracking and velocity obstacles. The viability of the solution is evaluated with an implementation on board a MAV by running trial tests in an indoor environment containing obstacles and by comparison with a human pilot, demonstrating the potential for the sensor to provide a more robust sense and avoid function in fully autonomous MAVs.

Multi-robot systems can benefit from reinforcement learning (RL) algorithms that learn behaviours in a small number of trials, a property known as sample efficiency. This research thus investigates the use of learned world models to improve sample efficiency. We present a novel multi-agent model-based RL algorithm: Multi-Agent Model-Based Policy Optimization (MAMBPO), utilizing the Centralized Learning for Decentralized Execution (CLDE) framework. CLDE algorithms allow a group of agents to act in a fully decentralized manner after training. This is a desirable property for many systems comprising of multiple robots. MAMBPO uses a learned world model to improve sample efficiency compared to model-free Multi-Agent Soft Actor-Critic (MASAC). We demonstrate this on two simulated multi-robot tasks, where MAMBPO achieves a similar performance to MASAC, but requires far fewer samples to do so. Through this, we take an important step towards making real-life learning for multi-robot systems possible.

The field of neuromorphic computing promises extremely low-power and low-latency sensing and processing. Challenges in transferring learning algorithms from traditional artificial neural networks (ANNs) to spiking neural networks (SNNs) have so far prevented their application to large-scale, complex regression tasks. Furthermore, realizing a truly asynchronous and fully neuromorphic pipeline that maximally attains the abovementioned benefits involves rethinking the way in which this pipeline takes in and accumulates information. In the case of perception, spikes would be passed as-is and one-by-one between an event camera and an SNN, meaning all temporal integration of information must happen inside the network. In this article, we tackle these two problems. We focus on the complex task of learning to estimate optical flow from event-based camera inputs in a self-supervised manner, and modify the state-of-the-art ANN training pipeline to encode minimal temporal information in its inputs. Moreover, we reformulate the self-supervised loss function for event-based optical flow to improve its convexity. We perform experiments with various types of recurrent ANNs and SNNs using the proposed pipeline. Concerning SNNs, we investigate the effects of elements such as parameter initialization and optimization, surrogate gradient shape, and adaptive neuronal mechanisms. We find that initialization and surrogate gradient width play a crucial part in enabling learning with sparse inputs, while the inclusion of adaptivity and learnable neuronal parameters can improve performance. We show that the performance of the proposed ANNs and SNNs are on par with that of the current state-of-the-art ANNs trained in a self-supervised manner.

Optimal control holds great potential to improve a variety of robotic applications. The application of optimal control on-board limited platforms has been severely hindered by the large computational requirements of current state of the art implementations. In this work, we make use of a deep neural network to directly map the robot states to control actions. The network is trained offline to imitate the optimal control computed by a time consuming direct nonlinear method. A mixture of time optimality and power optimality is considered with a continuation parameter used to select the predominance of each objective. We apply our networks (termed G&CNets) to aggressive quadrotor control, first in simulation and then in the real world. We give insight into the factors that influence the `reality gap' between the quadrotor model used by the offline optimal control method and the real quadrotor. Furthermore, we explain how we set up the model and the control structure on-board of the real quadrotor to successfully close this gap and perform time-optimal maneuvers in the real world. Finally, G&CNet's performance is compared to state-of-the-art differential-flatness-based optimal control methods. We show, in the experiments, that G&CNets lead to significantly faster trajectory execution due to, in part, the less restrictive nature of the allowed state-to-input mappings.

The combination of spiking neural networks and event-based vision sensors holds the potential of highly efficient and high-bandwidth optical flow estimation. This paper presents the first hierarchical spiking architecture in which motion (direction and speed) selectivity emerges in an unsupervised fashion from the raw stimuli generated with an event-based camera. A novel adaptive neuron model and stable spike-timing-dependent plasticity formulation are at the core of this neural network governing its spike-based processing and learning, respectively. After convergence, the neural architecture exhibits the main properties of biological visual motion systems, namely feature extraction and local and global motion perception. Convolutional layers with input synapses characterized by single and multiple transmission delays are employed for feature and local motion perception, respectively; while global motion selectivity emerges in a final fully-connected layer. The proposed solution is validated using synthetic and real event sequences. Along with this paper, we provide the cuSNN library, a framework that enables GPU-accelerated simulations of large-scale spiking neural networks. Source code and samples are available at https://github.com/tudelft/cuSNN.

Deep neural networks have lead to a breakthrough in depth estimation from single images. Recent work shows that the quality of these estimations is rapidly increasing. It is clear that neural networks can see depth in single images. However, to the best of our knowledge, no work currently exists that analyzes what these networks have learned. In this work we take four previously published networks and investigate what depth cues they exploit. We find that all networks ignore the apparent size of known obstacles in favor of their vertical position in the image. The use of the vertical position requires the camera pose to be known; however, we find that these networks only partially recognize changes in camera pitch and roll angles. Small changes in camera pitch are shown to disturb the estimated distance towards obstacles. The use of the vertical image position allows the networks to estimate depth towards arbitrary obstacles - even those not appearing in the training set - but may depend on features that are not universally present.

We study how autonomous robots can learn by themselves to improve their depth estimation capability. In particular, we investigate a self-supervised learning setup in which stereo vision depth estimates serve as targets for a convolutional neural network (CNN) that transforms a single still image to a dense depth map. After training, the stereo and mono estimates are fused with a novel fusion method that preserves high confidence stereo estimates, while leveraging the CNN estimates in the low-confidence regions. The main contribution of the article is that it is shown that the fused estimates lead to a higher performance than the stereo vision estimates alone. Experiments are performed on the KITTI dataset, and on board of a Parrot SLAMDunk, showing that even rather limited CNNs can help provide stereo vision equipped robots with more reliable depth maps for autonomous navigation.

One of the emerging tasks for Micro Air Vehicles (MAVs) is autonomous indoor navigation. While commonly employed platforms for such tasks are micro-quadrotors, insect-inspired flapping wing MAVs can offer many advantages, such as being inherently safe due to their low inertia, reciprocating wings bouncing of objects or potentially lower noise levels compared to rotary wings. Here, we present the first flapping wing MAV to perform an autonomous multi-room exploration task. Equipped with an on-board autopilot and a 4 g stereo vision system, the DelFly Explorer succeeded in combining the two most common tasks of an autonomous indoor exploration mission: room exploration and door passage. During the room exploration, the vehicle uses stereo-vision based droplet algorithm to avoid and navigate along the walls and obstacles. Simultaneously, it is running a newly developed monocular color based Snake-gate algorithm to locate doors. A successful detection triggers the heading-based door passage algorithm. In the real-world test, the vehicle could successfully navigate, multiple times in a row, between two rooms separated by a corridor, demonstrating the potential of flapping wing vehicles for autonomous exploration tasks.

Indoor navigation has been a major focus of drone research over the last few decades. The main reason for the term “indoor” came from the fact that in outdoor environments, drones could rely on global navigation systems such as GPS for their position and velocity estimates. By focusing on unknown indoor environments, the research had to focus on solutions using onboard sensors and processing. In this article, we present an overview of the state of the art and remaining challenges in this area, with a focus on small drones.

A control strategy is proposed to deal with the fundamental gain selection problem of optical flow landings. It involves detecting the height by means of an oscillating movement and setting the control gains accordingly at the start of a landing. Then, during descent, the gains are reduced exponentially, with mechanisms in place to ensure high-performance landings. Real-world experiments with a quadrotor demonstrate successful landings in both indoor and outdoor environments.

Pocket-drones are inherently safe for flight near humans, and their small size allows maneuvering through narrow indoor environments. However, achieving autonomous flight of pocket-drones is challenging because of strict on-board hardware limitations. Further challenges arise when multiple pocket-drones operate as a team and need to coordinate their movements. This paper presents a set-up that can achieve autonomous flight in an indoor environment with avoidance of both static obstacles and other pocket-drones.The pocket-drones use only on-board sensing and processing implemented on a STM32F4 microprocessor (168 MHz). Experiments were conducted with two 40g pocket-drones flying autonomously in a real-world office while avoiding walls, obstacles, and each-other.

One of the major challenges of evolutionary robotics is to transfer robot controllers evolved in simulation to robots in the real world. In this article, we investigate abstraction of the sensory inputs and motor actions as a tool to tackle this problem. Abstraction in robots is simply the use of preprocessed sensory inputs and low-level closed-loop control systems that execute higher-level motor commands. To demonstrate the impact abstraction could have, we evolved two controllers with different levels of abstraction to solve a task of forming an asymmetric triangle with a homogeneous swarm of micro air vehicles. The results show that although both controllers can effectively complete the task in simulation, the controller with the lower level of abstraction is not effective on the real vehicle, due to the reality gap. The controller with the higher level of abstraction is, however, effective both in simulation and in reality, suggesting that abstraction can be a useful tool in making evolved behavior robust to the reality gap. Additionally, abstraction aided in reducing the computational complexity of the simulation environment, speeding up the optimization process. Preeminently, we show that the optimized behavior exploits the environment (in this case the identical behavior of the other robots) and performs input shaping to allow the vehicles to fly into and maintain the required formation, demonstrating clear sensory-motor coordination. This shows that the power of the genetic optimization to find complex correlations is not necessarily lost through abstraction as some have suggested.

The development of autonomous lightweight MAVs, capable of navigating in unknown indoor environments, is one of the major challenges in robotics. The complexity of this challenge comes from constraints on weight and power consumption of onboard sensing and processing devices. In this paper, we propose the “Droplet” strategy, an avoidance strategy based on stereo vision inputs that outperforms reactive avoidance strategies by allowing constant speed maneuvers while being computationally extremely efficient, and which does not need to store previous images or maps. The strategy deals with nonholonomic motion constraints of most fixed and flapping wing platforms, and with the limited field-of-view of stereo camera systems. It guarantees obstacle-free flight in the absence of sensor and motor noise. We first analyze the strategy in simulation, and then show its robustness in real-world conditions by implementing it on a 20-gram flapping wing MAV.

Autonomous flight of pocket drones is challenging due to the severe limitations on on-board energy, sensing, and processing power. However, tiny drones have great potential as their small size allows maneuvering through narrow spaces while their small weight provides significant safety advantages. This paper presents a computationally efficient algorithm for determining optical flow, which can be run on an STM32F4 microprocessor (168 MHz) of a 4 gram stereo-camera. The optical flow algorithm is based on edge histograms. We propose a matching scheme to determine local optical flow. Moreover, the method allows for sub-pixel flow determination based on time horizon adaptation. We demonstrate velocity measurements in flight and use it within a velocity control-loop on a pocket drone.

Despite an intensive research on flapping flight and flapping wing MAVs in recent years, there are still no accurate models of flapping flight dynamics. This is partly due to lack of free flight data, in particular during manoeuvres. In this work, we present, for the first time, a comparison of free flight forces estimated using solely an on-board IMU with wind tunnel measurements. The IMU based estimation brings higher sampling rates and even lower variation among individual wingbeats, compared to what has been achieved with an external motion tracking system in the past. A good match was found in comparison to wind tunnel measurements; the slight differences observed are attributed to clamping effects. Further insight was gained from the on-board rpm sensor, which showed motor speed variation of ± 15% due to load variation over a wingbeat cycle. The IMU based force estimation represents an attractive solution for future studies of flapping wing MAVs as, unlike wind tunnel measurements, it allows force estimation at high temporal resolutions also during manoeuvres.

Obstacle detection by monocular vision is challenging because a single camera does not provide a direct measure for absolute distances to objects. A self-supervised learning approach is proposed that combines a camera and a very small short-range proximity sensor to find the relation between the appearance of objects in camera images and their corresponding distances. The method is efficient enough to run real time on a small camera system that can be carried onboard a lightweight MAV of 19 g. The effectiveness of the method is demonstrated by computer simulations and by experiments with the real platform in flight.

No quantitative procedure currently exists to evaluate the obstacle avoidance capabilities of robotic systems. Such an evaluation method is not only needed to compare different avoidance methods, but also to determine the operational limits of autonomous systems. This work proposes an evaluation framework which can find such limits. The framework comprises two types of tests: detection tests and avoidance tests. For each type, both environment and performance metrics need to be defined. For detection tests such metrics are well known, but for avoidance tests such metrics are not readily available. Therefore a new set of metrics is proposed. The framework is applied to a UAV that uses stereo vision to detect obstacles. Three different avoidance methods are compared in environments of varying difficulty.

This work extends the drag-force enhanced quadrotor model by denoting the free stream air velocity as the difference between the ground speed and the wind speed. It is demonstrated that a relatively simple nonlinear observer is capable of estimating the local wind components, provided accelerometer and GPS-velocity measurements are available. We perform a wind tunnel experiment at various wind speeds using a quadrotor vehicle with a low-cost Inertial Measurement Unit (IMU) and a motion tracking system to provide accurate ground speed measurements. It is shown that the onboard Extended Kalman Filter (EKF) accurately estimates the wind components.

Micro Aerial Vehicles (MAVs) are limited in their operation outdoors near obstacles by their ability to withstand wind gusts. Currently widespread position control methods such as Proportional Integral Derivative control do not perform well under the influence of gusts. Incremental Nonlinear Dynamic Inversion (INDI) is a sensor-based control technique that can control nonlinear systems subject to disturbances. This method was developed for the attitude control of MAVs, but in this paper we generalize this method to the outer loop control of MAVs under gust loads. Significant improvements over a traditional Proportional Integral Derivative (PID) controller are demonstrated in an experiment where the drone flies in and out of a fan's wake. The control method does not rely on frequent position updates, so it is ready to be applied outside with standard GPS modules.

Evolutionary Robotics allows robots with limited sensors and processing to tackle complex tasks by means of sensory-motor coordination. In this article we show the first application of the Behavior Tree framework on a real robotic platform using the evolutionary robotics methodology. This framework is used to improve the intelligibility of the emergent robotic behavior over that of the traditional neural network formulation. As a result, the behavior is easier to comprehend and manually adapt when crossing the reality gap from simulation to reality. This functionality is shown by performing real-world flight tests with the 20-g DelFly Explorer flapping wing micro air vehicle equipped with a 4-g onboard stereo vision system. The experiments show that the DelFly can fully autonomously search for and fly through a window with only its onboard sensors and processing. The success rate of the optimized behavior in simulation is 88%, and the corresponding real-world performance is 54% after user adaptation. Although this leaves room for improvement, it is higher than the 46% success rate from a tuned user-defined controller.

The autonomous capabilities of light-weight Flapping Wing Micro Air Vehicles (FWMAVs) have much to gain from onboard state estimation and attitude control. In this article, we present the first FWMAV with robust onboard state estimation and attitude control. The tailed FWMAV DelFly II was used, with the main goal to achieve active stabilization in the (passively unstable) hover condition. The attitude is estimated using an Inertial Measurement Unit with a gyroscope, accelerometer and magnetometer and the altitude is estimated using a barometer. A major challenge lies in the disturbance of the accelerometer measurements by the flapping motion of the wings. We propose a mechanical damping mechanism and flap-cycle based filtering to resolve this issue. The pitch estimates have a mean error of 1.5° with respect to the ground-truth measurement from a motion capture system. Using the onboard pitch estimate we can control the attitude of the FWMAV in the forward flight regime with a 30% lower standard deviation than in a trimmed flight. With a different set of gains, the FWMAV is able to perform a hovering flight - showing that a tailed FWMAV has enough control authority for this task. In a fully autonomous hover experiment, the DelFly II stays within a sphere of 0.75 m radius.

We introduce a novel setup of self-supervised learning (SSL), in which optical flow provides the supervised outputs. Optical flow requires significant movement for obstacle detection. The main advantage of the introduced method is that after learning, a robot can detect obstacles without moving - reducing the risk of collisions in narrow spaces. We investigate this novel setup of SSL in the context of a Micro Air Vehicle (MAV) that needs to select a suitable landing place. Initially, when the MAV flies over a potential landing area, the optical flow processing estimates a ‘surface roughness’ measure, capturing whether there are obstacles sticking out of the landing surface. This measure allows the MAV to select a safe landing place and then land with other optical flow measures such as the divergence. During flight, SSL takes place. For each image a texton distribution is extracted (capturing the visual appearance of the landing surface in sight), and mapped to the current roughness value by a linear regression function. We first demonstrate this principle to work with offline tests involving images captured on board an MAV, and then demonstrate the principle in flight. The experiments show that the MAV can land safely on the basis of optical flow. After learning it can also successfully select safe landing spots in hover. It is even shown that the appearance learning allows the pixel-wise segmentation of obstacles.

Autonomous flight of Flapping Wing Micro Air Vehicles (FWMAVs) is a major challenge in the field of robotics, due to their light weight and the flapping-induced body motions. In this article, we present the first FWMAV with onboard vision processing for autonomous flight in generic environments. In particular, we introduce the DelFly `Explorer', a 20-gram FWMAV equipped with a 0.98-gram autopilot and a 4.0-gram onboard stereo vision system. We explain the design choices that permit carrying the extended payload, while retaining the DelFly's hover capabilities. In addition, we introduce a novel stereo vision algorithm, LongSeq, designed specifically to cope with the flapping motion and the desire to attain a computational effort tuned to the frame rate. The onboard stereo vision system is illustrated in the context of an obstacle avoidance task in an environment with sparse obstacles.

For autonomous robots to operate successfully in unknown environments, their computer vision algorithms need to generalize over many different environments. However, due to practical considerations robotic vision experiments are typically limited to a single robot and a few (laboratory) environments. We propose crowdsourcing as a methodology for gathering large and varied robotic data sets. We evaluate the methodology by performing the first crowdsourcing experiment involving actual robots. In particular, we have made a space-game called `Astro Drone' for a toy quad rotor, the Parrot AR drone. Nine months after the game's release, there are 14,628 downloads and 840 contributions, consisting of visual features and drone state estimates. Data mining shows the methodology's potential, providing insights such as the relation between the number of visual features and obstacle distances.

We make use of self-adaptation in a Differential Evolution algorithm and of the asynchronous island model to design a complex interplanetary trajectory touring the Galilean Jupiter moons (Io, Europa, Ganymede and Callisto) using the multiple gravity assist technique. Such a problem was recently the subject of an international competition organized by the Jet Propulsion Laboratory (NASA) and won by a trajectory designed by aerospace experts and reaching the final score of 311/324. We apply our method to the very same problem finding new surprising designs and orbital strategies and a score of up to 316/324.

A highly efficient corner finding algorithm, named ACT-CORNER, is introduced. The algorithm uses active sub-sampling: it searches for corners by exploiting information from local image samples. ACT-CORNER is compared to both the Harris and FAST corner detectors. It is computationally much more efficient than the Harris corner detector. The comparison with FAST depends on the image size, with an advantage for ACT-CORNER for image dimensions of 320 × 240 pixels or larger. ACT-CORNER's performance is evaluated in the context of optic flow based Time-To-Contact (TTC) estimation. Image zoom experiments show that the accuracy of TTC estimations is similar when ACT-CORNER or FAST are used for corner detection. Finally, experiments with a Parrot AR drone show that the TTC estimates based on ACT-CORNER correspond well to sonar-based estimates.

The appearance variation cue captures the variation in texture in a single image. Its use for obstacle avoidance is based on the assumption that there is less such variation when the camera is close to an obstacle. For videos of approaching frontal obstacles, it is demonstrated that combining the cue with optic flow leads to better performance than using either cue alone. In addition, the cue is successfully used to control the 16-g flapping-wing micro air vehicle DelFly II.

In this article we investigate whether an artificial agent can successfully perform the complex task of object detection in static natural images. For this task, we evolve situated agents that map local image samples to gaze shifts in order to find object locations in the image. We apply the agents to both a task of mug detection in a domestic environment and a task of face detection in an office environment. The analysis of evolved agents shows that they employ sensory-motor coordination to exploit the object's visual context. The experimental results show that the situated agents achieve a detection performance equal to that of existing object detection methods, while extracting ~50 times fewer local image samples. This advantage comes at the expense of limited generalisation performance: evolved agents exploit the scene-specific contextual clues which may be confined to a single type of visual environment and may therefore not generalise to other types of visual environments. We conclude that the studied situated agents can efficiently and successfully perform object detection at the cost of application generality

Microscopic analysis is a standard approach in the study of robot behaviour. Typically, the approach comprises the analysis of a single (or sometimes a few) robot–environment system(s) to reveal specific properties of robot behaviour. In contrast to microscopic analysis, macroscopic analysis focuses on averaged properties of systems. The advantage is that such a property is easier to generalize so that it can be established to what extent the property is universal. This paper investigates whether a macroscopic analysis can reveal a universal property of adaptive behaviour in a robot model of foraging behaviour. Our analysis reveals that the step lengths of the most successful robots are distributed according to a Lévy-flight distribution. From studies on a variety of natural species, it is known that such a distribution constitutes a universal property of foraging behaviour. Thereafter, we discuss an example of how macroscopic analysis can be applied to existing research in evolutionary robotics, and relate the macroscopic and microscopic analyses of foraging behaviour to the framework of scientific research described by Cohen (1995, Empirical Methods for Artificial Intelligence (Cambridge MA: MIT Press)). We conclude that macroscopic analysis may predict universal properties of adaptive behaviour and that it may complement microscopic analysis in the study of adaptive behaviour.