We present an autonomous aerial system for safe and efficient through-the-canopy fruit counting. Aerial robot applications in large-scale orchards face significant challenges due to the complexity of fine-tuning flight paths based on orchard layouts, canopy density, and plant variability. Through-the-canopy navigation is crucial for minimizing occlusion by leaves and branches but is more challenging due to the complex and dense environment compared to traditional over-the-canopy flights. Our system addresses these challenges by integrating: i) a high-fidelity simulation framework for global path planning, ii) a low-cost autonomy stack for canopy-level navigation and data collection, and iii) a robust workflow for fruit detection and counting using RGB images. We validate our approach through fruit counting with canopy-level aerial images and by demonstrating the autonomous navigation capabilities of our experimental vehicle.

This paper proposes the ProxFly, a residual deep Reinforcement Learning (RL)-based controller for close proximity quadcopter flight. Specifically, we design a residual module on top of a cascaded controller (denoted as basic controller) to generate high-level control commands, which compensate for external disturbances and thrust loss caused by downwash effects from other quadcopters. First, our method takes only the ego state and controllers' commands as inputs and does not rely on any communication between quadcopters, thereby reducing the bandwidth requirement. Through domain randomization, our method relaxes the requirement for accurate system identification and fine-tuned controller parameters, allowing it to adapt to changing system models. Meanwhile, our method not only reduces the proportion of unexplainable signals from the black box in control commands but also enables the RL training to skip the time-consuming exploration from scratch via guidance from the basic controller. We validate the effectiveness of the residual module in the simulation with different proximities. Moreover, we conduct the real close proximity flight test to compare ProxFly with the basic controller and an advanced model-based controller with complex aerodynamic compensation. Finally, we show that ProxFly can be used for challenging quadcopter midair docking, where two quadcopters fly in extreme proximity, and strong airflow significantly disrupts flight. However, our method can stabilize the quadcopter in this case and accomplish docking. The resources are available at https://github.com/ruiqizhang99/ProxFly.

Defect diagnosis in urban infrastructure is crucial for public safety. Traditional manual inspections face significant challenges in terms of accuracy and cost-effectiveness. In this paper, we propose a lightweight and hardware-friendly large-scale infrastructure detector, CUPID, highly suitable for unmanned aerial vehicles (UAVs). Given the significant challenges in automatically detecting defects of varying intensity and size within complex infrastructure, along with the tendency of lightweight models to lose detail and fail to fully capture features during the defect extraction process, we propose the CUPID_Block, a multi-level information fusion block to construct the backbone, featuring the CUPID_Conv module equipped with our proposed CCA (CrissCross Attention). Furthermore, CUPID features an auxiliary training branch that assimilates lower feature maps, helping to recover details lost in deeper convolutional layers. To verify the effectiveness of CUPID and to address the lack of a suitable dataset in the community, we establish a multi-scenario infrastructure defect dataset, CUBIT2024, to conduct extensive experiments. Finally, to assess the efficiency and adaptability of CUPID in UAV for online infrastructure inspection, we design a compact autonomous drone, CU-Astro, where the proposed CUPID is deployed on the Jetson Orin NX computer onboard to evaluate the speed and power consumption of the inference.

This paper introduces a learning-based low-level controller for quadcopters, which adaptively controls quadcopters with significant variations in mass, size, and actuator capabilities. Our approach leverages a combination of imitation learning and reinforcement learning, creating a fast-adapting and general control framework for quadcopters that eliminates the need for precise model estimation or manual tuning. The controller estimates a latent representation of the vehicle's system parameters from sensor-action history, enabling it to adapt swiftly to diverse dynamics. Extensive evaluations in simulation demonstrate the controller's ability to generalize to unseen quadcopter parameters, with an adaptation range up to 16 times broader than the training set. In real-world tests, the controller is successfully deployed on quadcopters with mass differences of 3.7 times and propeller constants varying by more than 100 times, while also showing rapid adaptation to disturbances such as off-center payloads and motor failures. These results highlight the potential of our controller to simplify the design process and enhance the reliability of autonomous drone operations in unpredictable environments. Video and code are at:https://github.com/muellerlab/xadapt_ctrl

This paper proposes an adaptive near-hover position controller for quadcopters, which can be deployed to quadcopters of very different mass, size and motor constants, and also shows rapid adaptation to unknown disturbances during runtime. The core algorithmic idea is to learn a single policy that can adapt online at test time not only to the disturbances applied to the drone, but also to the robot dynamics and hardware in the same framework. We achieve this by training a neural network to estimate a latent representation of the robot and environment parameters, which is used to condition the behaviour of the controller, also represented as a neural network. We train both networks exclusively in simulation with the goal of flying the quadcopters to goal positions and avoiding crashes to the ground. We directly deploy the same controller trained in the simulation without any modifications on two quadcopters in the real world with differences in mass, size, motors, and propellers with mass differing by 4.5 times. In addition, we show rapid adaptation to sudden and large disturbances up to one-third of the mass of the quadcopters. We perform an extensive evaluation in both simulation and the physical world, where we outperform a state-of-the-art learning-based adaptive controller and a traditional PID controller specifically tuned to each platform individually. Video results can be found at https://youtu.be/U-c-LbTfvoA.

In this article, a novel quadcopter capable of changing shape midflight is presented, allowing for operation in four configurations with the capability of sustained hover in three. This is accomplished without requiring actuators beyond the four motors typical of a quadcopter. Morphing is achieved through freely rotating hinges that allow the vehicle arms to fold downwards by either reducing or reversing thrust forces. Constraints placed on the control inputs of the vehicle prevent the arms from folding or unfolding unexpectedly. This allows for the use of existing quadcopter controllers and trajectory generation algorithms with only minimal added complexity. For our experimental vehicle at hover, we find that these constraints result in a 36% reduction of the maximum yaw torque the vehicle can produce, but do not result in a reduction of the maximum thrust or roll and pitch torques. Experimental results show that, for a typical maneuver, the added limits have a negligible effect on the trajectory tracking performance. Finally, the ability to change configurations is shown to enable the vehicle to traverse small passages, perch on hanging wires, and perform limited grasping tasks.

This paper proposes a state estimator for legged robots operating in slippery environments. An Invariant Extended Kalman Filter (InEKF) is implemented to fuse inertial and velocity measurements from a tracking camera and leg kinematic constraints. The misalignment between the camera and the robot-frame is also modeled thus enabling auto-calibration of camera pose. The leg kinematics based velocity measurement is formulated as a right-invariant observation. Nonlinear observability analysis shows that other than the rotation around the gravity vector and the absolute position, all states are observable except for some singular cases. Discrete observability analysis demonstrates that our filter is consistent with the underlying nonlinear system. An online noise parameter tuning method is developed to adapt to the highly time-varying camera measurement noise. The proposed method is experimentally validated on a Cassie bipedal robot walking over slippery terrain. A video for the experiment can be found at https://youtu.be/VIqJL0cUr7s.

Multicopters with collision-resilient designs can operate with trajectories involving collisions. This paper presents a sampling-based method that can exploit collisions for better motion planning. The method is built upon the basis of the RRT* algorithm and takes advantages of fast motion primitive generation and collision checking for multicopters. It generates collision states by detecting potential intersections between motion primitives and obstacles, and connects these states with other sampled states to form collision-inclusive trajectories. We show that allowing collision helps improve the performance of the sampling-based planner in narrow spaces like tunnels. Finally, an experiment of tracking the trajectory generated by the collision-inclusive planner is presented.

The capabilities of autonomous flight with unmanned aerial vehicles (UAVs) have significantly increased in recent times. However, basic problems such as fast and robust geo-localization in GPS-denied environments still remain unsolved. Existing research has primarily concentrated on improving the accuracy of localization at the cost of long and varying computation time in various situations, which often necessitates the use of powerful ground station machines. In order to make image-based geo-localization online and pragmatic for lightweight embedded systems on UAVs, we propose a framework that is reliable in changing scenes, flexible about computing resource allocation and adaptable to common camera placements. The framework is comprised of two stages: offline database preparation and online inference. At the first stage, color images and depth maps are rendered as seen from potential vehicle poses quantized over the satellite and topography maps of anticipated flying areas. A database is then populated with the global and local descriptors of the rendered images. At the second stage, for each captured real-world query image, top global matches are retrieved from the database and the vehicle pose is further refined via local descriptor matching. We present field experiments of image-based localization on two different UAV platforms to validate our results.

We present a novel approach to increase the flight time of a multirotor via mid-air docking and in-flight battery switching. A main quadcopter flying using a primary battery has a docking platform attached to it. A `flying battery' - a small quadcopter carrying a secondary battery - is equipped with docking legs that can mate with the main quadcopter's platform. Connectors between the legs and the platform establish electrical contact on docking, and enable power transfer from the secondary battery to the main quadcopter. A custom-designed circuit allows arbitrary switching between the primary battery and secondary battery. We demonstrate the concept in a flight experiment involving repeated docking, battery switching, and undocking. This is shown in the video attachment. The experiment increases the flight time of the main quadcopter by a factor of 4.7× compared to solo flight, and 2.2× a theoretical limit for that given multirotor. Importantly, this increase in flight time is not associated with a large increase in overall vehicle mass or size, leaving the main quadcopter in fundamentally the same safety class.

In certain challenging environments, such as inside buildings on fire, the main sensors (e.g. cameras, LiDARs and GPS systems) used for multicopter localization can become unavailable. Direct integration of the inertial navigation sensors (the accelerometer and rate gyroscope), is however unaffected by external disturbances, but the rapid error accumulation quickly makes a naive application of such a strategy feasible only for very short durations. In this work we propose a motion strategy for reducing the inertial navigation state estimation error of multicopters. The proposed strategy breaks a long duration flight into multiple short duration hops between which the vehicle remains stationary on the ground. When the vehicle is stationary, zero-velocity pseudo-measurements are introduced to an extended Kalman Filter to reduce the state estimation error. We perform experiments for closed-loop control of a multicopter for evaluation. The mean absolute position estimation error was 3.4% over a total flight distance of 5m in the experiments. The results showed a 80% reduction compared to the standard inertial navigation method without using this strategy. In addition, an additional experiment with total flight distance of 10m is conducted to demonstrate the ability of this method to navigate a multicopter in real-world environment. The final trajectory tracking error was 3% of the total flight distance.

Energy sources such as batteries do not decrease in mass after consumption, unlike combustion-based fuels. We present the concept of staging energy sources, i.e. consuming energy in stages and ejecting used stages, to progressively reduce the mass of aerial vehicles in-flight which reduces power consumption, and consequently increases flight time. A flight time vs. energy storage mass analysis is presented to show the endurance benefit of staging to multirotors. We consider two specific problems in discrete staging - optimal order of staging given a certain number of energy sources, and optimal partitioning of a given energy storage mass budget into a given number of stages. We then derive results for a continuously staged case of an internal combustion engine driving propellers. Notably, we show that a multirotor powered by internal combustion has an upper limit on achievable flight time independent of the available fuel mass. Lastly, we validate the analysis with flight experiments on a custom two-stage battery-powered quadcopter. This quadcopter can eject a battery stage after consumption in-flight using a custom-designed mechanism, and continue hovering using the next stage. The experimental flight times match well with those predicted from the analysis for our vehicle. We achieve a 19% increase in flight time using the batteries in two stages as compared to a single stage.

Limited flight range is a common problem for multicopters. To alleviate this problem, we propose a method for finding the optimal speed and heading of a multicopter when flying a given path to achieve the longest flight range. Based on a novel multivariable extremum seeking controller with adaptive step size, the method (a) does not require any power consumption model of the vehicle, (b) can adapt to unknown disturbances, (c) can be executed online, and (d) converges faster than the standard extremum seeking controller with constant step size. We conducted indoor experiments to validate the effectiveness of this method under different payloads and initial conditions, and showed that it is able to converge more than 30% faster than the standard extremum seeking controller. This method is especially useful for applications such as package delivery, where the size and weight of the payload differ for different deliveries and the power consumption of the vehicle is hard to model.

Aerial vehicles with collision resilience can operate with more confidence in environments with obstacles that are hard to detect and avoid. This paper presents the methodology used to design a collision resilient aerial vehicle with icosahedron tensegrity structure. A simplified stress analysis of the tensegrity frame under impact forces is performed to guide the selection of its components. In addition, an autonomous controller is presented to reorient the vehicle from an arbitrary orientation on the ground to help it take off. Experiments show that the vehicle can successfully reorient itself after landing upside-down and can survive collisions with speed up to 6.5m/s.

This paper presents a novel quadcopter design that uses passive rotary joints to enable rapid aerial morphing without the use of additional actuators. The normally rigid connections between the arms of the quadcopter and the central body are replaced by sprung hinges that allow for the arms of the quadcopter to fold downward when low thrusts are produced by the propellers, resulting in a reduction of the largest dimension of the vehicle by approximately 50%. The ability of the vehicle to reduce its size during flight allows, e.g., for the traversal of gaps through which a non-morphing quadcopter could not pass. The vehicle is designed such that existing quadcopter controllers and trajectory generation algorithms can be used, provided that some additional constraints on the control inputs are met. The nonlinear dynamics of the system are presented, and design rules are given that minimize transition time between configurations and maximize the available range of control inputs. A method for performing gap traversal maneuvers is proposed and validated experimentally.

In this work we introduce an approach that allows a quadcopter to find the velocity which maximizes its flight time (endurance) or flight distance (range) while moving along a given path, using on-board power measurement. The proposed strategy is based on Extremum Seeking control and (a) does not require any model of the power consumption of the system, (b) can be executed on-line, and (c) guarantees adaptation to unknown disturbances. We show experimentally that hovering is not the most energy-efficient loitering strategy, and we demonstrate the proposed method's ability to adapt to different aerodynamic disturbances, such as payloads. The method may be especially useful in applications where a quadcopter carries an unknown payload, allowing it to adapt for improved range.

We present a continuous-time collision detection algorithm for quickly detecting whether certain polynomial trajectories in time intersect with convex obstacles. The algorithm is used in conjunction with an existing multicopter trajectory generation method to achieve rapid, obstacle-aware motion planning in environments with both static convex obstacles and dynamic convex obstacles whose boundaries do not rotate. In general, this problem is difficult because the presence of convex obstacles makes the feasible space of trajectories nonconvex. The performance of the algorithm is benchmarked using Monte Carlo simulations, and experimental results are presented that demonstrate the use of the method to plan collision-free multicopter trajectories in milliseconds in environments with both static and dynamic obstacles.

This paper presents a novel quadcopter design with an added momentum wheel for enhanced stability. The novel vehicle has improved torque disturbance rejection capabilities compared to a standard quadcopter. An analysis of the vehicle dynamics shows that the effect of torque disturbances decreases monotonically with increasing angular momentum of the momentum wheel. A framework for choosing the mass moment of inertia and speed of the momentum wheel is given based on an upper bound on the allowable energy stored in the wheel. Theoretical results are experimentally validated by comparing responses to torque impulses applied to the vehicle with and without the momentum wheel spinning.

This paper presents the “monospinner”: a mechanically simple flying vehicle with only one moving part. The vehicle is shown to be controllable in three translational degrees of freedom and two rotational degrees of freedom. The vehicle has a single scalar control input, the thrust magnitude, and is controlled by a cascaded control strategy with an inner attitude controller and an outer position controller. The vehicle design is chosen based on two robustness metrics: the ability to maintain hover under perturbations and the probability of input saturation based on a stochastic model. The resulting mechanical and control designs are experimentally demonstrated, where it is also shown that the vehicle is sufficiently robust to achieve hover after being thrown into the air.

A state estimator for a quadrocopter is presented, using measurements from an accelerometer, angular rate gyroscope, and a set of ultra-wideband ranging radios. The estimator uses an extended aerodynamic model for the quadrocopter, where the full 3D airspeed is observable through accelerometer measurements. The remaining quadrocopter states, including the yaw orientation, are rendered observable by fusing ultra-wideband range measurements, under the assumption of no wind. The estimator is implemented on a standard microcontroller using readily-available, low-cost sensors. Performance is experimentally investigated in a variety of scenarios, where the quadrocopter is flown under feedback control using the estimator output.

A method is presented for the rapid generation and feasibility verification of motion primitives for quadrocopters and similar multirotor vehicles. The motion primitives are defined by the quadrocopter's initial state, the desired motion duration, and any combination of components of the quadrocopter's position, velocity, and acceleration at the motion's end. Closed-form solutions for the primitives are given, which minimize a cost function related to input aggressiveness. Computationally efficient tests are presented to allow for rapid feasibility verification. Conditions are given under which the existence of feasible primitives can be guaranteed a priori . The algorithm may be incorporated in a high-level trajectory generator, which can then rapidly search over a large number of motion primitives which would achieve some given high-level goal. It is shown that a million motion primitives may be evaluated and compared per second on a standard laptop computer. The motion primitive generation algorithm is experimentally demonstrated by tasking a quadrocopter with an attached net to catch a thrown ball, evaluating thousands of different possible motions to catch the ball.

This paper presents periodic solutions for a quadrocopter maintaining a height around a position in space despite having lost a single, two opposing, or three propellers. In each case the control strategy consists of the quadrocopter spinning about a primary axis, fixed with respect to the vehicle, and tilting this axis for translational control. A linear, timeinvariant description of deviations from the attitude equilibrium is derived, allowing for a convenient cascaded control design. The results for the cases of losing one and two propellers are validated in experiment, while the case of losing three propellers is validated in a nonlinear simulation. These results have application in multicopter fault-tolerant control design, and also point to possible design directions for novel flying vehicles.

An algorithm is proposed allowing for the rapid generation and evaluation of quadrocopter state interception trajectories. These trajectories are from arbitrary initial states to final states defined by the vehicle position, velocity and acceleration with a specified end of time. Sufficient criteria are then derived allowing trajectories to be tested for feasibility with respect to thrust and body rates. It is also shown that the range of a linear combination of the vehicle state can be solved for in closed form, useful e.g. for testing that the position remains within a box. The algorithm is applied by revisiting the problem of finding a trajectory to hit a ball towards a target with a racket attached to a quadrocopter. The trajectory generator is used in a model predictive control like strategy, where thousands of trajectories are generated and evaluated at every controller update step, with the first input of the optimal trajectory being sent to the vehicle. It is shown that the method can generate and evaluate on the order of one million trajectories per second on a standard laptop computer.

An autonomous safety mechanism is presented, as implemented in an indoor flying vehicle research testbed. The safety mechanism relies on integration of onboard gyroscope measurements and thrust commands to estimate the vehicle state for short lengths of time. It is used in the case of loss of external control signal or loss of external measurement data, to reduce the likelihood of a vehicle crash, or at least reduce the severity of an unavoidable crash. As UAVs move into ever more mainstream applications with increased public interaction, such safety systems become more critical.

This paper presents a method for enabling a fleet of circularly arranged quadrocopters to throw and catch balls with a net. Based on a first-principles model of the net forces, nominal inputs for all involved vehicles are derived for arbitrary target trajectories of the net. Two algorithms that generate open-loop trajectories for throwing and catching a ball are also introduced. A set of throws and catches is demonstrated in the ETH Zurich Flying Machine Arena testbed.

This paper presents a method allowing a quadrocopter with a rigidly attached racket to hit a ball towards a target. An algorithm is developed to generate an open loop trajectory guiding the vehicle to a predicted impact point - the prediction is done by integrating forward the current position and velocity estimates from a Kalman filter. By examining the ball and vehicle trajectories before and after impact, the system estimates the ball's drag coefficient, the racket's coefficient of restitution and an aiming bias. These estimates are then fed back into the system's aiming algorithm to improve future performance. The algorithms are implemented for three different experiments: a single quadrocopter returning balls thrown by a human; two quadrocopters co-operatively juggling a ball back-and-forth; and a single quadrocopter attempting to juggle a ball on its own. Performance is demonstrated in the Flying Machine Arena at the ETH Zurich.