By combining rotary- and fixed-wing flight, hybrid uncrewed aerial vehicles (H-UAVs) can uniquely address missions combining long-range aerial transport and precise ground-relative tasks, such as the placement or retrieval of payloads. However, to leverage their full maneuverability, first, the fundamentally different operating modes of rotary- and fixed-wing vehicles need to be unified and second, the system be controlled precisely despite complex aerodynamic effects. This work presents a general and lightweight, cascaded control formulation for such versatile and accurate operation of H-UAVs. First, a novel guidance law unifies ground- and air-relative position control modes typical for the individual flight regimes. Second, we formulate a jerk-level feedback-linearization to accurately track the guidance outputs despite model errors and disturbances. In extensive real flight tests with a tiltwing H-UAV, we demonstrate the versatile allocation of (hybrid) flight states and the overall accuracy enabled by the control system. Position errors remain below 0.5 m (one quarter of the wingspan) in the full flight envelope, including accelerated maneuvers up to 10 ms2and gusting wind reaching 12 m/s. Finally, the control system demonstrates exploiting hybrid flight for transport-related missions with a precise, in-flight pickup of a payload.

Detecting air flows caused by phenomena such as heat convection is valuable in multiple scenarios, including leak identification and locating thermal updrafts for extending UAV flight duration. Unfortunately, the heat signature of these flows is often too subtle to be seen by a thermal camera. While convection also leads to fluctuations in air density and hence causes so-called schlieren – intensity and color variations in images – existing techniques such as Background-oriented schlieren (BOS) allow detecting them only against a known background and from a static camera, making these approaches unsuitable for moving vehicles. In this work we demonstrate the feasibility of visualizing air movement by predicting the corresponding schlieren-induced optical flow from a single greyscale image captured by a moving camera against an unfamiliar background. We first record and label a set of optical flows in an indoor setup using standard BOS techniques. We then train a convolutional neural network (CNN) by applying the previously collected optical flow distortions to a dataset containing a mixture of real and synthetically generated images to predict the two-dimensional optical flow from a single image. Finally, we evaluate our approach on the task of extracting the optical flow caused by schlieren from both a static and moving camera on previously unseen flow patterns and background images.

Small uncrewed aerial systems (sUASs) are useful tools for 3D reconstruction due to their speed, ease of use, and ability to access high-utility viewpoints. Today, most aerial survey approaches generate a preplanned coverage pattern assuming a planar target region. However, this is inefficient since it results in superfluous overlap and suboptimal viewing angles and does not utilize the entire flight envelope. In this work, we propose active path planning for photogrammetric reconstruction. Our main contribution is a view utility function based on Fisher information approximating the offline reconstruction uncertainty. The metric enables online path planning to make in-flight decisions to collect geometrically informative image data in complex terrain. We evaluate our approach in a photorealistic simulation. A viewpoint selection study shows that our metric leads to faster and more precise reconstruction than state-of-the-art active planning metrics and adapts to different camera resolutions. Comparing our online planning approach to an ordinary fixed-wing aerial survey yields 3.2 × faster coverage of 16 ha undulated terrain without sacrificing precision.

Hybrid unmanned aerial vehicles (H-UAVs) are highly versatile platforms with the ability to transition between rotary- and fixed-wing flight. However, their (aero)dynamics tend to be highly nonlinear which increases the risk of introducing safety-critical modeling errors in a controller. Designing a safe, yet not too cautious controller, requires a credible model which provides accurate dynamics uncertainty quantification. We present a data-efficient, probabilistic semi-parametric dynamics modeling approach that allows for online, filter-based inference. The proposed model leverages prior knowledge using a nominal parametric model, and combines it with residuals in the form of sparse Gaussian processes to account for possibly unmodeled forces and moments. Uncertain nominal and residual parameters are jointly estimated using Bayesian filtering. The resulting model accuracy and the reliability of its predicted uncertainty are analyzed for both a simulated and a real example, where we learn the 6DoF nonlinear dynamics of a tiltwing H-UAV from a few minutes of flight data. Compared to a residual-free nominal model, the proposed semi-parametric approach provides increased model accuracy in relevant parts of the flight envelope and substantially higher credibility overall.

Bilateral teleoperation offers an intriguing solution towards shared autonomy with aerial vehicles in contact-based inspection and manipulation tasks. Omnidirectional aerial robots allow for full pose operations, making them particularly attractive in such tasks. Naturally, the question arises whether standard bilateral teleoperation methodologies are suitable for use with these vehicles. In this work, a fully decoupled 6DoF bilateral teleoperation framework for aerial physical interaction is designed and tested for the first time. The method is based on the well established rate control, recentering and interaction force feedback policy. However, practical experiments evince the difficulty of performing de-coupled motions in a single axis only. As such, this work shows that the trivial extension of standard methods is insufficient for omnidirectional teleoperation, due to the operator's physical inability to properly decouple all input DoFs. This suggests that further studies on enhanced haptic feedback are necessary.

Autonomous navigation among people is a com-plex problem that also exhibits considerable variation depending on the type of environment and people involved. Here we consider navigation among crowds that exhibit flow-like behavior like people moving through a train station. We propose a novel pseudo-fluid model of crowd flow for such problems. These have an intuitive physical interpretation and do not require much tuning. We further formalize an observation model to infer flow properties from discrete sensor observations, including support for partial observability, and pair it with a flow-aware planner. We demonstrate the potential of the approach in simulated navigation scenarios. We achieve state of the art results on the CrowdBot navigation benchmark, and also compare favorably against a standard ROS planner on a partially observable environment, demonstrating that the flow-aware planner successfully estimates and plans around counter-flows in the crowd in real time. We conclude that flow-based planning shows great promise for crowded environments that may exhibit such flow-like behavior.

For asymptotically optimal sampling-based path planners such as RRT*, path quality improves as the number of samples added to the motion tree increases. However, each additional sample requires a nearest-neighbor search. Calculating state transition costs can be particularly difficult in cases with complex dynamics such as aerial vehicles in non-isotropic cost fields like wind. Computationally costly nearest neighbor searches increase the time required to add new samples to the search tree, thereby reducing the likelihood of finding low-cost paths in a given computational time. In this paper, we propose the use of a lightweight neural network to approximate nearest neighbor cost calculations. The network approach uses a low-dimensional encoding of the cost space along with a start and goal query pair and returns an estimate of the path cost that can be used for nearest neighbor and path validity estimation. We demonstrate our method for a Dubins airplane model in a 3D wind field and show that the network method achieves equivalent path lengths as an existing iterative solver 32% faster and, when given the same search time, up to 10.8% shorter.

This paper presents an online informative path planning approach for active information gathering on three-dimensional surfaces using aerial robots. Most existing works on surface inspection focus on planning a path offline that can provide full coverage of the surface, which inherently assumes the surface information is uniformly distributed hence ignoring potential spatial correlations of the information field. In this paper, we utilize manifold Gaussian processes (mGPs) with geodesic kernel functions for mapping surface information fields and plan informative paths online in a receding horizon manner. Our approach actively plans information-gathering paths based on recent observations that respect dynamic constraints of the vehicle and a total flight time budget. We provide planning results for simulated temperature modeling for simple and complex 3D surface geometries (a cylinder and an aircraft model). We demonstrate that our informative planning method outperforms traditional approaches such as 3D coverage planning and random exploration, both in reconstruction error and information-theoretic metrics. We also show that by taking spatial correlations of the information field into planning using mGPs, the information gathering efficiency is significantly improved.

Obtaining an accurate estimate of the local wind remains a significant challenge for small unmanned aerial vehicles (UAVs). Small UAVs often operate at low altitudes near terrain, where the wind environment can be more complex than at higher altitudes. Combined with their relatively low mass, this makes small UAVs particularly susceptible to wind. In this paper we present an approach for predicting high-resolution wind fields based on a terrain elevation model and known inflow conditions. Our approach uses a deep convolutional neural network (CNN) to generate 3D wind estimates. We show that our approach produces wind estimates with lower prediction error than existing methods, and that inference can be performed on an on-board computer in less than two seconds. By providing the wind estimate to a sampling-based planner we show that the improved estimates allow the planner to generate safer paths in strong wind scenarios than with alternative wind estimation techniques.

The informative soaring (IFS) problem involves a gliding unmanned aerial vehicle (UAV) exploiting energy from thermals to extend its information gathering capability. In this paper, we address the realistic situation of detecting new thermals drifting with the wind in the search environment. We consider complex target-search scenarios characterised by information clusters and propose a new set of algorithms designed to both explore for and exploit high-value thermals to maximise information gain. Our algorithms: 1) compute a thermal exploration map to detect useful thermals that eventually intercept clusters, 2) solve a boundary value problem for inter-thermal path segment (ITP) generation with moving thermals, 3) compute thermal time windows to gather information from clusters and form a cluster service schedule, and 4) use branch and bound (BnB) tree search for global planning, considering high-utility-rate ITPs to maximise information gain. Our solution is compared against a greedy method that neither considers the thermal exploration map nor cluster schedule and a full knowledge method that has access to all thermals. Numerical simulations show that on average, our solution outperforms the greedy method in one-third of 2400 Monte Carlo trials, and achieves similar performance to the full knowledge method when environmental conditions are favourable.

Robots often operate in built environments containing underlying structure that can be exploited to help predict future observations. In this work, we present a deep learning based approach to predict exit locations of buildings. This technique exploits the inherent structure of buildings to create a model. A convolutional neural network is trained using a database of building blueprints and used to guide a search within a building. This technique is compared to standard frontier exploration and a traditional image processing approach of extracting features through histogram of gradients (HOG) and training a support vector machine (SVM). After validation through simulation, we show that the proposed deep learning technique reduces the amount of building exploration required to find the goal by 36%.

Thermal soaring can vastly increase the effectiveness of Unmanned Aerial Vehicles (UAVs) in information gathering tasks. However, knowing when to best collect information or regain energy is non-trivial. In this work, the problem is posed as a graph search problem where nodes are thermal positions, and edges are inter-thermal trajectories. Previous work has shown that this search problem is NP-hard, such that computing a long-duration plan is difficult without significant computational effort. This paper introduces two mechanisms to make this tractable. Firstly, Monte Carlo Tree Search (MCTS) is used to provide an anytime search strategy capable of generating long plans without exhaustive search. Secondly, a novel clustering approach isolates areas of interest on the information map to solve local cluster subproblems, followed by dynamic programming to optimally allocate search time to each cluster. Results demonstrate the improved performance of these approaches on longer missions.

Exploiting wind currents in the environment allows autonomous gliders to gain altitude and energy and consequently extend flight duration. This paper considers the problem of a persistent monitoring mission using multiple autonomous gliders and exploiting thermal soaring. Communications constraints and non-homogeneous teams of gliders are considered. A distributed method based on coordination variables is proposed to monitor the area in a cooperative manner by following a partitioning patrolling strategy. A distributed one-to-one coordination technique is used to manage the gliders' access to thermals according to their states and known thermal locations. Gliders perform a model-based estimation about their energy losses between thermals to estimate the optimal time to remain in a thermal to maintain persistent surveillance with minimum refresh time. Simulated test results are provided to evaluate how the proposed approach is able to extend the mission while maintaining near-optimal performance.

This paper presents the iGP-SARSA(λ) algorithm for temporal difference reinforcement learning (RL) with non-myopic information gain considerations. The proposed algorithm uses a Gaussian process (GP) model to approximate the state-action value function, Q, and incorporates the variance measure from the GP into the calculation of the discounted information gain value for all future state-actions rolled out from the current state-action. The algorithm was compared against a standard SARSA(λ) algorithm on two simulated examples: a battery charge/discharge problem, and a soaring glider problem. Results show that incorporating the information gain value into the action selection encouraged exploration early on, allowing the iGP-SARSA(λ) algorithm to converge to a more profitable reward cycle, while the e-greedy exploration strategy in the SARSA(λ) algorithm failed to search beyond the local optimal solution.

Autonomous aerial soaring presents a unique opportunity to extend the flight duration of Unmanned Aerial Vehicles (UAVs). In this paper, we examine the problem of a gliding UAV searching for a ground target while simultaneously collecting energy from known thermal energy sources. The problem is posed as a tree search problem by noting that a long-duration mission can be divided into similar segments of flying between and climbing in thermals. The algorithm attempts to maximise the probability of detecting a target by exploring a tree of the possible thermal-to-thermal transitions to a fixed search depth and executing the highest utility plan. The sensitivity of the algorithm to different search depths is explored, and the method is compared against a locally-optimal myopic search algorithm. In larger, more complicated problems, the suggested method outperforms myopic search by sacrificing short-term utility to reach more valuable exploration areas later in the mission.

An autonomous aircraft capable of utilising soaring flight in a dynamic wind field could considerably extend flight duration by limiting the use of on-board energy for propulsion. While soaring flight is relatively well understood for known wind, an autonomous soaring aircraft would have to generate paths based only on local observations of the wind made during the flight. This paper presents a method to simultaneously map and utilise a wind field using Gaussian process regression to generate a spatio-temporal map of the wind, and a path planning and dynamic target assignment algorithm to generate energy-gain paths from the current wind estimate. The planning architecture is tested in simulation for dynamic wind fields and shows consistent energy gain through exploration and exploitation of the wind environment.

Soaring is the process of gaining energy from the atmosphere in-flight using an aerodynamic free-flying platform. Dynamic soaring utilizes the energy available in vertical wind gradients and is commonly used by soaring birds. This research aims to develop a guidance and control strategy to utilize dynamic soaring for a fixed-wing gliding UAV. The basic strategies for dynamic soaring in vertical wind shear are explored and a simple piecewise trajectory based controller is developed to identify regions suitable for soaring and attempt traveling energy-neutral trajectories.