The efficient collection of samples is an important factor in outdoor information gathering applications on account of high sampling costs such as time, energy, and potential destruction to the environment. Utilization of available a-priori data can be a powerful tool for increasing efficiency. However, the relationships of this data with the quantity of interest are often not known ahead of time, limiting the ability to leverage this knowledge for improved planning efficiency. To this end, this work combines transfer learning and active learning through a Multi-Task Gaussian Process and an information-based objective function. Through this combination it can explore the space of hypothetical inter-quantity relationships and evaluate these hypotheses in real-time, allowing this new knowledge to be immediately exploited for future plans. The performance of the proposed method is evaluated against synthetic data and is shown to evaluate multiple hypotheses correctly. Its effectiveness is also demonstrated on real datasets. The technique is able to identify and leverage hypotheses which show a medium or strong correlation to reduce prediction error by a factor of 1.4–3.4 within the first 7 samples, and poor hypotheses are quickly identified and rejected eventually having no adverse effect.

Deep learning-based video segmentation methods can offer a good performance after being trained on the large-scale pixel labeled datasets. However, a pixel-wise manual labeling of animal images is challenging and time consuming due to irregular contours and motion blur. To achieve desirable tradeoffs between the accuracy and speed, a novel one-shot learning-based approach is proposed in this article to segment animal video with only one labeled frame. The proposed approach consists of the following three main modules: guidance frame selection utilizes “BubbleNet” to choose one frame for manual labeling, which can leverage the fine-tuning effects of the only labeled frame; Xception-based fully convolutional network localizes dense prediction using depthwise separable convolutions based on one single labeled frame; and postprocessing is used to remove outliers and sharpen object contours, which consists of two submodules—test time augmentation and conditional random field. Extensive experiments have been conducted on the DAVIS 2016 animal dataset. Our proposed video segmentation approach achieved mean intersection-over-union score of 89.5% on the DAVIS 2016 animal dataset with less run time, and outperformed the state-of-art methods (OSVOS and OSMN). The proposed one-shot learning-based approach achieves real-time and automatic segmentation of animals with only one labeled video frame. This can be potentially used further as a baseline for intelligent perception-based monitoring of animals and other domain-specific applications.11The source code, datasets, and pre-trained weights for this work are publicly [Online]. Available:https://github.com/tengfeixue-victor/One-Shot-Animal-Video-Segmentation.

Trajectory optimization of sensing robots to actively gather information of targets has received much attention in the past. It is well-known that under the assumption of linear Gaussian target dynamics and sensor models the stochastic Active Information Acquisition problem is equivalent to a deterministic optimal control problem. However, the above-mentioned assumptions regarding the target dynamic model are limiting. In real-world scenarios, the target may be subject to disturbances whose models or statistical properties are hard or impossible to obtain. Typical scenarios include abrupt maneuvers, jumping disturbances due to interactions with the environment, anomalous misbehaviors due to system faults/attacks, etc. Motivated by the above considerations, in this paper we consider targets whose dynamic models are subject to arbitrary unknown inputs whose models or statistical properties are not assumed to be available. In particular, with the aid of an unknown input decoupled filter, we formulate the sensor trajectory planning problem to track evolution of the target state and analyse the resulting performance for both the state and unknown input evolution tracking. Inspired by concepts of Reduced Value Iteration, a suboptimal solution that expands a search tree via Forward Value Iteration with informativeness-based pruning is proposed. Concrete suboptimality performance guarantees for tracking both the state and the unknown input are established. Numerical simulations of a target tracking example are presented to compare the proposed solution with a greedy policy.

Sensor array-based systems, which adopt time difference of arrival (TDOA) measurements among the sensors, have found many robotic applications. However, for existing frameworks and systems to be useful, the sensor array needs to be calibrated accurately. Of particular interest in this article are microphone array-based robot audition systems. In our recent work, by using a moving sound source, and the graph-based formulation of simultaneous localization and mapping (SLAM), we have proposed a framework for joint sound source localization and calibration of microphone array geometrical information, together with the estimation of microphone time offset and clock difference/drift rates. However, a thorough study on the identifiability question, termed observability analysis here, in the SLAM framework for microphone array calibration and sound source localization, is still lacking in the literature. In this article, we will fill the abovementioned gap via a Fisher information matrix approach. Motivated by the equivalence between the full column rankness of the Fisher information matrix and the Jacobian matrix, we leverage the structure of the latter associated with the SLAM formulation, and present necessary and sufficient conditions guaranteeing its full column rankness, which lead to parameter identifiability. We have thoroughly discussed the 3-D case with asynchronous (with both time offset and clock drifts, or with only one of them) and synchronous microphone array, respectively. These conditions are closely related to the motion varieties of the sound source and the microphone array configuration, and have intuitive and physical interpretations. Based on the established conditions, we have also discovered some particular cases where observability is impossible. Connections with calibration of other sensors will also be discussed, amongst others. To our best knowledge, this is the first systematic work on observability analysis of SLAM-based microphone array calibration and sound source localization. The tools and concepts used in this article are also applicable to other TDOA sensing modalities such as ultrawide band (UWB) sensors.

State of the art methods for robotic path planning in dynamic environments, such as crowds or traffic, rely on hand crafted motion models for agents. These models often do not reflect interactions of agents in real world scenarios. To overcome this limitation, this paper proposes an integrated path planning framework using generative Recurrent Neural Networks within a Monte Carlo Tree Search (MCTS). This approach uses a learnt model of social response to predict crowd dynamics during planning across the action space. This extends our recent work using generative RNNs to learn the relationship between planned robotic actions and the likely response of a crowd. We show that the proposed framework can considerably improve motion prediction accuracy during interactions, allowing more effective path planning. The performance of our method is compared in simulation with existing methods for collision avoidance in a crowd of pedestrians, demonstrating the ability to control future states of nearby individuals. We also conduct preliminary real world tests to validate the effectiveness of our method.

The control of field robots in varying and uncertain terrain conditions presents a challenge for autonomous navigation. Online estimation of the wheel-terrain slip characteristics is essential for generating the accurate control predictions necessary for tracking trajectories in off-road environments. Receding horizon estimation (RHE) provides a powerful framework for constrained estimation, and when combined with receding horizon control (RHC), yields an adaptive optimisation-based control method. Presently, such methods assume slip to be constant over the estimation horizon, while our proposed structured blocking approach relaxes this assumption, resulting in improved state and parameter estimation. We demonstrate and compare the performance of this method in simulation, and propose an overlapping-block strategy to ameliorate some of the limitations encountered in applying noise-blocking in a receding horizon estimation and control (RHEC) context.

Soft strain resistive sensors based on eutectic gallium-indium liquid metal can play an important role in proprioceptive sensing for soft robots. However, there are no available mathematical models to accurately estimate the strain as a function of the measured resistance. Furthermore, non-uniform strain in the microchannels has not been analysed yet. In this paper, we introduce a new model to estimate the strain or elongation in sub-millimetre scale, and analyse its accuracy through a customised testing set-up and procedure. The effect of strain rate on the measurement accuracy is also studied. We compare existing theoretical models with our experimental results, and discuss the differences between them. Moreover, we analyse the effect of strain rate on hysteresis caused by the viscoelastic behaviour and introduce a new model for it to be potentially used for future work. This paper demonstrates, among other things, that rational models could provide high accuracy in strain estimation, and might help to enhance proprioceptive sensing and state control of soft robots.

Autonomous Science is a field of study which aims to extend the autonomy of exploration robots from low level functionality, such as on-board perception and obstacle avoidance, to science autonomy, which allows scientists to specify missions at task level. This will enable more remote and extreme environments such as deep ocean and other planets to be studied, leading to significant science discoveries. This paper presents an approach to extend the high level autonomy of robots by enabling them to model and reason about scientific knowledge on-board. We achieve this by using Bayesian networks to encode scientific knowledge and adapting Monte Carlo Tree Search techniques to reason about the network and plan informative sensing actions. The resulting knowledge representation and reasoning framework is anytime, handles large state spaces and robust to uncertainty making it highly applicable to field robotics. We apply the approach to a Mars exploration mission in which the robot is required to plan paths and decide when to use its sensing modalities to study a scientific latent variable of interest. Extensive simulation results show that our approach has significant performance benefits over alternative methods. We also demonstrate the practicality of our approach in an analog Martian environment where our experimental rover, Continuum, plans and executes a science mission autonomously.

The informative soaring (IFS) problem involves a gliding unmanned aerial vehicle (UAV) exploiting energy from thermals to extend its information gathering capability. In this paper, we address the realistic situation of detecting new thermals drifting with the wind in the search environment. We consider complex target-search scenarios characterised by information clusters and propose a new set of algorithms designed to both explore for and exploit high-value thermals to maximise information gain. Our algorithms: 1) compute a thermal exploration map to detect useful thermals that eventually intercept clusters, 2) solve a boundary value problem for inter-thermal path segment (ITP) generation with moving thermals, 3) compute thermal time windows to gather information from clusters and form a cluster service schedule, and 4) use branch and bound (BnB) tree search for global planning, considering high-utility-rate ITPs to maximise information gain. Our solution is compared against a greedy method that neither considers the thermal exploration map nor cluster schedule and a full knowledge method that has access to all thermals. Numerical simulations show that on average, our solution outperforms the greedy method in one-third of 2400 Monte Carlo trials, and achieves similar performance to the full knowledge method when environmental conditions are favourable.

Actively articulated wheel-on-leg rovers offer high degrees of mobility for traversing unstructured and scientifically interesting terrain. An ability to actively conform to the terrain to increase traversability is sought. A kinematic model based on recursive kinematic propagation in combination with a terrain map generated online is used to solve desired limb articulation angles that meet operator defined rover body trajectories. This technique is validated in field tests on a Martian analog terrain.

Multi-robot information gathering teams typically require communication for data fusion and cooperative decision making. However, when communication takes place over wireless networks, stringent bandwidth limits apply. These limits raise the need for efficient utilisation of available communication resources in a manner that balances information gathering utility with communication costs or limits. In our previous work, we introduced the dynamic information flow (DIF) problem as a general formulation of this trade-off. We introduced two variants of the problem addressing the issue of communication efficiency for data fusion only. In this paper, we extend one of the variants to address communication efficiency for both data fusion and cooperative decision making in a synergistic manner. We present a solution to this new variant that integrates a multi-cast routing algorithm with information structure optimisation. This solution allows teams that involve high-data-rate sensors and tight coordination to respect bandwidth limits. Through several simulations we verify that our solution significantly reduces bandwidth usage of such teams while maintaining information gathering performance.

Autonomous docking for airborne energy transfer is an important unmanned aerial vehicle capability that has yet to be accomplished. The implications of this technology are far reaching because vehicle endurance can be significantly extended without requiring additional onboard energy storage or environmental energy collection. A major barrier to docking with a drogue is reliable and accurate knowledge of the drogue's state, relative to the aircraft that is performing the manoeuvre. We address this by propagating a derived cable-drogue dynamic model using onboard sensor measurements, then correcting the state and aerodynamic coefficients with air-to-air visual observations. The approach is verified through a series of close formation flights with a marker based infra-red vision system. These flights represent significant progress toward an autonomous docking demonstration.

Thermal soaring can vastly increase the effectiveness of Unmanned Aerial Vehicles (UAVs) in information gathering tasks. However, knowing when to best collect information or regain energy is non-trivial. In this work, the problem is posed as a graph search problem where nodes are thermal positions, and edges are inter-thermal trajectories. Previous work has shown that this search problem is NP-hard, such that computing a long-duration plan is difficult without significant computational effort. This paper introduces two mechanisms to make this tractable. Firstly, Monte Carlo Tree Search (MCTS) is used to provide an anytime search strategy capable of generating long plans without exhaustive search. Secondly, a novel clustering approach isolates areas of interest on the information map to solve local cluster subproblems, followed by dynamic programming to optimally allocate search time to each cluster. Results demonstrate the improved performance of these approaches on longer missions.

Unmanned aerial vehicle (UAV) formation flight can vastly increase operational range and persistence through autonomous aerial refuelling or efficient flight on a wingman's wake vortices. Differencing individual UAV state estimates is not sufficiently accurate for close formation operations and must be augmented with vehicle-to-vehicle observations. To this end, we propose a quaternion based unscented Kalman filter to fuse information from each UAV sensor suite with relative vision observations. The result is a vastly improved relative state estimate that is resilient to brief vision dropouts and degrades gracefully during extended dropouts. Simulated formation flight results validate the approach and provide a numerical analysis of the algorithm performance. Ground based experiments demonstrate the algorithm running in real-time on a dual-UAV system. This represents a significant step towards an airborne implementation.

Exploiting wind currents in the environment allows autonomous gliders to gain altitude and energy and consequently extend flight duration. This paper considers the problem of a persistent monitoring mission using multiple autonomous gliders and exploiting thermal soaring. Communications constraints and non-homogeneous teams of gliders are considered. A distributed method based on coordination variables is proposed to monitor the area in a cooperative manner by following a partitioning patrolling strategy. A distributed one-to-one coordination technique is used to manage the gliders' access to thermals according to their states and known thermal locations. Gliders perform a model-based estimation about their energy losses between thermals to estimate the optimal time to remain in a thermal to maintain persistent surveillance with minimum refresh time. Simulated test results are provided to evaluate how the proposed approach is able to extend the mission while maintaining near-optimal performance.

This paper presents the iGP-SARSA(λ) algorithm for temporal difference reinforcement learning (RL) with non-myopic information gain considerations. The proposed algorithm uses a Gaussian process (GP) model to approximate the state-action value function, Q, and incorporates the variance measure from the GP into the calculation of the discounted information gain value for all future state-actions rolled out from the current state-action. The algorithm was compared against a standard SARSA(λ) algorithm on two simulated examples: a battery charge/discharge problem, and a soaring glider problem. Results show that incorporating the information gain value into the action selection encouraged exploration early on, allowing the iGP-SARSA(λ) algorithm to converge to a more profitable reward cycle, while the e-greedy exploration strategy in the SARSA(λ) algorithm failed to search beyond the local optimal solution.

Learning the flight model for an Unmanned Aerial Vehicle (UAV) involves estimating stability and control parameters from flight data. A non-parametric approach to perform this task is to use Dependent Gaussian Processes (DGPs). It has many benefits, including not having to know a prior model structure, captures any dependencies embodied in the outputs and learns the system noise. However, the main drawback of this approach is the heavy computational cost which makes it prohibitive to learn the model from large test flight data sets. In addition, DGPs do not capture any non-stationary behavior in the aerodynamic coefficients. This paper presents a novel approach to address these issues while maintaining all the other benifits that was gained using DGPs. The proposed algorithm uses an additive sparse model that combines global and local Gaussian processes to learn a multi-output system. We demonstrate that having a combined approximation makes the model suitable for all regions of the flight envelope. To capture the global properties we introduce a new sampling method to gather information about the output correlations. Local properties were captured using a non-stationary covariance function with KD-trees for neighbourhood selection. This makes the model scalable to learn from high dimensional large-scale data sets. Finally, the method explained in this paper was demonstrated in several examples using real flight tests from a delta-winged UAV.

A critical requirement for safe autonomous navigation of a planetary rover is the ability to accurately estimate the traversability of the terrain. This work considers the problem of predicting the attitude and configuration angles of the platform from terrain representations that are often incomplete due to occlusions and sensor limitations. Using Gaussian Processes (GP) and exteroceptive data as training input, we can provide a continuous and complete representation of terrain traversability, with uncertainty in the output estimates. In this paper, we propose a novel method that focuses on exploiting the explicit correlation in vehicle attitude and configuration during operation by learning a kernel function from vehicle experience to perform GP regression. We provide an extensive experimental validation of the proposed method on a planetary rover. We show significant improvement in the accuracy of our estimation compared with results obtained using standard kernels (Squared Exponential and Neural Network), and compared to traversability estimation made over terrain models built using state-of-the-art GP techniques.

Autonomous aerial soaring presents a unique opportunity to extend the flight duration of Unmanned Aerial Vehicles (UAVs). In this paper, we examine the problem of a gliding UAV searching for a ground target while simultaneously collecting energy from known thermal energy sources. The problem is posed as a tree search problem by noting that a long-duration mission can be divided into similar segments of flying between and climbing in thermals. The algorithm attempts to maximise the probability of detecting a target by exploring a tree of the possible thermal-to-thermal transitions to a fixed search depth and executing the highest utility plan. The sensitivity of the algorithm to different search depths is explored, and the method is compared against a locally-optimal myopic search algorithm. In larger, more complicated problems, the suggested method outperforms myopic search by sacrificing short-term utility to reach more valuable exploration areas later in the mission.

Fixed-wing Unmanned Aerial Vehicle (UAV) rendezvous is necessary for reduced fuel consumption during leader-follower formation flight. We propose a heuristic direct search algorithm that plans a time-optimal path for a follower UAV to rendezvous with a leader whose future path is known and unchanging. The kinematic constraints of the UAV are considered and discontinuities inherent to the minimum-length paths are dealt with. Experiments using quadrotors to emulate fixed-wings, demonstrate the algorithm planning and replanning optimal paths to rendezvous in real-time.

This paper addresses the coordination of a decentralised robot team for target tracking. In many approaches to coordination, robots jointly plan their actions through negotiation, which incurs communication costs. Previous work examined the use of learning to reduce the need for negotiations in a network of static robots. Robots incrementally learn how each team member impacts the team utility and can thus make coordinated, team-wide decisions. In this paper, we extend the concept of learning utility models to a team of mobile robots. We also propose a mechanism by which robots switch between negotiating and using the learnt model. This mechanism reduces the communications required for coordination whilst maintaining the same level of tracking performance. Hardware experiments demonstrated that our approach resulted in coordinated behaviours while only negotiating intermittently. Simulation results show that our approach reduced the data communicated for negotiations by up to 70%, without making a statistically significant impact on the tracking performance.

Formal methods based on the Markov decision process formalism, such as probabilistic computation tree logic (PCTL), can be used to analyse and synthesise control policies that maximise the probability of mission success. In this paper, we consider a different objective. We wish to minimise time-to-completion while satisfying a given probabilistic threshold of success. This important problem naturally arises in motion planning for outdoor robots, where high quality mobility prediction methods are available but stochastic path planning typically relies on an arbitrary weighted cost function that attempts to balance the opposing goals of finding safe paths (minimising risk) while making progress towards the goal (maximising reward). We propose novel algorithms for model checking and policy synthesis in PCTL that (1) provide a quantitative measure of safety and completion time for a given policy, and (2) synthesise policies that minimise completion time with respect to a given safety threshold. We provide simulation results in a stochastic outdoor navigation domain that illustrate policies with varying levels of risk.

This paper presents a cooperative system architecture that extends the flight duration of multiple gliding fixed-wing Unmanned Aerial Vehicles (UAVs) for long endurance missions. The missions are defined by a set of Points of Interest (PoI) and UAVs should pass through them. A module to detect and identify thermals is implemented to exploit their energy and extend the flight duration, known as static soaring. A collision-free trajectory planner based on the RRT∗ (Optimal Rapidly-exploring Random Trees) planning algorithm is implemented. The proposed system allows applications in real time because of its low computational needs. Simulations and experiments carried out in the airfield of La Cartuja (Seville, Spain) show the performance and advantages of the proposed system.

It is well recognized that many scientifically interesting sites on Mars are located in rough terrains. Therefore, to enable safe autonomous operation of a planetary rover during exploration, the ability to accurately estimate terrain traversability is critical. In particular, this estimate needs to account for terrain deformation, which significantly affects the vehicle attitude and configuration. This paper presents an approach to estimate vehicle configuration, as a measure of traversability, in deformable terrain by learning the correlation between exteroceptive and proprioceptive information in experiments. We first perform traversability estimation with rigid terrain assumptions, then correlate the output with experienced vehicle configuration and terrain deformation using a multi-task Gaussian Process (GP) framework. Experimental validation of the proposed approach was performed on a prototype planetary rover and the vehicle attitude and configuration estimate was compared with state-of-the-art techniques. We demonstrate the ability of the approach to accurately estimate traversability with uncertainty in deformable terrain.

This paper presents a multi-class image segmentation approach to automate fruit segmentation. A feature learning algorithm combined with a conditional random field is applied to multi-spectral image data. Current classification methods used in agriculture scenarios tend to use hand crafted application-based features. In contrast, our approach uses unsupervised feature learning to automatically capture most relevant features from the data. This property makes our approach robust against variance in canopy trees and therefore has the potential to be applied to different domains. The proposed algorithm is applied to a fruit segmentation problem for a robotic agricultural surveillance mission, aiming to provide yield estimation with high accuracy and robustness against fruit variance. Experimental results with data collected in an almond farm are shown. The segmentation is performed with features extracted from multi-spectral (colour and infrared) data. We achieve a global classification accuracy of 88%.

The stability and control derivatives of an unmanned aerial vehicle (UAV) map the platform's control inputs to its dynamic response. The modeling is labor intensive and requires coarse approximations. Similarly, models constructed through flight tests are only applicable to a narrow flight envelope, and classical system identification approaches require prior knowledge of the model structure, which, in some instances, may only be partially known. The goal of this study is to tackle these problems by introducing a new system identification method based on the dependent Gaussian processes. This allows high-fidelity nonlinear flight dynamic models to be constructed through experimental data. The proposed algorithm captures the cross coupling between input parameters and learns the system stability and control derivatives. In addition, it captures any dependences embodied in the outputs. This paper provides both the theoretical underpinnings and practical application of this approach. The theory was tested in simulation on a highly coupled oblique wing aircraft and was demonstrated on a delta-wing UAV platform using real flight data. The results are compared against an alternative parameteric model and show improvements in identifying the coupling between flight modes, the ability to provide uncertainty estimates and robustness, and applicability to a broader flight envelope.

We introduce a new technique for coordinating teams of unmanned aerial vehicles (UAVs) when deployed to collect live aerial imagery of the scene of a disaster. We define this problem as one of task assignment where the UAVs dynamically coordinate over tasks representing the imagery collection requests. To measure the quality of the assignment of one or more UAVs to a task, we propose a novel utility function which encompasses several constraints, such as the task's importance and the UAVs' battery capacity so as to maximise performance. We then solve the resulting optimisation problem using a fully asynchronous and decentralised implementation of the max-sum algorithm, a well known message passing algorithm previously used only in simulated domains. Finally, we evaluate our approach both in simulation and on real hardware. First, we empirically evaluate our utility and show that it yields a better trade off between the quantity and quality of completed tasks than similar utilities that do not take all the constraints into account. Second, we deploy it on two hexacopters and assess its practical viability in the real world.

This paper addresses the problem of coordinating a team of mobile autonomous sensor agents performing a cooperative mission while explicitly avoiding inter-agent collisions in a team negotiation process. Many multi-agent cooperative approaches disregard the potential hazards between agents, which are an important aspect to many systems and especially for airborne systems. In this work, team negotiation is performed using a decentralized gradient-based optimization approach whereas safety distance constraints are specifically designed and handled using Lagrangian multiplier methods. The novelty of our work is the demonstration of a decentralized form of inter-agent collision avoidance in the loop of the agents' real-time group mission optimization process, where the algorithm inherits the properties of performing its original mission while minimizing the probability of inter-agent collisions. Explicit constraint gradient formulation is derived and used to enhance computational advantage and solution accuracy. The effectiveness and robustness of our algorithm has been verified in a simulated environment by coordinating a team of UAVs searching for targets in a large-scale environment.

This paper introduces an unsupervised graph cut based object segmentation algorithm, ShadowCut, for robotic aerial surveillance applications. By exploiting the spatial setting of the aerial imagery, ShadowCut algorithm differs from state-of-the-art object segmentation algorithms ([1] [2] [3] [4] [5]) by not requiring a large number of labelled training data set, nor constant user interaction ([6] [7] [8]). In this paper it is shown that, by combining robotic navigation data and a shadow model, it is possible to provide these seed labels with a probabilistic sampling model for object segmentation in aerial imagery. Experiments were performed on aerial data sets consisting of data collected in outback Australia with an aerial robotic platform during an ecological surveillance mission, and aerial images with various natural targets from Google Earth. The segmentation results from the unsupervised ShadowCut algorithm are shown to be comparable with those from supervised graph cut algorithms.

Advantages of decentralised decision making systems for multi-agent robotic tasks are limited by the heavy demand they impose on communication. This paper presents an approach to control communication for the LQ team problem, namely a team of agents with linear dynamics and quadratic team cost. Communication costs are added to the objective of the LQ optimal control linear matrix inequality formulation, allowing for a well-defined balancing of communication costs and team performance. Results show a reduction in communication consistent with the specified cost and in a manner that upholds team performance relative to the reduced communication footprint. The applicability of the approach has also been extended to information gathering tasks through local LQ approximations along the agents' paths. Simulation testing on a sample two-agent problem shows a 40% reduction in communication with negligible impact on performance.

In decentralised target tracking, a set of sensors observes moving targets. When the sensors are static but steerable, each sensor must dynamically choose which target to observe in a decentralised manner. We show that the information exchanged by the sensors to synchronise their beliefs can be exploited to learn a model of the utility function that drives each others' decisions. Instead of communicating utilities to enable negotiation, each sensor regresses on the learnt model to predict the utilities of other team members. This approach bridges the gap between coordinating implicitly, a locally-greedy solution, and negotiating explicitly. We validated our approach in both hardware and simulations, and found that it out-performed implicit coordination by a statistically significant margin with both ideal and limited communications.

This paper presents a novel approach to depth estimation from monocular images that is based on the Gaussian Derivative Process (GDP) formulation. We use an inverse depth parametrisation and learn the mapping from image pixel coordinates to the inverse depth of the corresponding scene point given an estimate of the relative camera motion. We show that information about the geometry of the measurements can be integrated into Gaussian Process (GP) models and learnt jointly with the measurements. We provide a novel formulation of the inverse depth and its derivatives and learn their joint distribution. Experimental results are presented using synthesised examples and real monocular images captured from an Unmanned Aerial Vehicle (UAV). Results show improvement in depth estimation over standard Gaussian Process Regression (GPR). This improvement is presented by a reduction in the GP depth prediction errors and the predictive variance. Finally, we show mathematically that this improvement is due to the augmented derivative covariance terms and the correlations between the inverse depth and the derivatives.

This paper presents a new data driven utility function for an unmanned aerial vehicle (UAV) mapping and exploiting a wind field. The proposed utility function provides a continuous scale between exploration and exploitation which is dependent on the difference between the current platform energy level and the uncertainty along a planned path. Tests were carried out in a VICON testbed using quadrotors programmed to emulate fixed-wing aircraft. Results show a 47.7% reduction in energy gain loitering time when compared to a pure information gain approach.

Motion planning for planetary rovers must consider control uncertainty in order to maintain the safety of the platform during navigation. Modelling such control uncertainty is difficult due to the complex interaction between the platform and its environment. In this paper, we propose a motion planning approach whereby the outcome of control actions is learned from experience and represented statistically using a Gaussian process regression model. This model is used to construct a control policy for navigation to a goal region in a terrain map built using an on-board RGB-D camera. The terrain includes flat ground, small rocks, and non-traversable rocks. We report the results of 200 simulated and 35 experimental trials that validate the approach and demonstrate the value of considering control uncertainty in maintaining platform safety.

In this paper, we present a novel method to fuse observations from an inertial measurement unit (IMU) and visual sensors, such that initial conditions of the inertial integration, including gravity estimation, can be recovered quickly and in a linear manner, thus removing any need for special initialization procedures. The algorithm is implemented using a graphical simultaneous localization and mapping like approach that guarantees constant time output. This paper discusses the technical aspects of the work, including observability and the ability for the system to estimate scale in real time. Results are presented of the system, estimating the platforms position, velocity, and attitude, as well as gravity vector and sensor alignment and calibration on-line in a built environment. This paper discusses the system setup, describing the real-time integration of the IMU data with either stereo or monocular vision data. We focus on human motion for the purposes of emulating high-dynamic motion, as well as to provide a localization system for future human-robot interaction.

This paper presents a novel contribution to the problem of coordinating a team of autonomous sensor agents searching for targets in a large scale environment. Team negotiation is performed using a decentralized gradient-based optimization algorithm. Conventional approaches use finite differencing to approximate the gradient information that is computationally less efficient and exposes the gradient-based optimizer to potential numerical errors and instability. The novelty of our work is the explicit formulation of the gradient for the target search problem that significantly enhances the efficiency in gradient evaluation and robustness for the gradient-based optimization algorithm. We present results by firstly showing the computational advantage and robustness of this explicit gradient model against the finite differencing approach and further demonstrate its application in simulation by coordinating multiple UAVs searching a large scale environment in a decentralized network.

A mathematical model for a complex system such as an Unmanned Aerial Vehicle (UAV) requires estimation of aerodynamic, inertial and structural properties of the many elements of the platform. This physical modeling approach is labor intensive and requires coarse approximations to be made in calculations. Similarly, models constructed through flight tests are only applicable to a narrow flight envelope and classical system identification approaches require prior knowledge of the model structure, which in some instance may only be partially known. To tackle these problems, we introduce a novel aircraft system identification method based on dependent Gaussian processes. The approach allows high fidelity non linear flight dynamic models to be constructed through flight testing. The proposed algorithm learns the system parameters as well as captures any dependencies between them. The method is demonstrated by generating a model of the force and moment coefficients for the Brumby Mklll UAV from real flight data. The learnt dynamic model identifies coupling between flight modes, provides an estimate of uncertainty, and is applicable to a broader range of the flight envelope.

An autonomous aircraft capable of utilising soaring flight in a dynamic wind field could considerably extend flight duration by limiting the use of on-board energy for propulsion. While soaring flight is relatively well understood for known wind, an autonomous soaring aircraft would have to generate paths based only on local observations of the wind made during the flight. This paper presents a method to simultaneously map and utilise a wind field using Gaussian process regression to generate a spatio-temporal map of the wind, and a path planning and dynamic target assignment algorithm to generate energy-gain paths from the current wind estimate. The planning architecture is tested in simulation for dynamic wind fields and shows consistent energy gain through exploration and exploitation of the wind environment.

This paper presents an automated approach for the classification of vegetation in natural environments based on high resolution aerial imagery acquired by a low flying Unmanned Aerial Vehicle (UAV). Standard colour and texture descriptors are extracted on a frame by frame basis to build a representation of appearance, which is probabilistically classified by a novel multi-class generalisation of the Gaussian Process (GP) developed for this work. A GP approach was selected for probabilistic outputs, and the ability to automatically determine the relevance of each input dimension to each of the C classes in the problem. When learning hyperparameters from N training examples, the new formulation scales at O(N ), rather than O(CN3) for the standard one-vs-all approach. The novel classification framework is trained and validated on a set of manual labels, and then queried to visualise a map of vegetation type under the UAV flight path. Mapping results are presented for a region of farmland in Northern Queensland, Australia that is infested with two invasive introduced tree species.

The ability to coordinate can offer significant performance advantages. This paper proposes an algorithm for decentralised control that is a function of both discrete and continuous variables. The algorithm is applied to a multi robot, multi-target mapping scenario, where decisions that couple the continuous motion controls and discrete robot-to target assignments are made. The algorithm out-performed benchmarks including implicit coordination, best-response, and the decoupling of continuous and discrete decision variables.

This paper presents and compares two different approaches to integrating sensor information from an Inertial Measuring Unit (IMU), Global Positioning System (GPS) receiver and monocular vision camera mounted to a low-flying Unmanned Aerial Vehicle (UAV) for building large-scale 3D terrain reconstructions. Both approaches utilise a statistically optimal bundle adjustment formulation that incorporates Vision, IMU and GPS observations into the map and pose optimisation process. Our first approach employs a novel pose-only formulation that optimises relative camera poses based on vision feature matches between frames, while incorporating IMU and GPS information. Our approach is related to, but differs from existing pose-graph techniques by formulating a set of 1D epipolar constraints for features matched between two camera frames, rather than minimising 4D feature re-projection errors, or marginalising feature states. We compare results of the method to a second approach which estimates both 3D features and poses together, using airborne vision, IMU and GPS data collected in an ecology mapping application. The results demonstrate a reduction in the computational complexity during optimisation for the pose-only approach, while producing equivalent accuracy in the reconstructed 3D terrain map.

In wildlife radio tagging, small radio transmitters attached to animals are located by human operators using directional antennas and analog receivers which provide audio output. The location of the transmitter is determined by listening to the signal and scanning the area while closing in. This procedure can be very tedious, especially in rough terrain. Searching radio tags with autonomous unmanned aerial vehicles (UAVs) offers a number of advantages, including better line-of-sight signal reception, terrain-independence and faster localization. In this paper we continue upon previous work by presenting a received-signal-strength (RSS) sensor implementation based on a modified commercial wildlife tracking receiver that is designed to operate on an autonomous fixed-wing UAV. Furthermore, an extension of the search and tracking framework for multiple targets that are undistinguishable from the sensors' point of view is proposed. After a brief system overview and a summary of the particle filter based approach, the signal processing theory and realization of the RSS sensor are outlined, including strategies for frequency tracking and receiver gain control. The paper also presents experimental results.

An efficient and analytical continuous-curvature path-smoothing algorithm, which fits an ordered sequence of waypoints generated by an obstacle-avoidance path planner, is proposed. The algorithm is based upon parametric cubic Bézier curves; thus, it is inherently closed-form in its expression, and the algorithm only requires the maximum curvature to be defined. The algorithm is, thus, computational efficient and easy to implement. Results show the effectiveness of the analytical algorithm in generating a continuous-curvature path, which satisfies an upper bound-curvature constraint, and that the path generated requires less control effort to track and minimizes control-input variability.

This paper presents a framework for integrating sensor information from an inertial measuring unit (IMU), Global Positioning System (GPS) receiver and monocular vision camera mounted to a low-flying unmanned aerial vehicle (UAV) for building large-scale terrain reconstructions. Our method seeks to integrate all of the sensor information using a statistically optimal non-linear least squares smoothing algorithm to estimate vehicle poses simultaneously to a dense point feature map of the terrain. A visualisation of the terrain structure is then created by building a textured mesh-surface from the estimated point features. The resulting terrain reconstruction can be used for a range of environmental monitoring missions such as invasive plant detection and biomass mapping.

This paper addresses the problem of communication range limitations for decentralized multi-vehicle mapping. We present a novel integrated communication and planning protocol that enables all vehicles to form a common global map. The fusion of mapping information is facilitated through the Information Filter and performed over a connected acyclic wireless communication network with limited communication range. The formation of the acyclic connected communication network is achieved by partitioning the landmark graph using graph theoretic tools during the planning phase. We provide results that illustrate the effectiveness of our approach over different distributions of landmarks.

Soaring is the process of gaining energy from the atmosphere in-flight using an aerodynamic free-flying platform. Dynamic soaring utilizes the energy available in vertical wind gradients and is commonly used by soaring birds. This research aims to develop a guidance and control strategy to utilize dynamic soaring for a fixed-wing gliding UAV. The basic strategies for dynamic soaring in vertical wind shear are explored and a simple piecewise trajectory based controller is developed to identify regions suitable for soaring and attempt traveling energy-neutral trajectories.

The use of accelerometer and gyro observations in a visual SLAM implementation is beneficial especially in high dynamic situations. The downside of using inertial is that traditionally high prediction rates are required as observations are provided at high sample rates. An accurate orientation and velocity estimate must also be maintained at all times in order to integrate the inertial observations and correct for the effect of gravity. This paper presents a way to pre-integrate the high rate inertial observations without the need for an initial orientation or velocity estimate. This allows for a slower filter prediction rate and use of inertial observations when the initial velocity and attitude of the platform are unknown. Additionally the initial velocity and roll and pitch of the platform become observable over time and an estimate of these values is provided by the filter. An estimate of the gravity vector is also provided. Results are presented using a delayed state information smoother implementation however due to the linearity of the equations this technique can be applied to extended Kalman filter (EKF) implementations just as easily.

This paper identifies various scale factor biases commonly introduced into monocular SLAM implementations as a result of the true scale factor of the map not being observable. A way to make the scale factor observable and remove any scale biases via the use of an inertial measurement unit (IMU) is presented and implemented. Results show that with an IMU the true scale of the map becomes observable over time and the use of a square root information filter allows the effect of initial scale biases to be removed completely from the solution resulting in an unbiased solution no matter what the initial scale assumptions are.

This paper presents a 3D path planing algorithm for an unmanned aerial vehicle (UAV) operating in cluttered natural environments. The algorithm satisfies the upper bounded curvature constraint and the continuous curvature requirement. In this work greater attention is placed on the computational complexity in comparison with other path-planning considerations. The rapidly-exploring random trees (RRTs) algorithm is used for the generation of collision free waypoints. The unnecessary waypoints are removed by a simple path pruning algorithm generating a piecewise linear path. Then a path smoothing algorithm utilizing cubic Bezier spiral curves to generate a continuous curvature path that satisfies the minimum radius of curvature constraint of UAV is implemented. The angle between two waypoints is the only information required for the generation of the continuous curvature path. The result shows that the suggested algorithm is simple and easy to implement compared with the Clothoids method.

This paper presents an algorithm which can effectively constrain inertial navigation drift using monocular camera data. It is capable of operating in unknown and large scale environments and assumes no prior knowledge of the size, appearance or location of potential environmental features. Low cost inertial navigation units are found on most autonomous vehicles and a large number of smaller robots. Depending on the grade of the sensor, when used alone, inertial data for control and navigation will only be reliable for a matter of seconds or minutes. An algorithm is presented that simultaneously estimates relative feature location in sensor space and inertial position, velocity and attitude in world coordinates. Feature locations are maintained in sensor space to ensure measurement linearity. Image depth is represented by an inverse function which permits un-delayed feature initialization and improves linearity and convergence. It is shown that the resulting navigation solution is able to be constrained, providing results comparable to inertial-GPS systems. Results are presented for an autonomous aircraft operating in a large semi-structured environment.

This paper presents the benefits of using a low cost inertial measurement unit to aid in an implementation of inverse depth initialized SLAM using a hand-held monocular camera. Results are presented with and without inertial observations for different assumed initial ranges to features on the same dataset. When using only the camera, the scale of the scene is not observable. As expected, the scale of the map depends on the prior used to initialize the depth of the features and may drift when exploring new terrain, precluding loop closure. The results show that the inertial observations help to improve the estimated trajectory of the camera leading to a better estimation of the map scale and a more accurate localization of features.

In this paper we present an observability analysis for a mobile robot performing SLAM with a single monocular camera. The aim is to get a better understanding of the well known intuitive behavior of these systems, such as the need for triangulation to features from different positions in order to get accurate relative pose estimates. The characterisation of the unobservable directions is made using the nullspace basis of the stripped observability matrix. This allows us to identify which vehicle motions are required to maximise the number of observable states in the system, which in turn affects accuracy in the estimation process. The analysis is performed by modelling the system in the continuous time domain as piecewise constant. Simulation results using an extended information filter are shown to verify the results of the observability analysis.

In many multi-objective missions there are situations when actions based on maximum information gain may not be the best given the overall mission objectives. In addition to properties such as entropy, information also has value which is situationally dependent. This paper presents a derivation of information value that considers both the context of information, via a fused world belief state, and a system mission. A simulated security operation in a structured environment is implemented, with a system of mobile heterogeneous sensors tasked with information gathering. Information value is then used to determine the expected reward for sensor observations of various points in area of operations. A brief comparison is then made to a selection of pure information gain schemes. From simulation results, it has been found that information value often captures elements of the mission and context not apparent using other information entropy metrics

This paper presents a method for accounting for uncertainty in auxiliary observation parameters (such as observer and sensor poses and sensor calibration) in mapping and tracking estimators. The method retains joint correlations between multiple features, resulting in improved relative accuracy in feature estimates. The method is demonstrated in simulation for a bearing-only feature tracking application. The key update step is described for Gaussian forms and in a general Bayesian probabilistic form. Simulations verify the consistency and show the benefit of cross-feature correlations

This paper presents a robust multi-loop airborne SLAM structure which also augments wind information. The air velocity observation from an air data system can be used to estimate the error of the on-board Inertial Navigation System (INS). However, due to a priori unknown wind velocity, it cannot directly be used for this purpose. This can be tackled by augmenting the unknown wind velocity into the state vector of SLAM, simultaneously estimating INS, map and wind. This paper proposes a multi-loop SLAM architecture, where the periodic velocity-level SLAM loop limits the INS errors of the velocity effectively, and the aperiodic position-level SLAM loop bounds the overall position error growth. This can significantly increase the consistency of airborne SLAM at the time of loop closure. Simulation results show that the unknown wind vector can be estimated consistently and the robustness of airborne SLAM improves significantly

We present a data fusion and decision making framework to perform prioritized searching for moving objects within an environment using ground and aerial sensors. A generalized Bayesian formulation is proposed to construct a joint probabilistic representation of the current situation is used as a basis in conjunction with predefined priority information in the decision making process. To cope with the computational intractability of a full probabilistic solution, two methods of approximation were studied. Instead of maintaining the full probabilistic representation of the environment, the first method utilizes a utility function created from the initial joint probability density. The utility function is then evolved according to sensor observations taken of the environment. The second method samples the initial density and track its evolution via a particle filter. It is shown that the second method out performs the first

This paper presents a decentralised particle filtering algorithm that enables multiple vehicles to jointly track 3D features under limited communication bandwidth. This algorithm, applied within a decentralised data fusion (DDF) framework, deals with correlated estimation errors due to common past information when fusing two discrete particle sets. Our solution is to transform the particles into Gaussian mixture models (GMMs) for communication and fusion. Not only can decentralised fusion be approximated by GMMs, but this representation also provides summaries of the particle set. Less bandwidth per communication step is required to communicate a GMM than the particle set itself hence conversion to GMMs for communication is an advantage. Real airborne data is used to demonstrate the accuracy of our decentralised particle filtering algorithm for airborne tracking and mapping

This paper presents an intelligent, on-line guidance scheme for maximising vehicle navigation and snap information for an uninhabited aerial vehicle while operating over unknown terrain. The tasks of localisation and mapping are performed concurrently using the well known extended Kalman filter implementation of the simultaneous localisation and mapping (SLAM) algorithm. A guidance scheme is formulated using vehicle actions in order to maximise the quality of the resulting vehicle pose estimates and environment map by maximising the entropic information in vehicle and map estimates. Results are presented using a high-fidelity six-degree of freedom simulation of an aerial vehicle using inertial navigation and a range/hearing sensor.

This paper addresses the computational complexity associated with 6-DoF inertial SLAM by recasting the filter into an indirect or complimentary structure. By doing so the standard non-linear inertial SLAM algorithm is piecewise linearised, providing two significant breakthroughs. Firstly, through the process of linearisation, the filter estimates the errors in the inertial states (errors in position, velocity and attitude) as opposed to the inertial states themselves, thus significantly reducing the amount of prediction updates required and hence the computational burden generally associated with inertial SLAM. Secondly, due to the piecewise linear nature of the underlying model, a new level of understanding of the observability of SLAM can be carried out through the use of the stripped observability matrix. This paper focuses on the 6-DoF implementation of SLAM in an airborne vehicle. Results illustrating that the new filter structure can efficiently estimate the errors and provide a navigation solution which is comparable to the standard implementation with significantly less computational cost is provided, along with theoretical results on the observability of 6-DoF SLAM.

This paper presents the system architecture of a real time multi-UAV simulator (RMUS). The simulator has been implemented as both a testing and validation mechanism for the real demonstration of multiple UAVs conducting both decentralised data fusion and control. These mechanisms include the off-line simulation of complex scenarios, hardware-in-the-loop tests, validation of real test results, and online mission control system demonstrations. The paper also present CommLibX, a novel communication framework for the system which allows simulation modules to communicate over single or multiple virtual channels. This unique communication system is then easily ported onto the real hardware allowing for maximum reuse of software and integrity.

This paper presents results of the application of simultaneous localisation and map building (SLAM) for an uninhabited aerial vehicle (UAV). Single vision camera and inertial measurement unit (IMU) are installed in a UAV platform. The data taken from a flight test is used to run the SLAM algorithm. Results show that both the map and the vehicle uncertainty are corrected even though the model of the system and observation are highly non-linear. The results, however, also indicate that further work of observability and the relationship between vehicle model drift and the number and the location of landmarks need to be further analysed given the highly dynamic nature of the system.

This paper presents a solution to the real-time control of cooperative unmanned air vehicles (UAVs) that engage multiple targets in a time-optimal manner. Techniques to dynamically allocate vehicles to targets and to find the time-optimal control actions of vehicles are proposed. The effectiveness of the time-optimal control technique is first demonstrated through numerical examples. The proposed strategy is then applied to a practical battlefield problem where ten vehicles are required to engage four targets, and numerical results show the efficiency of the proposed strategy.

This paper presents a new method for improving the accuracy of inertial measurement units (IMUs) mounted on land vehicles. The algorithm exploits nonholonomic constraints that govern the motion of a vehicle on a surface to obtain velocity observation measurements which aid in the estimation of the alignment of the IMU as well as the forward velocity of the vehicle. It is shown that this can be achieved without any external sensing provided that certain observability conditions are met. A theoretical analysis is provided together with a comparison of experimental results between a nonlinear implementation of the algorithm and an IMU/GPS navigation system. This comparison demonstrates the effectiveness of the algorithm. The real time implementation is also addressed through a multiple observation inertial aiding algorithm based on the information filter. The results show that the use of these constraints and vehicle speed guarantees the observability of the velocity and the attitude of the inertial unit, and hence bounds the errors associated with these states. The strategies proposed provides a tighter navigation loop which can sustain outages of GPS for a greater amount of time as compared to when the inertial unit is used with standard integration algorithms.

Describes a real time, on-the-fly, roll and pitch alignment algorithm for inertial measurement units (IMUs) mounted on land vehicles. Unlike conventional strategies, the alignment is achieved without external observations. This is achieved by exploiting the nonholonomic constraints that govern the motion of a vehicle on a surface to obtain the roll and pitch of the IMU. The position of the vehicle along its path is still unobservable and the algorithm is effective only when there is sufficient excitation in all degrees of freedom. However, with the proposed alignment algorithm, the IMU is able to provide sufficiently accurate position information for substantially longer periods of time compared with conventional methods. Experimental results are provided along with a comparison of an IMU/GPS navigation loop showing the effectiveness of the algorithm.

This paper describes the development and implementation of a high integrity navigation system, based on the combined use of the Global Positioning System (GPS) and an inertial measurement unit (IMU), for autonomous land vehicle applications. The paper focuses on the issue of achieving the integrity required of the navigation loop for use in autonomous systems. The paper highlights the detection of possible faults both before and during the fusion process in order to enhance the integrity of the navigation loop. The implementation of this fault detection methodology considers both low frequency faults in the IMU caused by bias in the sensor readings and the misalignment of the unit, and high frequency faults from the GPS receiver caused by multipath errors. The implementation, based on a low-cost, strapdown IMU, aided by either standard or carrier phase GPS technologies, is described. Results of the fusion process are presented.

The objective of this paper is to introduce and investigate the issue of integrity in an INS/GPS navigation loop for autonomous land vehicle applications. The paper briefly outlines the standard fusion algorithm for the INS/GPS loop, while the focus is on the detection of possible faults both before and during the fusion process. The concept of fault detection focuses on the low frequency faults of the INS, caused by bias and drift, and the high frequency faults of the GPS unit caused by multipath errors and changes in satellite geometry.