Reinforcement Learning has revolutionized decision-making processes in dynamic environments, yet it often struggles with autonomously detecting and achieving goals without clear feedback signals. For example, in a Source Term Estimation problem, the lack of precise environmental information makes it challenging to provide clear feedback signals and to define and evaluate how the source's location is determined. To address this challenge, the Autonomous Goal Detection and Cessation (AGDC) module was developed, enhancing various RL algorithms by incorporating a self-feedback mechanism for autonomous goal detection and cessation upon task completion. Our method effectively identifies and ceases undefined goals by approximating the agent's belief, significantly enhancing the capabilities of RL algorithms in environments with limited feedback. To validate effectiveness of our approach, we integrated AGDC with deep Q-Network, proximal policy optimization, and deep deterministic policy gradient algorithms, and evaluated its performance on the Source Term Estimation problem. The experimental results showed that AGDC-enhanced RL algorithms significantly outperformed traditional statistical methods such as infotaxis, entrotaxis, and dual control for exploitation and exploration, as well as a non-statistical random action selection method. These improvements were evident in terms of success rate, mean traveled distance, and search time, highlighting AGDC's effectiveness and efficiency in complex, real-world scenarios.

Unmanned Aerial Vehicles (UAVs) are gaining attention for inspections due to their improved safety, efficiency, and accuracy, alongside reduced costs and environmental risks. Visual servoing is crucial for autonomous UAV flight in GPS-degraded environments, guiding the UAV by minimizing errors between observed and desired visual features. This study focuses on Image-Based Visual Servoing (IBVS) control for quadrotor UAVs under complex dynamics and environmental disturbances. A nonlinear model predictive control (MPC) framework is first integrated with visual servoing to handle dynamics nonlinearity, control optimality, and constraints. To address uncertainties and disturbances, a Generalized Extended State Observer (GESO) is incorporated into the MPC, forming the Disturbance-Resilient (DR-) MPC. The GESO estimates the lumped disturbance to improve model predictions within the MPC horizon. The proposed algorithm is validated in a realistic Gazebo environment for UAV pipeline inspection in 3D scenarios, showing better control accuracy and reduced inspection time compared to three baseline methods: IBVS, IBVS-MPC(K) with kinematics, and IBVS-MPC(D) with dynamics.1

This paper studies the motion planning problem of the pick-and-place of an aerial manipulator that consists of a quadcopter flying base and a Delta arm. We propose a novel partially decoupled motion planning framework to solve this problem. Compared to the state-of-the-art approaches, the proposed one has two novel features. First, it does not suffer from increased computation in high-dimensional configuration spaces. That is because it calculates the trajectories of the quadcopter base and the end-effector separately in Cartesian space based on proposed geometric feasibility constraints. The geometric feasibility constraints can ensure the resulting trajectories satisfy the aerial manipulator’s geometry. Second, collision avoidance for the Delta arm is achieved through an iterative approach based on a pinhole mapping method, so that the feasible trajectory can be found in an efficient manner. The proposed approach is verified by five experiments on a real aerial manipulation platform. The experimental results show the effectiveness of the proposed method for the aerial pick-and-place task.Note to Practitioners—Aerial manipulators have attracted increasing research interest in recent years due to their potential applications in various domains. In this paper, we particularly focus on the motion planning problem of the pick-and-place of aerial manipulators. We propose a novel partially decoupled motion planning framework, which calculates the trajectories of the quadcopter base and the end-effector in Cartesian space, respectively. Geometric feasibility constraints are proposed to coordinate the trajectories to ensure successful execution. Five experiments on a real aerial manipulator platform demonstrate the effectiveness of the approach. In future research, we will address the motion planning problem of aerial manipulators in complex environments.

In this article, a multi-estimator based computationally efficient algorithm is developed for autonomous search in an unknown environment with an unknown source. Different from the existing approaches that require massive computational power to support nonlinear Bayesian estimation and complex decision-making process, an efficient cooperative active-learning-based dual control for exploration and exploitation (COAL-DCEE) is developed for source estimation and path planning. Multiple cooperative estimators are deployed for environment learning process, which is helpful to improving the search performance and robustness against noisy measurements. The number of estimators used in COAL-DCEE is much smaller than that of the particles required for Bayesian estimation in information-theoretic approaches. Consequently, the computational load is significantly reduced. As an important feature of this study, the convergence and performance of COAL-DCEE are established in relation to the characteristics of sensor noises and turbulence disturbances. Numerical and experimental studies have been carried out to verify the effectiveness of the proposed framework. Compared with the existing approaches, COAL-DCEE not only provides convergence guarantee but also yields comparable search performance using much less computational power.

Source term estimation of a hazardous release remains a topic of significant interest in the robotics and state estimation communities, with application to many safety critical scenarios including gas or nuclear release, locating suspicious smells or response to emergency incidents. Limited sensing resources and time constraints mean that deciding on how to act in order to improve efficiency of estimation is also of significant interest. This paper has two main focuses: a sequential Monte Carlo technique for performing source term estimation from gas concentration measurements taken on a mobile sensor platform and a Monte Carlo tree search (MCTS) framework to perform sensor motion planning to maximise Kullback-Leibler divergence (KLD). Both algorithms are implemented in the open source tracking and estimation framework: Stone Soup, creating several key contributions to this Python based toolkit. The presented algorithm demonstrates superior performance when compared to a greedy myopic alternative when considering source position estimation error, release rate error and successful rate performance measures.

Robotic gas distribution mapping improves the understanding of a hazardous gas dispersion while putting the human operator out of danger. Generating an accurate gas distribution map quickly is of utmost importance in situations such as gas leaks and industrial incidents, so that the efficient use of resources in response to incidents can be facilitated. In this paper, to incorporate the operational requirement on map granularity, we propose a low-to-high resolution path planner that first guides a single robots to quickly and sparsely sample the region of interest to generate a low resolution gas distribution map, followed by high resolution sampling informed by the low resolution map as a prior. The low resolution prior acts as a coverage survey allowing the algorithm to perform a relatively exploitative search of high concentration regions, resulting in overall shorter mission times. The proposed framework is designed to iteratively identify the next best T locations to sample, which prioritises the potentially high reward locations, while ensuring that the robot can travel to and sample the chosen locations within a user specified map update cycle. We present a simulation study to demonstrate the alternating exploration-exploitation like behaviour along with bench-marking its performance in contrast to the traditional sampling path planners and various reward functions.

This paper studies the tracking control problem of an aerial manipulator that consists of a quadcopter flying base and a Delta robotic arm. We propose a novel control approach that consists of extended state observers (ESOs) for dynamic coupling estimation, ESO-based flight controllers, and a cooperative trajectory planner. Compared to the state-of-the-art approaches, the proposed one has some attractive features. First, it requires much less measurement information as opposed to the full-body control approaches and hence can be implemented conveniently and efficiently in practice. Second, while the existing approaches estimate the coupling effect based on precise models, the proposed ESOs can do that based on much less information about the system model. The proposed approach is verified by four experiments on a real aerial manipulation platform. The experimental results show that the average tracking error can reach 1 cm by the proposed approach as opposed to 10 cm by the PX4 baseline controller. Although force control is not considered specifically in the approach, the system can complete aerial weaving tasks thanks to the ESOs in the presence of drag forces applied to the end-effector during manipulation.Note to Practitioners—Aerial manipulators have received increasing research attention in recent years due to their wide range of applications. In this paper, we particularly focus on the high-precision and robust control of aerial manipulators. We propose a novel control approach that consists of extended state observers (ESOs) for dynamic coupling estimation, ESO-based flight controllers, and a cooperative trajectory planner. Four experiments on a real aerial manipulation platform demonstrate the effectiveness of the approach. In future research, we will address the control problem when the aerial manipulator contacts the environment.

This paper proposes a new 3D gas distribution mapping technique based on Gaussian belief propagation, which is capable of resolving in real time, the concentration estimates in 3D space whilst accounting for the obstacle information within the scenario, the first of its kind in the literature. The gas mapping problem is formulated as a 3D factor graph of Gaussian potentials, the connections of which are conditioned on local occupancy values. The Gaussian belief propagation framework is introduced as the solver and a new hybrid message scheduler is introduced to increase the rate of convergence. The factor graph problem is then redesigned as a dynamically expanding inference task, coupling the information of consecutive gas measurements with local spatial structure obtained by the robot. The proposed algorithm is compared to the state of the art methods in 2D and 3D simulations and is found to resolve distribution maps orders of magnitude quicker than typical direct solvers. The proposed framework is then deployed onboard a ground robot in a 3D mapping and exploration task. The system is shown to be able to resolve multiple sensor inputs and output high resolution 3D gas distribution maps in a GPS denied cluttered scenario in real time. This online inference of complicated gas dispersion provides a new layer of contextual information over its 2D counterparts and enables autonomous systems to take advantage of real time estimates to inform potential next best sampling locations.Note to Practitioners—The motivation of this work arises from the need to develop the robotic gas distribution mapping capability that can provide real-time situational awareness of the scenarios. The output distribution maps can be used to inform human first responders as to what areas of the environment contain a hazard, but looking towards autonomous robots, they can also be used by the robot itself to inform where should be measured next to gather more information about the environment. When performing these mapping tasks in unknown indoor environments, it is very important that the sensing robot can build up the knowledge of its physical surroundings together with how the obstacles in the environment affect the 3D gas distribution. The Gaussian belief propagation algorithm allows us to achieve all of this in real-time onboard the sensing robot, something that is yet to be achieved in the literature.

Automating sensing processes is of high interest to both the target tracking and the control community. Active sensing is focused on solving this task, usually with information based or task driven selection of optimal sensing actions. This paper presents an active sensing formulation that combines task based, in the form of standoff tracking, and information based active sensing by implementing the dual control for exploitation and exploration (DCEE) concept to control a mobile sensor platform with a limited field-of-view. The DCEE based cost function is integrated into the Monte Carlo tree search (MCTS) framework for non-myopic decision making. Using the Bernoulli particle filter for single target tracking with bearing-only measurements, the DCEE observer control method is benchmarked against the popular Rényi divergence information metric with two different parameterisations. Whilst the Rényi divergence performs marginally better when considering existence estimation, spatial results clearly demonstrate that our formulation is able to outperform the benchmark algorithm with improved target localisation performance resulting from outmanoeuvring of the target.

This article addresses the particle filtering problem for a class of nonlinear/non-Gaussian systems with quantized measurements and multiple degrading sensors. A degradation variable described by the Wiener process is proposed to describe the phenomenon of sensor degradation that is often encountered in engineering practice. The measurement output of each sensor is quantized by a uniform quantizer before being sent to the remote filter. An augmented system is constructed, which aggregates the original system state and the degradation variables. In the presence of the sensor degradation and the quantization errors, a new likelihood function at the remote filter is calculated by resorting to all the transmitted measurements. According to the mathematical characterization of the likelihood function, a novel particle filtering algorithm is developed, where the parameters of both the degradation processes and the quantization functions are exploited to obtain the modified importance weights. Finally, the effectiveness of the proposed method is shown via a target tracking example with bearing measurements.

This paper advocates the Gaussian belief propagation solver for factor graphs in the case of gas distribution mapping to support an olfactory sensing robot. The local message passing of belief propagation moves away from the standard Cholesky decomposition technique, which avoids solving the entire factor graph at once and allows for only areas of interest to be updated more effectively. Implementing a local solver means that iterative updates to the distribution map can be achieved orders of magnitude quicker than conventional direct solvers which scale computationally to the size of the map. After defining the belief propagation algorithm for gas mapping, several state of the art message scheduling algorithms are tested in simulation against the standard Cholesky solver for their ability to converge to the exact solution. Testing shows that under the wildfire scheduling method for a large urban scenario, that distribution maps can be iterated at least 10 times faster whilst still maintaining exact solutions. This move to an efficient local framework allows future works to consider 3D mapping, predictive utility and multi-robot distributed mapping.

Agriculture is facing severe challenges from crop stresses, threatening its sustainable development and food security. This article exploits aerial visual perception for yellow rust disease monitoring, which seamlessly integrates state-of-the-art techniques and algorithms, including unmanned aerial vehicle sensing, multispectral imaging, vegetation segmentation, and deep learning U-Net. A field experiment is designed by infecting winter wheat with yellow rust inoculum, on top of which multispectral aerial images are captured by DJI Matrice 100 equipped with RedEdge camera. After image calibration and stitching, multispectral orthomosaic is labeled for system evaluation by inspecting high-resolution RGB images taken by Parrot Anafi Drone. The merits of the developed framework drawing spectral-spatial information concurrently are demonstrated by showing improved performance over purely spectral-based classifier by the classical random forest algorithm. Moreover, various network input band combinations are tested, including three RGB bands and five selected spectral vegetation indices, by sequential forward selection strategy of wrapper algorithm.

Mobile robotic gas distribution mapping (GDM) is a useful tool for hazardous scene assessment where a quick and accurate representation of gas concentration levels is required throughout a staging area. However, research in robotic path planning for GDM has primarily focused on mapping in open spaces or estimating the source term in dispersion models. Whilst this may be appropriate for environment monitoring in general, the vast majority of GDM applications involve obstacles, and path planning for autonomous robots must account for this. This paper aims to tackle this challenge by integrating a GDM function with an informative path planning framework. Several GDM methods are explored for their suitability in cluttered environments and the GMRF method is chosen due to its ability to account for obstacle interactions within the plume. Based on the outputs of the GMRF, several reward functions are proposed for the informative path planner. These functions are compared to a lawnmower sweep in a high fidelity simulation, where the RMSE of the modelled gas distribution is recorded over time. It is found that informing the robot with uncertainty, normalised concentration and time cost, significantly reduces the time required for a single robot to achieve an accurate map in a large-scale, urban environment. In the context of a hazardous gas release scenario, this time reduction could save lives as well as further gas ingress.

As a virtual sensor, disturbance observer provides an alternative approach to reconstruct lumped disturbances (including external disturbances and system uncertainties) based upon system states/outputs measured by physical sensors. Not surprisingly, measurement errors bring adverse effects on the control performance and even the stability of the closed-loop system. Toward this end, this paper investigates the problem of disturbance observer-based control for a class of disturbed uncertain nonlinear systems in the presence of unknown output measurement errors. Instead of inheriting from the estimation-error-driven structure of Luenberger-type observer, the proposed disturbance observer only explicitly uses the control input. It has been proved that the proposed method endows the closed-loop system with strong robustness against output measurement errors and system uncertainties. With rigorous analysis under the semiglobal stability criterion, the guideline of gain choice based upon the proposed structure is provided. To better demonstrate feature and validity of the proposed method, numerical simulation and comparative experiments of a helicopter model are implemented.

A method was previously developed by this author to optimise the flight path of a fixed wing UAV performing aerial surveys of complex concave agricultural fields. This relies heavily on a flight time in wind prediction model as its cost function. This paper aims to validate this model by comparing flight test results with the model prediction. There are a number of assumptions that this model relies on. The major assumption is that wind is steady and uniform over the small area and time scales involved in a survey. To show that this is reasonable, wind fields measurements will be taken from a multi rotor UAV with an ultrasonic windspeed sensor.

This paper presents experiments to assess the plume mapping performance of autonomous robots. The paper compares several mapping algorithms including Gaussian Process regression, Neural networks and polynomial and piecewise linear interpolation. The methods are compared in Monte Carlo simulations using a well known plume model and in indoor experiments using a ground robot. Unlike previous work on mapping using unmanned vehicles, the indoor experiments were performed in a controlled and repeatable manner where a steady state ground truth could be obtained in order to properly assess the various regression methods using data from a real dispersive source and sensor. The effect of sampling time during data collection was assessed with regards to the mapping accuracy, and the data collected during the experiments have been made available. Overall, the Gaussian Process method was found to perform the best among the regression algorithms, showing more robustness to the noisy measurements obtained from short sampling periods, enabling an accurate map to be produced in significantly less time. Finally, plume mapping results are presented in uncontrolled outdoor conditions, using an unmanned aerial vehicle, to demonstrate the system in a realistic uncontrolled environment.

Early driver intention prediction plays a significant role in intelligent vehicles. Drivers exhibit various driving characteristics impairing the performance of conventional algorithms using all drivers' data indiscriminatingly. This paper develops a personalized driver intention prediction system at unsignalized T intersections by seamlessly integrating clustering and classification. Polynomial regression mixture (PRM) clustering and Akaike's information criterion are applied to individual drivers trajectories for learning in-depth driving behaviors. Then, various classifiers are evaluated to link low-level vehicle states to high-level driving behaviors. CART classifier with Bayesian optimization excels others in accuracy and computation. The proposed system is validated by a real-world driving dataset. Comparative experimental results indicate that PRM clustering can discover more in-depth driving behaviors than manually defined maneuver due to its fine ability in accounting for both spatial and temporal information; the proposed framework integrating PRM clustering and CART classification provides promising intention prediction performance and is adaptive to different drivers.

Awareness of driver workload (DW) plays a paramount role in enhancing driving safety and convenience for intelligent vehicles. The DW prediction systems proposed so far learn either from individual driver's data (termed personalized system) or existing drivers' data indiscriminately (termed average system). As a result, they either do not work or lead to a limited performance for new drivers without labeled data. To this end, we develop clustering-aided approaches exploiting group characteristics of the existing drivers' data. Two clustering aided predictors are proposed. The first is clustering-aided regression (CAR) model, where the regression model for the cluster with the highest likelihood is adopted. The second is clustering-aided multiple model regression model, where the concept of multiple models is further augmented to CAR. A recent dataset from real-world driving experiments is adopted to validate the algorithms. Comparative results against the conventional average system demonstrate that by incorporating clustering information, both the proposed approaches significantly improve workload prediction performance.

Adapting in-vehicle systems (e.g., advanced driver assistance systems and in-vehicle information systems) to individual drivers' workload can enhance both safety and convenience. To make this possible, it is a prerequisite to infer driver workload so that adaptive aiding can be provided to the driver at the right time and in an appropriate manner. Rather than developing an average model for all drivers, a personalized driver workload inference (PDWI) system considering individual drivers driving characteristics is developed using machine learning techniques via easily accessed vehicle related measurements (VRMs). The proposed PDWI system comprises two stages. In offline training, individual drivers workload is first automatically splitted into different categories according to its inherent data characteristics using fuzzy C-means (FCM) clustering. Then an implicit mapping between VRMs and different levels of workload is constructed via classification algorithms. In online implementation, VRMs samples are classified into different clusters, consequently driver workload type can be successfully inferred. A recently collected dataset from real-world naturalistic driving experiments is drawn to validate the proposed PDWI system. Comparative experimental results indicate that the proposed framework integrating FCM clustering and support vector machine classifier provides a promising workload recognition performance in terms of accuracy, precision, recall, F1-score, and prediction time. The interindividual differences in term of workload are also identified and can be accommodated by the proposed framework due to its adaptiveness.

Ahstract- This paper presents a method to estimate the original location and the mass of an instantaneous release of hazardous material into the atmosphere. It is formulated as an inverse problem, where concentration observations from a mobile sensor are fused with meteorological information and a Gaussian puff dispersion model to characterise the source. Bayes' theorem is used to estimate the parameters of the release taking into account the uncertainty that exists in the dispersion parameters and meteorological variables. An information based reward is used to guide an unmanned aerial vehicle equipped with a chemical sensor to the expected most informative measurement locations. Simulation results compare the performance between a single mobile sensor with various amounts of static sensors.

This paper presents a new standoff tracking framework of a moving ground target using UAVs with a limited sensing capability such as sensor field-of-view and motion constraints. To maintain persistent track of the target even in case of target loss (out of surveillance) for a certain period, this study predicts the target existence area using the particle filter, and produces control commands to ensure that all predicted particles can be covered by the field-of-view of the UAV sensor at all times. To improve target prediction/estimation accuracy, the road information is incorporated into the constrained particle filter where the road boundaries are modelled as nonlinear inequality constraints. Both Lyapunov vector field guidance and nonlinear model predictive control methods are applied for the standoff tracking and phase angle control, and the advantages and disadvantages of them are compared using numerical simulation results.

Autonomous control of unmanned helicopters has drawn a great deal of interests in recent years. This paper investigates two aspects on this topic, namely the flight control and local path planning. The former is designed only using the low-level helicopter dynamics, but the latter take into account the high-level kinematics in an optimisation based framework. A guidance compensator is therefore developed to integrate them in an interacting way. The overall control principle is demonstrated in an indoor flight environment with promising results.