This paper introduces EDeformNet, a novel method for real-time 3D reconstruction of fishing nets using sparse positional measurements. Currently, net deployment during large-scale fishing operations is challenging as the submerged lattice deformations that occur in response to the various environmental factors are not visible to the vessel operator. EDeformNet extends Embedded Deformation Graphs (EDGs), a commonly used technique in template-based nonrigid 3D reconstruction that allows control of embedded spaces through sparse control point correspondences. These can be suitably derived from acoustic tracking beacons attached to the net. EDeformNet enhances the standard EDG optimization scheme by including constraints that preserve surface normals at control points and guard distances between vertices in the template mesh. These improvements are proven to enable an accurate representation of the complex deformations and movements typical in purse seine nets, the fishing technique where the algorithm has been tested, which standard EDG is unable to attain. Moreover, EDeformNet also proposes a tailored strategy that dynamically adjusts the net template according to the known length of the deployed portion of the fishing net. This approach reconstructs exclusively the submerged portion of the fishing net, avoiding extraneous data from above-water sections and enhancing accuracy under realistic fishing conditions. The proposed method is validated using realistic 3D physics simulations in Blender, where quantifiable comparisons demonstrate that EDeformNet effectively captures the spatial dynamics of purse-seining. Compared to standard EDG, EDeformNet achieves superior performance, resulting in at least a 25% improvement across the array of challenging temporal scenarios studied.

This paper proposes a novel hierarchical methodology to planning safe UAV trajectories in complex environments. We start by improving a canonical hybrid A* in relation to high memory requirements, performance degradation, and the low efficiency customarily observed in the initial global trajectory suggested by the planner. Then, the Marden theorem is applied - for the first time in local path planning - to generate continuous, non-intersecting, enclosed, and safe flight corridors, termed homotopic enclosed safe motion corridors (HESMCs) hereafter. This is efficiently realized through a series of unique ellipsoids along the initial route. Meanwhile, the optimized motion trajectory along the corridors is built by considering two waypoints and prescribed performance functions. The resolved path is safe and complete, with a comprehensive Lyapunov stability analysis included to ensure accurate and efficient trajectory tracking. The simulation and physical tests demonstrate the superiority of our proposed planner over existing state-of-the-art methods, with consistent and significant improvements in processing time and guaranteed completeness. Note to Practitioners—The authors perceived the contribution of the manuscript of particular relevance to users of UAVs seeking advanced safety in their guidance and navigational solutions, offering a blend of theoretical innovation and practical applicability. The work introduces a distinct hierarchical motion planner specifically designed to enhance safety and reliability in UAV navigation. Key to this is the development of an improved hybrid A* algorithm for global planning, which effectively tackles practical issues such as high memory consumption and performance degradation. A significant theoretical contribution is the application of the Marden theorem in local optimization. This facilitates the generation of homotopic enclosed motion corridors using unique safe boundary ellipsoids, thus reducing navigation complexity and the risk of failure during task execution. Additionally, the proposed scheme emphasizes the generation of motion trajectories considering position errors and prescribed performance functions, supplemented by a thorough Lyapunov stability analysis. Looking ahead, we aim to extend the proposed scheme in the context of UAV swarms for more efficient navigation in complex environments.

This paper proposes an improved solver for the maximal continuity graph painting problem. The problem is motivated by the real-world surface non-revisiting coverage path planning (NCPP) task carried out by manipulators, where the physical meaning of maximising the colouring continuity in the graph translates to minimising the number of undesirable transitions between end-effector force/torque control discontinuities. Early works have formulated the graph-based representation of the task and finitely solved the graph. However, the exponential growth of its algorithmic complexity makes the problem intractable for even relatively simple graphs. The improved solver proposed in this paper demonstrates exponential improvement compared to the state-of-the-art algorithm, setting guaranteed bounds on performance, whereby the algorithmic complexity is proven reduced by a factor of$2^N$,$N$being the number of internal edges in the graph. Challenging simulated experiments are presented to validate the computational advantage, and an open source implementation is also provided for the benefit of the community.Note to Practitioners—To solve a non-redundant manipulator NCPP task, the first step is collecting all valid inverse kinematics configurations that lead to coverage on the surface. Continuous configurations can then be grouped and assigned the same colour. This process creates a spatial distribution of colours on the target surface, forming a topological graph as detailed in this paper. Using the proposed algorithm, each coverable point on the surface is assigned a colour, which specifies the unique inverse kinematic configuration that the manipulator should adopt to cover such a point. A suitable geometric coverage planner can then be employed to generate the path that a manipulator end-effector must follow on the surface that is guaranteed to have the minimum number of end-effector discontinuities. In this paper, an (open-sourced) solver is proposed to solve the graph optimally from a computational point of view, most notably increasing productivity from a manufacturing/automated perspective.

A predictive model for mobility systems capable of understanding the trajectory a user intends to follow in the environment is proposed. Understanding user intention is paramount for any shared-control navigation strategy between a user and an active robotic agent. Equally important however is being able to go beyond simple sample generation to assign probabilistic meaning to the set of possible future trajectories, so most likely scenarios can be assumed. The framework estimates a distribution over possible intentions, proposing a novel generative model predicated on Normalizing Flows which accounts for past behaviours, as traditionally reported in the literature, but also incorporates visual scene information. As the model permits trajectories to be assigned exact likelihoods, tractable density estimates can be readily exploited to finalize an executable intention. Baseline comparisons with the publicly available and widely used KITTI navigational dataset show significant improvements (up to 11.08%) with respect to traditional metrics such as Average and Final Displacement Errors. A novel metric that stands independent of the number of samples is also proposed as a more fitting comparison for future works.

This work considers the optimal non-revisiting coverage tasks with a single non-redundant manipulator for the case when the object can be positioned at a predefined set of locations within the workcell. The scenario is often encountered in typical industrial settings, for instance when the object presents itself along a conveyor belt and its surface can not be serviced at a single location - the object being large or complex for that endeavour. Given the non-bijective nature of manipulator kinematics between task and joint space, without explicit consideration of joint-space continuity during its construction, a continuous coverage path designed in task-space may easily be truncated into intermittent segments where the manipulator needs to adopt a different configuration to continue the task, resulting in manipulator motions where the end-effector will need to lift off the surface, an altogether undesirable characteristic affecting the quality of the final product for smooth operations on objects such as polishing, painting or deburring. In this work, a novel algorithm to optimally partition the task-space whilst considering the various finite locations where the object may be stationed is proposed that ensures joint-space coverage continuity with minimal lift-offs. Results from the algorithm being challenged to achieve coverage of a number of objects, both in simulation and in real tests with an industrial manipulator, prove the effectiveness of the proposed planner when compared with classical coverage strategies faced with the same problem.

A multi-modal framework to generate user intention distributions when operating a mobile vehicle is proposed in this work. The model learns from past observed trajectories and leverages traversability information derived from the visual surroundings to produce a set of future trajectories, suitable to be directly embedded into a perception-action shared control strategy on a mobile agent, or as a safety layer to supervise the prudent operation of the vehicle. We base our solution on a conditional Generative Adversarial Network with Long-Short Term Memory cells to capture trajectory distributions conditioned on past trajectories, further fused with traversability probabilities derived from visual segmentation with a Convolutional Neural Network. The proposed data-driven framework results in a significant reduction in error of the predicted trajectories (versus the ground truth) from comparable strategies in the literature (e.g. Social-GAN) that fail to account for information other than the agent’s past history. Experiments were conducted on a dataset collected with a custom wheelchair model built onto the open-source urban driving simulator CARLA, proving also that the proposed framework can be used with a small, unannotated dataset.

Advancements in technology propose a future where systems work collaboratively sharing the same workspace as humans. Navigation is one such crucial aspect of daily life where collaborative technologies can offer major assistance. Ageing population dictates a likely increase in personal mobility devices (PMDs), whilst autonomous cars are bringing intelligent vehicles to the road today. However, in such scenarios the expected assistance can only be given if the device is aware of its user's intention, so that controls can be applied in a tightly collaborative manner. Moreover, they should be robust to different environments, users and mobile platforms. A user driven navigation framework is proposed in this work to complement end-to-end sensing-only solutions to estimate controls as joint intention from vehicle states and user inputs. The solution is proven to be an improvement over similar strategies that rely on exteroceptive data and omit inputs from the driving agent. Furthermore, the developed framework is proven capable of transferring the learning into different environments and mobility platforms using a small amount of training data. Data from the autonomous driving community (Udacity dataset) and other obtained in-house with an instrumented power wheelchair are given to demonstrate the validity of the proposed approach.

Knowing the geometry of a space is desirable for many applications, e.g. sound source localization, sound field reproduction or auralization. In circumstances where only acoustic signals can be obtained, estimating the geometry of a room is a challenging proposition. Existing methods have been proposed to reconstruct a room from the room impulse responses (RIRs). However, the sound source and microphones must be deployed in a feasible region of the room for it to work, which is impractical when the room is unknown. This work propose to employ a robot equipped with a sound source and four acoustic sensors, to follow a proposed path planning strategy to moves around the room to collect first image sources for room geometry estimation. The strategy can effectively drives the robot from a random initial location through the room so that the room geometry is guaranteed to be revealed. Effectiveness of the proposed approach is extensively validated in a synthetic environment, where the results obtained are highly promising.

This paper presents an active assistance framework for mobility systems, such as Power Mobility Devices (PMD), with the distinctive goal of being able to operate within a local moving window, as opposed to the common reliance upon persistent global environments and objectives. Demonstration data from able experts driving a simulated mobility aid in a representative indoor setting is used off-line to build behavioral models of navigation postulated separately upon user joystick inputs and on-board sensor data. These models are built respectively via Gaussian Processes for the joystick signals, and a Deep Convolutional Neural Network for the sensor data; in this case a planar LIDAR. Their combined outputs form a continuous distribution of estimated traversal likelihood within the user's immediate space, allowing for real-time stochastic optimal path planning to guide a user to its intended local destination. Moreover, the computational efficiency of the decoupled models permits rapid replanning on-the-fly for a smooth assistive action. On-line and off-line evaluations substantiate the advantages of the framework in generalising intelligent navigational assistance, of particular relevance for users who experience difficulty in safe mobility.

In this paper, we develop a system for the low-cost indoor localization and tracking problem using radio signal strength indicator, Inertial Measurement Unit (IMU), and magnetometer sensors. We develop a novel and simplified probabilistic IMU motion model as the proposal distribution of the sequential Monte-Carlo technique to track the robot trajectory. Our algorithm can globally localize and track a robot with a priori unknown location, given an informative prior map of the Bluetooth Low Energy (BLE) beacons. Also, we formulate the problem as an optimization problem that serves as the Backend of the algorithm mentioned above (Front-end). Thus, by simultaneously solving for the robot trajectory and the map of BLE beacons, we recover a continuous and smooth trajectory of the robot, corrected locations of the BLE beacons, and the time-varying IMU bias. The evaluations achieved using hardware show that through the proposed closed-loop system the localization performance can be improved; furthermore, the system becomes robust to the error in the map of beacons by feeding back the optimized map to the Front-end.

This paper presents a control architecture readily suitable for surface treatment tasks such as polishing, grinding, finishing or deburring as carried out by a human operator, with the added benefit of accuracy, recurrence and physical strength as administered by a robotic manipulator partner. The shared strategy effectively couples the human operator propioceptive abilities and fine skills through his interactions with the autonomous physical agent. The novel proposed control scheme is based on task prioritization and a non-conventional sliding mode control, which is considered to benefit from its inherent robustness and low computational cost. The system relies on two force sensors, one located between the last link of the robot and the surface treatment tool, and the other located in some place of the robot end-effector: the former is used to suitably accomplish the conditioning task, while the latter is used by the operator to manually guide the robotic tool. When the operator chooses to cease guiding the tool, the robot motion safely switches back to an automatic reference tracking. The paper presents the theories for the novel collaborative controller, whilst its effectiveness for robotic surface treatment is substantiated by experimental results using a redundant 7R manipulator and a mock-up conditioning tool.

In this paper, we propose a real-time classification scheme to cope with noisy Radio Signal Strength Indicator (RSSI) measurements utilized in indoor positioning systems. RSSI values are often converted to distances for position estimation. However due to multipathing and shadowing effects, finding a unique sensor model using both parametric and non-parametric methods is highly challenging. We learn decision regions using the Gaussian Processes classification to accept measurements that are consistent with the operating sensor model. The proposed approach can perform online, does not rely on a particular sensor model or parameters, and is robust to sensor failures. The experimental results achieved using hardware show that available positioning algorithms can benefit from incorporating the classifier into their measurement model as a meta-sensor modeling technique.

Active assistive mobility systems are largely limited to a-priori mapped environments, whereas their reactive assistive counterparts are in general location independent and focus on the provision of collision avoidance in the immediate space surrounding the platform. This paper presents a framework capable of providing active short-term navigation, combining the intelligence of active assistance with the freedom of location independence. Demonstration data from an able expert while driving the mobility aid in a standard indoor setting is used off-line to learn reference behavioral models of navigation given perceptual information from the platform surroundings and the input controls exerted by the user while navigating. These serve as the foundation for on-line probabilistic short-term destination inference using the instantaneously available data from the user and on-board sensors. This is coupled with a real-time stochastic optimal path generation able to exploit the same short term demonstration paths from the expert with the belief they capture both the driver's awareness of the platform's physical geometry and appropriate behaviors for their surroundings. Experimental results with users of varying proficiency in a setting unvisited in training data show promise in using the framework in assisting users experiencing difficulty in safe power mobility aid use.

In this paper, we present a method for real-time 3D sound sources mapping using an off-the-shelf robotic perception sensor equipped with a linear microphone array. Conventional approaches to map sound sources in 3D scenarios use dedicated 3D microphone arrays, as this type of arrays provide two degrees of freedom (DOF) observations. Our method addresses the problem of 3D sound sources mapping using a linear microphone array, which only provides one DOF observations making the estimation of the sound sources location more challenging. In the proposed method, multi hypotheses tracking is combined with a new sound source parametrisation to provide with a good initial guess for an online optimisation strategy. A joint optimisation is carried out to estimate 6 DOF sensor poses and 3 DOF landmarks together with the sound sources locations. Additionally, a dedicated sensor model is proposed to accurately model the noise of the Direction of Arrival (DOA) observation when using a linear microphone array. Comprehensive simulation and experimental results show the effectiveness of the proposed method. In addition, a real-time implementation of our method has been made available as open source software for the benefit of the community.

Building large-scale 2.5D maps when spatial correlations are considered can be quite expensive, but there are clear advantages when fusing data. While optimal submapping strategies have been explored previously in covariance-form using Gaussian Process for large-scale mapping, this paper focuses on transferring such concepts into information form. By exploiting the conditional independence property of the Gaussian Markov Random Field (GMRF) models, we propose a submapping approach to build a nearly optimal global 2.5D map. In the proposed approach data is fused by first fitting a GMRF to one sensor dataset; then conditional independent submaps are inferred using this model and updated individually with new data arrives. Finally, the information is propagated from submap to submap to later recover the fully updated map. This is efficiently achieved by exploiting the inherent structure of the GMRF, fusion and propagation all in information form. The key contribution of this paper is the derivation of the algorithm to optimally propagate information through submaps by only updating the common parts between submaps. Our results show the proposed method reduces the computational complexity of the full mapping process while maintaining the accuracy. The performance is evaluated on synthetic data from the Canadian Digital Elevation Data.

Acquiring the accurate 3-D position of a target person around a robot provides fundamental and valuable information that is applicable to a wide range of robotic tasks, including home service, navigation and entertainment. This paper presents a real-time robotic 3-D human tracking system which combines a monocular camera with an ultrasonic sensor by the extended Kalman filter (EKF). The proposed system consists of three sub-modules: monocular camera sensor tracking model, ultrasonic sensor tracking model and multi-sensor fusion. An improved visual tracking algorithm is presented to provide partial location estimation (2-D). The algorithm is designed to overcome severe occlusions, scale variation, target missing and achieve robust re-detection. The scale accuracy is further enhanced by the estimated 3-D information. An ultrasonic sensor array is employed to provide the range information from the target person to the robot and Gaussian Process Regression is used for partial location estimation (2-D). EKF is adopted to sequentially process multiple, heterogeneous measurements arriving in an asynchronous order from the vision sensor and the ultrasonic sensor separately. In the experiments, the proposed tracking system is tested in both simulation platform and actual mobile robot for various indoor and outdoor scenes. The experimental results show the superior performance of the 3-D tracking system in terms of both the accuracy and robustness.

In this paper, we tackle the direct and inverse problems for the Remote-Field Eddy-Current (RFEC) technology. The direct problem is the sensor model, where given the geometry the measurements are obtained. Conversely, the inverse problem is where the geometry needs to be estimated given the field measurements. These problems are particularly important in the field of Non-Destructive Testing (NDT) because they allow assessing the quality of the structure monitored. We solve the direct problem in a parametric fashion using Least Absolute Shrinkage and Selection Operation (LASSO). The proposed inverse model uses the parameters from the direct model to recover the thickness using least squares producing the optimal solution given the direct model. This study is restricted to the 2D axisymmetric scenario. Both, direct and inverse models, are validated using a Finite Element Analysis (FEA) environment with realistic pipe profiles.

2.5D maps are preferable for representing the environment owing to their compactness. When noisy observations from multiple diverse sensors at different resolutions are available, the problem of 2.5D mapping turns to how to compound the information in an effective and efficient manner. This paper proposes a generic probabilistic framework for fusing efficiently multiple sources of sensor data to generate amendable, high-resolution 2.5D maps. The key idea is to exploit the sparse structure of the information matrix. Gaussian Markov Random Fields are employed to learn a prior map, which uses the conditional independence property between spatial location to obtain a representation of the state with a sparse information matrix. This prior map encoded in information form can then be updated with other sources of sensor data in constant time. Later, mean state vector and variances can be also efficiently recovered using sparse matrices techniques. The proposed approach allows accurate estimation of 2.5D maps at arbitrary resolution, while incorporating sensor noise and spatial dependency in a statistically sound way. We apply the proposed framework to pipe wall thickness mapping and fuse data from two diverse sensors that have different resolutions. Experimental results are compared with three other methods, showing that, while greatly reducing computation time, the proposed framework is able to capture in large extend the spatial correlation to generate equivalent results to the computationally expensive optimal fusion method in covariance form with a Gaussian Process prior.

This work exploits modeling spatial correlation in 2.5D data using Gaussian Processes (GPs), and produces constrained sampling realizations on these models to improve certainty in the predictions by means of integrating additional sparse information. Data organized in 2.5D such as elevation and thickness maps has been extensively studied in the fields of robotics and geostatistics. These maps are typically represented as a probabilistic 2D grid that stores an estimated value (height or thickness) for each cell. With the increasing popularity and deployment of robotic devices for infrastructure inspection, 2.5D data becomes a common interpretation of the condition of the target being inspected. Modeling the spatial dependencies and making inferences on new grid locations is a common task that has been addressed using GPs, but inference results on locations which are weakly correlated with the training data are generally not sufficiently informative and distinctly uncertain. The predictive capability of the proposed framework, which is applicable to any 2.5D data, is demonstrated with field inspection data from pipelines. Specifically, sparse and complementary measurements from alternative sensing modalities have been incorporated into the model to predict in more detail local thickness conditions where GP training data is limited. The output of this work aims to probabilistically present variations of the target in the case that both accuracy and reasonable diversity are of significant interest.

This paper investigates sound source mapping in a real environment using a mobile robot. Our approach is based on audio ray tracing which integrates occupancy grids and sound source localization using a laser range finder and a microphone array. Previous audio ray tracing approaches rely on all observed rays and grids. As such observation errors caused by sound reflection, sound occlusion, wall occlusion, sounds at misdetected grids, etc. can significantly degrade the ability to locate sound sources in a map. A three-layered selective audio ray tracing mechanism is proposed in this work. The first layer conducts frame-based unreliable ray rejection (sensory rejection) considering sound reflection and wall occlusion. The second layer introduces triangulation and audio tracing to detect falsely detected sound sources, rejecting audio rays associated to these misdetected sounds sources (short-term rejection). A third layer is tasked with rejecting rays using the whole history (long-term rejection) to disambiguate sound occlusion. Experimental results under various situations are presented, which proves the effectiveness of our method.

In this paper, we propose a framework to map stationary sound sources while simultaneously localise a moving robot. Conventional methods for localisation and sound source mapping rely on a microphone array and either, 1) a proprioceptive sensor only (such as wheel odometry) or 2) an additional exteroceptive sensor (such as cameras or lasers) to get accurately the robot locations. Since odometry drifts over time and sound observations are bearing-only, sparse and extremely noisy, the former can only deal with relatively short trajectories before the whole map drifts. In comparison, the latter can get more accurate trajectory estimation over long distances and a better estimation of the sound source map as a result. However, in most of the work in the literature, trajectory estimation and sound source mapping are treated as uncorrelated, which means an update on the robot trajectory does not propagate properly to the sound source map. In this paper, we proposed an efficient method to correlate robot trajectory with sound source mapping by exploiting the conditional independence property between two maps estimated by two different Simultaneous Localisation and Mapping (SLAM) algorithms running in parallel. In our approach, the first map has the flexibility that can be built with any SLAM algorithm (filtering or optimisation) to estimate robot poses with an exteroceptive sensor. The second map is built by using a filtering-based SLAM algorithm locating all stationary sound sources parametrised with Inverse Depth Parametrisation (IDP). Robot locations used during IDP initialisation are the common features shared between the two SLAM maps, which allow to propagate information accordingly. Comprehensive simulations and experimental results show the effectiveness of the proposed method.

Typically 2.5D maps provide a compact and efficient representation of the environment. When sensor data is obtained from multiple sets of noisy measurements at differing resolutions, the problem of compounding this information together to provide an effective and efficient means of mapping is not trivial, particularly as the size of the environment increases. In this paper, we propose a general framework for integrating heterogeneous sensor data to obtain large-scale 2.5D probabilistic maps. Gaussian Processes are used to generate a prior map that learns the spatial correlation between nearby points. Bayesian data fusion is then employed to update these prior maps with new measurements from distinct sensor modalities. In order to deal with large scale data, a novel submapping strategy is introduced to perform the fusion step efficiently in dealing with large covariance matrices. Submaps are first marginalised from the learned correlated prior and then updated based on the property of conditional independence. Most notably, the technique lends itself to generate accurate estimates at arbitrary resolutions and is able to handle varying noise from disparate sensor sources. The framework is applied to pipeline thickness mapping, with experimental results in fusing a high-resolution sensor and a low-resolution sensor showing the ability of the proposed technique to capture spatial correlations to come up with more accurate results when compared with a naïve fusion approach.

An algorithm is proposed for matching data from different sensing modalities. The problem is formalised as a kidnapped robot problem, where Bayesian fusion is used to find the most likely location where both modalities agree. The key idea of our algorithm is to model the correlation between the two modalities as a likelihood used to update a location prior. Data, in this case, is represented as 2.5D thickness maps from a laser scanner and a Remote Field Eddy Current (RFEC) tool, used in non-destructive testing to assess the condition of infrastructures. The laser data is limited, while RFEC data is continuous. Given some prior in location, the aim is to find the 2.5D thickness map from the laser that corresponds to the RFEC data, which should be noted is highly noisy. Real data from CCTV inspections of water pipes are used to validate the proposed approach.

The problem of active perception with an autonomous robot is studied in this paper. It is proposed that the exploratory behavior of the robot be controlled using mutual information (MI) surfaces between the current map and a one-step look ahead measurements. MI surfaces highlight informative areas for exploration. A novel method for computing these surfaces is described. An approach that exploits structural dependencies of the environment and handles sparse sensor measurements to build a continuous model of the environment, that can then be used to generate MI surfaces is also proposed. A gradient field of occupancy probability distribution is regressed from sensor data as a Gaussian Process and provide frontier boundaries for further exploration. The continuous global frontier surface completely describes unexplored regions and, inherently, provides an automatic termination criterion for a desired sensitivity. The results from publicly available datasets confirm an average improvement of the proposed methodology over comparable standard and state-of-the-art exploratory methods available in the literature by more than 20% and 13% in travel distance and map entropy reduction rate, respectively.

In this paper, an approach for sound source localisation and calibration of an asynchronous microphone array is proposed to be solved simultaneously. A graph-based Simultaneous Localisation and Mapping (SLAM) method is used for this purpose. Traditional sound source localisation using a microphone array has two main requirements. Firstly, geometrical information of microphone array is needed. Secondly, a multichannel analog-to-digital converter is required to obtain synchronous readings of the audio signal. Recent works aim at releasing these two requirements by estimating the time offset between each pair of microphones. However, it was assumed that the clock timing in each microphone sound card is exactly the same, which requires the clocks in the sound cards to be identically manufactured. A methodology is hereby proposed to calibrate an asynchronous microphone array using a graph-based optimisation method borrowed from the SLAM literature, effectively estimating the array geometry, time offset and clock difference/drift rate of each microphone together with the sound source locations. Simulation and experimental results are presented, which prove the effectiveness of the proposed methodology in achieving accurate estimates of the microphone array characteristics needed to be used on realistic settings with asynchronous sound devices.

This paper proposes an intelligent mobility aid framework aimed at mitigating the impact of cognitive and/or physical user deficiencies by performing suitable mobility assistance with minimum interference. To this end, a user action model using Gaussian Process Regression (GPR) is proposed to encapsulate the probabilistic and nonlinear relationships among user action, state of the environment and user intention. Moreover, exploiting the analytical tractability of the predictive distribution allows a sequential Bayesian process for user intention estimation to take place. The proposed scheme is validated on data obtained in an indoor setting with an instrumented robotic wheelchair augmented with sensorial feedback from the environment and user commands as well as proprioceptive information from the actual vehicle, achieving accuracy in near real-time of ~80%. The initial results are promising and indicating the suitability of the process to infer user driving behaviors within the context of ambulatory robots designed to provide assistance to users with mobility impairments while carrying out regular daily activities.

An non-linear Bayesian regression engine for robotic tracking based on an ultrasonic/RF sensor unit is presented in this paper. The proposed system is able to maintain systematic tracking of a leading human in indoor/outdoor settings with minimalistic instrumentation. Compared to popular camera based localization system the sonar array/RF based system has the advantage of being insensitive to background light intensity changes, a primary concern in outdoor environments. In contrast to single-plane laser range finder based tracking the proposed scheme is able to better adapt to small terrain variations, while at the same time being a significantly more affordable proposition for tracking with a robotic unit. A key novelty in this work is the utilisation of Gaussian Process Regression (GPR) to build a model for the sensor unit, which is shown to compare favourably against traditional linear triangulation approaches. The covariance function yield by the GPR sensor model also provides the additional benefit of outlier rejection. We present experimental results of indoors and outdoors tracking by mounting the sensor unit on a Garden Utility Transportation System (GUTS) robot and compare the proposed approach with linear triangulation which clearly show the inference engine capability to generalise relative localisation of human and a marked improvement in tracking accuracy and robustness.

An information-driven autonomous robotic exploration method on a continuous representation of unknown environments is proposed in this paper. The approach conveniently handles sparse sensor measurements to build a continuous model of the environment that exploits structural dependencies without the need to resort to a fixed resolution grid map. A gradient field of occupancy probability distribution is regressed from sensor data as a Gaussian process providing frontier boundaries for further exploration. The resulting continuous global frontier surface completely describes unexplored regions and, inherently, provides an automatic stop criterion for a desired sensitivity. The performance of the proposed approach is evaluated through simulation results in the well-known Freiburg and Cave maps.

The problem of inferring human behaviour is naturally complex: people interact with the environment and each other in many different ways, and dealing with the often incomplete and uncertain sensed data by which the actions are perceived only compounds the difficulty of the problem. In this paper, we propose a framework whereby these elaborate behaviours can be naturally simplified by decomposing them into smaller activities, whose temporal dependencies can be more efficiently represented via probabilistic hierarchical learning models. In this regard, patterns of a number of activities typically carried out by users of an ambulatory aid device have been identified with the aid of a Hierarchical Hidden Markov Model (HHMM) framework. By decomposing the complex behaviours into multiple layers of abstraction the approach is shown capable of modelling and learning these tightly coupled human-machine interactions. The inference accuracy of the proposed model is proven to compare favourably against more traditional discriminative models, as well as other compatible generative strategies to provide a complete picture that highlights the benefits of the proposed approach, and opens the door to more intelligent assistance with a robotic mobility aid.

Pipe thickness maps are used to assess the condition in pipelines. Thickness maps are a 2.5D representation similar to elevation maps in robotics. Probabilistic frameworks, however, have barely been used in this context. This paper presents a general approach for generating probabilistic maps from heterogeneous sensor data. The key idea is to learn the spatial correlation of a sensor through Gaussian Process models and use it as priors for Bayesian fusion. This approach is applied to the novel application of pipe thickness mapping. Data from a 3D laser scanner on the outer surface of the pipe and thickness measurements from a contact ultrasonic sensor are fused into a single thickness map with associated uncertainty. Moreover, a dedicated algorithm to model the ultrasonic sensor using kernel density estimation is also proposed. The overall approach is evaluated using the full 3D profile (outer and inner surfaces) of the pipe section as ground truth.

This article proposes a probabilistic approach to account for robot stability uncertainty when planing motions over uneven terrains. A novel probabilistic stability criterion derived from the cumulative distribution of a tip-over metric is introduced that allows a safety constraint to be dynamically updated by available sensor data as it becomes available. The proposed safety constraint authorizes the planner to generates more conservative motion plans for areas with higher levels of uncertainty, while avoids unnecessary caution in well-known areas. The proposed systematic approach is particularly applicable to reconfigurable robots that can assume safer postures when required, although is equally valid for fixed-configuration platforms to choose safer paths to follow. The advantages of planning with the proposed probabilistic stability metric are demonstrated with data collected from an indoor rescue arena, as well as an outdoor rover testing facility.

Assessment of the condition of underground pipelines is crucial to avoid breakages. Autonomous in-line inspection tools provided with Non-destructive Technology (NDT) sensors to assess large sections of the pipeline are commonly used for these purposes. An example of such sensors based on Eddy currents is the Remote Field Technology (RFT). A crucial step during in-line inspections is the detection of construction features, such as joints and elbows, to accurately locate and size specific defects within pipe sections. This step is often performed manually with the aid of visual data, which results in slow data processing. In this paper, we propose a generic framework to automate the detection and verification of these construction features using both NDT sensor data and visual images. Firstly, supervised learning is used to identify the construction features in the NDT sensor signals. Then, image processing is employed to verify the selection. Results are presented with data from a RFT tool, for which a specialised descriptor has been designed to characterise and classify its signal features. Furthermore, the construction feature is displayed in the image, once it is identified in the RFT data and detected in the visual data. A visual odometry algorithm has been implemented to locate the visual data with respect to the RFT data. About 800 meters of these multi-modal data are evaluated to test the validity of the proposed approach.

Stability prediction is an important concern for mobile robots operating in rough environments. Having the capacity to predict areas of instability means pro-actively being able to plan safer traversable paths. The most influential tip-over stability measures are based on two criteria, the robot's center of mass (CM) and the supporting polygon (SP) defined by the convex area spanned between the ground contact-points. However, there is significant uncertainty associated with many parameters in the planning pipe-line: the actual robot kino-dynamic model, its localisation in the ground, and the terrain models, particularly in uneven terrain. This article proposes a statistical analysis of stability prediction to account for some of the uncertainties. This is accomplished using the force angle (FA) stability measure for a reconfigurable multi-tracked vehicle fitted with flippers, a manipulator arm and a sensor head. Probability density function (PDF) of contact-points, CM and the FA stability measure are numerically estimated, with simulation results performed on the open dynamics engine (ODE) simulator based on uncertain parameters. Two techniques are presented: a conventional Monte Carlo scheme, and a structured unscented transform (UT) which results in significant improvement in computational efficiency. Experimental results on maps obtained from a range camera fitted on the sensor head while the robot traverses over a ramp and a series of steps are presented that confirms the validity of the proposed probabilistic stability prediction method.

In this paper we use a Hierarchical Hidden Markov Model (HHMM) to represent and learn complex activities/task performed by humans/robots in everyday life. Action primitives are used as a grammar to represent complex human behaviour and learn the interactions and behaviour of human/robots with different objects. The main contribution is the use of a probabilistic model capable of representing behaviours at multiple levels of abstraction to support the proposed hypothesis. The hierarchical nature of the model allows decomposition of the complex task into simple action primitives. The framework is evaluated with data collected for tasks of everyday importance performed by a human user.

This paper presents the development of a low-cost vision-based robotic wheelchair system towards autonomous convoying. The non-holonomic follower vehicle obtains visual real-time pose data of a known coplanar target installed on the back of the leading vehicle. This allows the tracking vehicle to mimic the path of the preceding vehicle, while maintaining a safe distance behind it with the aid of a controller based on the robot's kinematics constraints. A back-end visual filter is proposed in the planning strategy to overcome the noisy environmental information acquired from the camera as it tracks the vehicle in front. The effectiveness of the approach is evaluated in an indoor setting using data obtained from an instrumented wheelchair platform and a low-cost camera, and validated with observations from a laser range finder and derived (known) maps of the environment.

This paper proposes a motion planning strategy for reconfigurable mobile robots in uneven terrain. Paths that guarantee stability while at the same time maximise the height of the sensor payload, thereby enhancing the capacity of the robot to explore the environment are obtained using a search algorithm based on A*. This is particularly applicable to operations such as search and rescue where observing the environment for locating victims is the major objective, although the proposed technique can be generalised to incorporate other potentially conflicting objectives such as minimising energy. The proposed planning strategy looks at exploiting the (possibly incomplete) environment information available to the robot and/or operator as it explores novel terrain. The effectiveness of the approach is evaluated using data obtained from a multi-tracked robot fitted with a manipulator arm and a range camera in a mock-up search and rescue arena.

We present an active exploration strategy that complements Pose SLAM [1] and optimal navigation in Pose SLAM [2]. The method evaluates the utility of exploratory and place revisiting sequences and chooses the one that minimizes overall map and path entropies. The technique considers trajectories of similar path length taking marginal pose uncertainties into account. An advantage of the proposed strategy with respect to competing approaches is that to evaluate information gain over the map, only a very coarse prior map estimate needs to be computed. Its coarseness is independent and does not jeopardize the Pose SLAM estimate. Moreover, a replanning scheme is devised to detect significant localization improvement during path execution. The approach is tested in simulations in a common publicly available dataset comparing favorably against frontier based exploration.

Detection of individuals' intention from a sequence of actions is an open and complex problem. In this paper we present a smart walker as mobility aid which can interpret the users' behaviour patterns to recognize their intentions and consequently act as an intelligent assistant. The result of the experiments performed in this paper demonstrates the potential of dynamic bayesian networks (DBN), in relation to their dynamic and unsupervised nature, for realistic human-robot interaction modelling.

This paper presents a framework for modelling the interaction between a human operator and a robotic device, that enables the robot to collaborate with the human to jointly accomplish tasks. States of the system are captured in a model based on a partially observable Markov decision process (POMDP). States representing the human operator are motivated by behaviours from the psychology of the human action cycle. Hierarchical nature of these states allows the exploitation of data structures based on algebraic decision diagrams (ADD) to efficiently solve the resulting POMDP. The proposed framework is illustrated using two examples from assistive robotics; a robotic wheel chair and an intelligent walking device. Experimental results from trials conducted in an office environment with the wheelchair is used to demonstrate the proposed technique.

An efficient automatic solution to the feature-based simultaneous localisation and mapping (SLAM) of mobile robots operating in conditions where a number of competing objectives operate simultaneously is proposed. The formulation quantitatively measures the merit of incoming data with respect to multiple priorities, automatically adjusting the amount of observations to be used in the estimation process for the best possible combined outcome. The methodology enables a selection mechanism which can efficiently exploit the observations available to the robot to best fulfil the objectives of differing tasks throughout the course of a mission, e.g. localisation, mapping, exploration, feature distribution, searching for specific objects or victims, etc. The work is particularly motivated by navigation in three-dimensional terrains, and an example considering the objectives of robot localisation and map expansion in a search and rescue environment using an RGB-D camera is utilised for discussion and results.

A generic methodology to plan increasingly stable paths for mobile platforms travelling over uneven terrain is proposed in this paper. This is accomplished by extending the Fast Marching level-set method of propagating interfaces in 3D lattices with an analytical kyno-dynamic metric which embodies robot stability in the given terrain. This is particularly relevant for reconfigurable platforms which significantly modify their mass distribution through posture adaptation, such as robots equipped with manipulator arms or varying traction arrangements. Results obtained from applying the proposed strategy in a mobile rescue robot operating on simulated and real terrain data illustrate the validity of the proposed strategy.

This paper describes a simultaneous localization and mapping (SLAM) algorithm for use in unstructured environments that is effective regardless of the geometric complexity of the environment. Features are described using B-splines as modeling tool, and the set of control points defining their shape is used to form a complete and compact description of the environment, thus making it feasible to use an extended Kalman-filter (EKF) based SLAM algorithm. This method is the first known EKF-SLAM implementation capable of describing general free-form features in a parametric manner. Efficient strategies for computing the relevant Jacobians, perform data association, initialization, and map enlargement are presented. The algorithms are evaluated for accuracy and consistency using computer simulations, and for effectiveness using experimental data gathered from different real environments.

This paper presents an intelligent decision-making agent to assist wheelchair users in their daily navigation activities. Several navigational techniques have been successfully developed in the past to assist with specific behaviours such as "door passing" or "corridor following". These shared control strategies normally require the user to manually select the level of assistance required during use. Recent research has seen a move towards more intelligent systems that focus on forecasting users' intentions based on current and past actions. However, these predictions have been typically limited to locations immediately surrounding the wheelchair. The key contribution of the work presented here is the ability to predict the users' intended destination at a larger scale, that of a typical office arena. The systems relies on minimal user input - obtained from a standard wheelchair joystick - in conjunction with a learned Partially Observable Markov Decision Process (POMDP), to estimate and subsequently drive the user to his destination. The projection is constantly being updated, allowing for true user- platform integration. This shifts users' focus from fine motor- skilled control to coarse control broadly intended to convey intention. Successful simulation and experimental results on a real wheelchair robot demonstrate the validity of the approach.

This paper presents a novel solution for building three-dimensional dense maps in unknown and unstructured environment with reduced computational costs. This is achieved by giving the robot the ‘intelligence’ to select, out of the steadily collected data, the maximally informative observations to be used in the estimation of the robot location and its surroundings. We show that, although the actual evaluation of information gain for each frame introduces an additional computational cost, the overall efficiency is significantly increased by keeping the matrix compact. The noticeable advantage of this strategy is that the continuously gathered data is not heuristically segmented prior to be input to the filter. Quite the opposite, the scheme lends itself to be statistically optimal and is capable of handling large data sets collected at realistic sampling rates. The strategy is generic to any 3D feature-based simultaneous localization and mapping (SLAM) algorithm in the information form, but in the work presented here it is closely coupled to a proposed novel appearance-based sensory package. It consists of a conventional camera and a range imager, which provide range, bearing and elevation inputs to visual salient features as commonly used by three-dimensional point-based SLAM, but it is also particularly well adapted for lightweight mobile platforms such as those commonly employed for Urban Search and Rescue (USAR), chosen here to demonstrate the excellences of the proposed strategy.

This paper presents a novel vision-based sensory package and an information-efficient simultaneous localization and mapping (SLAM) algorithm. Together, we offer a solution for building 3-D dense map in an unknown and unstructured environment with minimal computational costs. The sensory package we adopt consists of a conventional camera and a range imager, which provide range and bearing and elevation inputs as commonly used by 3-D feature-based SLAM. In addition, we propose an algorithm to give the robots the `intelligencerdquo to select, out of the steadily collected data, the maximally informative observations to be used in the estimation process. We show that, although the actual evaluation of information gain for each frame introduces an additional computational cost, the overall efficiency is significantly increased by keeping the matrix compact. The noticeable advantage of this strategy is that the continuously gathered data are not heuristically segmented prior to being input to the filter. Quite the opposite, the scheme lends itself to be statistically optimal and is capable of handling large datasets collected at realistic sampling rates.

The main contribution of this paper is a novel stereo-based algorithm which serves as a tool to examine the viability of stereo vision solutions to the simultaneous localisation and mapping (SLAM) for large indoor environments. Using features extracted from the scale invariant feature transform (SIFT) and depth maps from a small vision system (SVS) stereo head, an extended Kalman filter (EKF) based SLAM algorithm, that allows the independent use of information relating to depth and bearing, is developed. By means of a map pruning strategy for managing the computational cost, it is demonstrated that statistically consistent location estimates can be generated for a small (6 m times 6 m) structured office environment, and in a robotics search and rescue arena of similar size. It is shown that in a larger office environment, the proposed algorithm generates location estimates which are topologically correct, but statistically inconsistent. A discussion on the possible reasons for the inconsistency is presented. The paper highlights that, despite recent advances, building accurate geometric maps of large environments with vision only sensing is still a challenging task