Underwater autonomous robotic operations require online localization and 3D mapping. Because of the absence of absolute positioning underwater, these tasks strongly rely on embedded sensors, including proprioceptive or navigation sensors - which can be fused for an odometry, - and exteroceptive sensors. One of the most popular exteroceptive sensors for underwater is the imaging sonar, which emits a large fan-shaped acoustic signal and estimates the position of the surrounding obstacles from a measure of the reflected signal. This paper addresses underwater online localization and 3D mapping using a forward looking, wide-aperture imaging sonar and vehicle's intrinsic navigation estimates. We introduce 3DSSDF (3D Sonar Reconstruction Using Signed Distance Functions), a new localization and 3D mapping algorithm based on signed distance functions, which is evaluated in simulation and on real data, in man-made and natural environments. Comparisons to reference trajectories and maps demonstrate that, in our tests, 3DSSDF efficiently corrects navigation drift and that trajectory and map accuracy is always below 1 m and below 1% of the distanced travelled, which can be sufficient for the safe inspection of natural or artificial underwater structures.

Simulations are highly valuable in marine robotics, offering a cost-effective and controlled environment for testing in the challenging conditions of underwater and surface operations. Given the high costs and logistical difficulties of real-world trials, simulators capable of capturing the operational conditions of subsea environments have become key in developing and refining algorithms for remotely-operated and autonomous underwater vehicles. This paper highlights recent enhancements to the Stonefish simulator, an advanced open-source platform supporting development and testing of marine robotics solutions. Key updates include a suite of additional sensors, such as an event-based camera, a thermal camera, and an optical flow camera, as well as, visual light communication, support for tethered operations, improved thruster modelling, more flexible hydrodynamics, and enhanced sonar accuracy. These developments and an automated annotation tool significantly bolster Stonefish's role in marine robotics research, especially in the field of machine learning, where training data with a known ground truth is hard or impossible to collect. https://github.com/patrykcieslak/stonefish

Autonomous motion planning is critical for efficient and safe underwater manipulation in dynamic marine environments. Current motion planning methods often fail to effectively utilize prior motion experiences and adapt to real-time uncertainties inherent in underwater settings. In this paper, we introduce an Adaptive Heuristic Motion Planner framework that integrates a Heuristic Motion Space (HMS) with Bayesian Networks to enhance motion planning for autonomous under-water manipulation. Our approach employs the Probabilistic Roadmap (PRM) algorithm within HMS to optimize paths by minimizing a composite cost function that accounts for distance, uncertainty, energy consumption, and execution time. By leveraging HMS, our framework significantly reduces the search space, thereby boosting computational performance and enabling real-time planning capabilities. Bayesian Networks are utilized to dynamically update uncertainty estimates based on real-time sensor data and environmental conditions, thereby refining the joint probability of path success. Through extensive simulations and real-world test scenarios, we showcase the advantages of our method in terms of enhanced performance and robustness. This probabilistic approach significantly advances the capability of autonomous underwater robots, ensuring optimized motion planning in the face of dynamic marine challenges.

This study introduces MarineGym, a high-performance reinforcement learning platform tailored for underwater robotics. It aims to address the limitations of existing underwater simulation environments in terms of reinforcement learning compatibility, training efficiency, and standardized benchmarking. MarineGym integrates a proposed GPU-accelerated hydrodynamic plugin based on Isaac Sim, achieving a rollout speed of 250,000 frames per second on a single NVIDIA RTX 3060 GPU. It also provides five models of unmanned underwater vehicles, multiple propulsion systems, and a set of predefined tasks covering core underwater control challenges. Additionally, the domain randomization toolkit allows flexible adjustments of the simulation and task parameters during training to improve the Sim2Real transfer. Further benchmark experiments demonstrate that MarineGym improves training efficiency over existing platforms and supports robust policy adaptation under various perturbations in the marine environment. We expect this platform to drive further advancements in RL research for underwater robotics. For more details about MarineGym and its applications, please visit our project page: https://marine-gym.com/.

Despite the potential of mobile manipulators and applications where robots require a force-controlled physical interaction with the environment, the majority of robot automation nowadays is still based on fixed manipulators for free-motion tasks (e.g. welding, pick and place, or painting). In this work, we propose a control solution for omnidirectional mobile manipulators in force-tracking tasks, interacting with unknown surface geometries and with a human teleoperator in the control loop. Keeping a teleoperator in the loop makes the system widely applicable to unstructured environments. With little effort, a human can take care of the mobile base navigation, self-collisions, and collisions with the environment, as well as selecting the area of the asset surface to process. The teleoperator interfaces with the robot platform by commanding motion in the mobile base to increase the arm’s workspace and manoeuvrability. The operator can also command the movement of the end-effector, sliding on the surface geometry to process a specific area. Alternatively, he can let the controller execute a parametric trajectory (spiral or raster) for an autonomous area coverage and meanwhile telecommand the base in order to keep the arm in configurations with good dexterity. The autonomous controller, on the other hand, takes responsibility for following the unknown contour on the manipulated surface by only taking observations from a force/torque sensor attached to the arm’s wrist, exerting a prescribed force, and handling the motion control in the base and the arm so that both can follow their respective task requests. Overall, we have developed a user-friendly control scheme, where an operator with little training and using a joystick, can guide the robot system to perform a physically interactive task on the surface of an asset.

In robotics, occupancy grids serve as required repositories of information about the environment in numerous applications. One such critical application is Simultaneous Localization and Mapping (SLAM), where robots dynamically scan and explore their surroundings while in motion. In the context of extended-duration missions, it becomes imperative to confront the complexities linked to the expansion of occupancy grids as well as handling loop closure detection. These challenges primarily revolve around two key aspects: enabling the seamless expansion of the map on multiple occasions, thus avoiding the need to map smaller regions in numerous separate missions, and ensuring real-time updates to the map to sustain the robot’s knowledge base and enhance its responsiveness. To address these challenges, we introduce an innovative map called Frustum Accelerated GPU-Based Grid Map (FRAGG-Map). This map adopts a highly parallelizable 3D grid structure and leverages the power of CUDA kernels to facilitate efficient insertion of point-clouds and enables real-time updates of the map. FRAGG-Map identifies the portions of the map that require updates and utilises the GPU to update them, significantly enhancing computational performance. Our results show that FRAGG-Map can run 31 times faster than OctoMap, significantly outperforming state-of-the-art methods.

In this paper, we propose using online public maps, e.g., OpenStreetMap (OSM), for large-scale radar-based localization without needing a prior sensing map. This can potentially extend the localization system to anywhere worldwide without building, saving, or maintaining a sensing map, as long as an online public map covers the operating area. Existing methods using OSM only use route network or semantics information. These two sources of information are not combined in the previous works, while our proposed system fuses them to improve localization accuracy. Our experiments, on three open datasets collected from three different continents, show that the proposed system outperforms the state-of-the-art localization methods, reducing up to 50% of position errors. We release an open-source implementation for the community.

This paper presents an adaptive heading approach for perception awareness during trajectory following. By adapting the heading of a robot to improve the feature tracking in the current mapped environment, the accuracy in localisation can be improved. This can have a significant advantage for autonomous operations in GPS-denied environments such as subsea or in caves. The aim of the proposed approach is to position the sensor used for perception and feature tracking in such a way that it; obtains a view that contains a good observation of the previously mapped environment, face forward along the direction of travel, reduces the change in heading and view the perceived environment along the surface's estimated normals. These 4 objectives create a weighted utility function that is used to find the most beneficial heading. The benefit is a system that improves feature tracking for simultaneous localisation and mapping (SLAM) while considering the safety of the robot by being aware of its surrounding. To sense the environment, a simulated sensor is discretised to a set of vertical rays based on the vertical field of view. The vertical rays are swept 360 degrees around a position to evaluate for a new heading. This allows for the simulated sensor data from ray casting to be reused and therefore reduces the computational load to find the heading which maximises the utility function. The paper is focused on holonomic robots capable of controlling the robot's heading or sensor orientation independently from the position. We present results and evaluation in a simulated environment where we show a great improvement in the SLAM's pose estimation. In addition, we endow an autonomous underwater vehicle (AUV) with the proposed approach during field trials and present the result in two different environments.

The extrinsic parameters play a crucial role in multi-sensor fusion, such as visual-inertial Simultaneous Localization and Mapping(SLAM), as they enable the accurate alignment and integration of measurements from different sensors. However, extrinsic calibration is challenging in scenarios, such as underwater, where in-view structures are scanty and visibility is limited, causing incorrect extrinsic calibration due to insufficient motion on all degrees of freedom. In this paper, we propose an entropy-based active extrinsic calibration algorithm leverages observability analysis and information entropy to enhance the accuracy and reliability of extrinsic calibration. It determines the system observability numerically by using singular value decomposition (SVD) of the Fisher Information Matrix (FIM). Furthermore, when the extrinsic parameter is not fully observable, our method actively searches for the next best motion to recover the system's observability via entropy-based optimization. Experimental results on synthetic data, in a simulation, and using an actual underwater vehicle verify that the proposed method is able to avoid the calibration failure while improving the calibration accuracy and reliability.

Unmanned underwater vehicles are crucial for deep-sea exploration and inspection without imposing any danger to human life due to extreme environmental conditions. But, designing a robust controller that can cope with model uncertainties, external disturbances, and time delays for such vehicles is a challenge. This paper implements a sliding mode position control algorithm with a time-delay estimation term to a remotely operated underwater vehicle to deal with disturbances, such as waves, and time delays. The controller is implemented on an underwater vehicle (BlueRov) and compared with a proportional-integral-derivative (PID) controller in a wave tank with different disturbances and when there exist delays within the communication channel. The experimental results show that the proposed control method provides better performance than the conventional PID in the presence of extreme disturbances with less control efforts.

Safe autonomous navigation is an essential and challenging problem for robots operating in highly unstructured or completely unknown environments. Under these conditions, not only robotic systems must deal with limited localization information but also their maneuverability is constrained by their dynamics and often suffers from uncertainty. In order to cope with these constraints, this article proposes an uncertainty-based framework for mapping and planning feasible motions online with probabilistic safety guarantees. The proposed approach deals with the motion, probabilistic safety, and online computation constraints by: 1) incrementally mapping the surroundings to build an uncertainty-aware representation of the environment and 2) iteratively (re)planning trajectories to goal that is kinodynamically feasible and probabilistically safe through a multilayered sampling-based planner in the belief space. In-depth empirical analyses illustrate some important properties of this approach, namely: 1) the multilayered planning strategy enables rapid exploration of the high-dimensional belief space while preserving asymptotic optimality and completeness guarantees and 2) the proposed routine for probabilistic collision checking results in tighter probability bounds in comparison to other uncertainty-aware planners in the literature. Furthermore, real-world in-water experimental evaluation on a nonholonomic torpedo-shaped autonomous underwater vehicle and simulated trials in an urban environment on an unmanned aerial vehicle demonstrate the efficacy of the method and its suitability for systems with limited onboard computational power. Note to Practitioners—Emergent robotic applications require operating in previously unmapped scenarios. This article presents a unified mapping–planning strategy that enables robots to navigate autonomously and safely in harsh environments.

In this paper, we propose an approach for robust visual Simultaneous Localisation and Mapping (SLAM) in underwater environments leveraging acoustic, inertial and altimeter/depth sensors. Underwater visual SLAM is challenging due to factors including poor visibility caused by suspended particles in water, a lack of light and insufficient texture in the scene. Because of this, many state-of-the-art approaches rely on acoustic sensing instead of vision for underwater navigation.Building on the sparse visual SLAM system ORB-SLAM2, this paper proposes to improve the robustness of camera pose estimation in underwater environments by leveraging acoustic odometry, which derives a drifting estimate of the 6-DoF robot pose from fusion of a Doppler Velocity Log (DVL), a gyroscope and an altimeter or depth sensor. Acoustic odometry estimates are used as motion priors and we formulate pose residuals that are integrated within the camera pose tracking, local and global bundle adjustment procedures of ORB-SLAM2.The original design of ORB-SLAM2 supports a single map and it enters relocalisation when tracking is lost. This is a significant problem for scenarios where a robot does a continuous scanning motion without returning to a previously visited location. One of our main contributions is to enable the system to create a new map whenever it encounters a new scene where visual odometry can work. This new map is connected with its predecessor in a common graph using estimates from the proposed acoustic odometry. Experimental results on two underwater vehicles demonstrate the increased robustness of our approach compared to baseline ORB-SLAM2 in both controlled, uncontrolled and field environments.

Underwater scenarios are challenging for visual Simultaneous Localization and Mapping (SLAM) due to limited visibility and intermittently losing structures in image views. In this paper, we propose a visual acoustic bundle adjustment system which fuses a camera and a Doppler Velocity Log (DVL) in a graph SLAM framework for reliable underwater localization and mapping. In order to fuse the vision with the acoustic measurements, an calibration algorithm is also designed to estimate extrinsic parameters between a camera and a DVL using features detected in scenes. Experimental results in a tank and an offshore wind farm show the proposed method can achieve better robustness and localization accuracy than pure visual SLAM, especially in visually challenging scenarios, and the extrinsic calibration parameters can be accurately estimated, even when initialized with a random guess.

The sensor technology for water quality monitoring (WQM) has improved during recent years. The cost-effective sensorised tools that can autonomously measure the essential physical-chemical-biological (PCB) variables are now readily available and are being deployed on buoys, boats, and ships. Yet, there is a disconnect between the data quality, data gathering, and data analysis due to the lack of standardized approaches for data collection and processing, spatiotemporal variation of key parameters in water bodies and new contaminants. Such gaps can be bridged with a network of multiparametric sensor systems deployed in water bodies using autonomous vehicles, such as marine robots and aerial vehicles to broaden the data coverage in space and time. Furthermore, intelligent algorithms [e.g., artificial intelligence (AI)] could be employed for standardized data analysis and forecasting. This article presents a comprehensive review of the sensors, deployment, and analysis technologies for WQM. A network of networked water bodies could enhance the global data intercomparability and enable WQM at a global scale to address global challenges related to food (e.g., aqua/agriculture), drinking water, and health (e.g., water-borne diseases).

Reasoning about object grasp affordances allows an autonomous agent to estimate the most suitable grasp to execute a task. While current approaches for estimating grasp affordances are effective, their prediction is driven by hypotheses on visual features rather than an indicator of a proposal's suitability for an affordance task. Consequently, these works cannot guarantee any level of performance when executing a task and, in fact, not even ensure successful task completion. In this work, we present a pipeline for self-assessment of grasp affordance transfer (SAGAT) based on prior experiences. We visually detect a grasp affordance region to extract multiple grasp affordance configuration candidates. Using these candidates, we forward simulate the outcome of executing the affordance task to analyse the relation between task outcome and grasp candidates. The relations are ranked by performance success with a heuristic confidence function and used to build a library of affordance task experiences. The library is later queried to perform one-shot transfer estimation of the best grasp configuration on new objects. Experimental evaluation shows that our method exhibits a significant performance improvement up to 11.7% against current state-of-the-art methods on grasp affordance detection. Experiments on a PR2 robotic platform demonstrate our method's highly reliable deployability to deal with real-world task affordance problems.

Numerous Simultaneous Localization and Mapping (SLAM) algorithms have been presented in last decade using different sensor modalities. However, robust SLAM in extreme weather conditions is still an open research problem. In this paper, RadarSLAM, a full radar based graph SLAM system, is proposed for reliable localization and mapping in large-scale environments. It is composed of pose tracking, local mapping, loop closure detection and pose graph optimization, enhanced by novel feature matching and probabilistic point cloud generation on radar images. Extensive experiments are conducted on a public radar dataset and several self-collected radar sequences, demonstrating the state-of-the-art reliability and localization accuracy in various adverse weather conditions, such as dark night, dense fog and heavy snowfall.

Autonomous vehicles and robots are increasingly being deployed to remote, dangerous environments in the energy sector, search and rescue and the military. As a result, there is a need for humans to interact with these robots to monitor their tasks, such as inspecting and repairing offshore wind-turbines. Conversational Agents can improve situation awareness and transparency, while being a hands-free medium to communicate key information quickly and succinctly. As part of our user-centered design of such systems, we conducted an in-depth immersive qualitative study of twelve marine research scientists and engineers, interacting with a prototype Conversational Agent. Our results expose insights into the appropriate content and style for the natural language interaction and, from this study, we derive nine design recommendations to inform future Conversational Agent design for remote autonomous systems.

Visual place recognition is a fundamental problem for many vision based applications. Sparse feature and deep learning based methods have been successful and dominant over the decade. However, most of them do not explicitly leverage high-level semantic information to deal with challenging scenarios where they may fail. This paper proposes a novel visual place recognition algorithm, termed TextPlace, based on scene texts in the wild. Since scene texts are high-level information invariant to illumination changes and very distinct for different places when considering spatial correlation, it is beneficial for visual place recognition tasks under extreme appearance changes and perceptual aliasing. It also takes spatial-temporal dependence between scene texts into account for topological localization. Extensive experiments show that TextPlace achieves state-of-the-art performance, verifying the effectiveness of using high-level scene texts for robust visual place recognition in urban areas.

Global localization lies at the heart of autonomous navigation and Simultaneous Localization and Mapping (SLAM). The appearance-based approach has been successful, but still faces many open challenges in environments where visual conditions vary significantly over time. In this paper, we propose an integrated solution to leverage object-level dense semantics and spatial understanding of the environment for global localization. Our approach models an environment with 3D dense semantics, semantic graph and their topology. This object-level representation is then used for place recognition via semantic object association, followed by 6-DoF pose estimation by the semantic-level point alignment. Extensive experiments show that our approach can achieve robust global localization under extreme appearance changes. It is also capable of coping with other challenging scenarios, such as dynamic environments and incomplete query observations.

Dynamic environments are challenging for visual SLAM as moving objects can impair camera pose tracking and cause corruptions to be integrated into the map. In this paper, we propose a method for robust dense RGB-D SLAM in dynamic environments which detects moving objects and simultaneously reconstructs the background structure. While most methods employ implicit robust penalisers or outlier filtering techniques in order to handle moving objects, our approach is to simultaneously estimate the camera motion as well as a probabilistic static/dynamic segmentation of the current RGB-D image pair. This segmentation is then used for weighted dense RGB-D fusion to estimate a 3D model of only the static parts of the environment. By leveraging the 3D model for frame-to-model alignment, as well as static/dynamic segmentation, camera motion estimation has reduced overall drift - as well as being more robust to the presence of dynamics in the scene. Demonstrations are presented which compare the proposed method to related state-of-the-art approaches using both static and dynamic sequences. The proposed method achieves similar performance in static environments and improved accuracy and robustness in dynamic scenes.

SONAR mapping of underwater environments leads to dense point-clouds. These maps have large memory footprints, are inherently noisy and consist of raw data with no semantic information. This paper presents an approach to underwater semantic mapping where known man-made structures that appear in multibeam SONAR data are automatically recognised. From a set of SONAR images acquired by an Autonomous Underwater Vehicle (AUV) and a catalogue of `a-priori' 3D CAD models of structures that may potentially be found in the data, our algorithm proceeds in two phases. First we recognise objects using an efficient, rotation-invariant 2D descriptor combined with a histogram matching method. Then, we determine pose using a 6 degree-of-freedom registration of the 3D object to the local scene using a fast 2D correlation, refined with an iterative closest point (ICP)-based method. Once the structures located and identified, we build a semantic representation of the world based on the initial CAD models, resulting in a lightweight yet accurate world model. We demonstrate the applicability of our method on field data acquired by an AUV in Loch Linnhe, Scotland. Our method proves to be suitable for online semantic mapping of a partially man-made underwater environment such as a typical oil field.

In this paper we investigate the application of semi-dense visual Simultaneous Localisation and Mapping (SLAM) to the humanoid robotics domain. Challenges of visual SLAM applied to humanoids include the type of dynamic motion executed by the robot, a lack of features in man-made environments and the presence of dynamics in the scene. Previous research on humanoid SLAM focused mostly on feature-based methods which result in sparse environment reconstructions. Instead, we investigate the application of a modern direct method to obtain a semi-dense visually interpretable map which can be used for collision free motion planning. We tackle the challenge of using direct visual SLAM on a humanoid by proposing a more robust pose tracking method. This is formulated as an optimisation problem over a cost function which combines information from the stereo camera and a low-drift kinematic-inertial motion prior. Extensive experimental demonstrations characterise the performance of our method using the NASA Valkyrie humanoid robot in a laboratory environment equipped with a Vicon motion capture system. Our experiments demonstrate pose tracking robustness to challenges such as sudden view change, motion blur in the image, change in illumination and tracking through sequences of featureless areas in the environment. Finally, we provide a qualitative evaluation of our stereo reconstruction against a LIDAR map.

This paper introduces an adaptive tuning method for the controllers of a 4 degrees-of-freedom autonomous underwater vehicle. The proposed scheme consists of two control loops, one for position control and an inner one for velocity control. The gains of the controller are determined on-line, according to the position/velocity errors. Using the proposed adaptive architecture, the uncertainties in the parameters of the system are addressed and the system is able to operate when hydrodynamic disturbances are present. The complexity of the fixed gain tuning procedure is also greatly decreased for underwater vehicles when the algorithm suggested here is used. Experimental results with the Nessie VII AUV show that the adaptive controller is beneficial for underwater vehicles. Finally it is shown that the current approach reduces the energy consumption of the system.

This paper proposes a general framework to control underwater vehicle-manipulator systems from external sensors. The design of the control law follows several constraints and criteria to optimize the overall behavior. The task is defined in the sensor space, that is suited for visual servoing from camera or sonar images. The control input distribution between the arm and the body is considered and separated between the approach and the intervention phases. The proposed framework is validated in a realistic simulation environment, imposing constant and varying disturbances at the velocity level.

We are interested in the coverage path planning problem with imperfect sensors, within the context of robotics for mine countermeasures. In the studied problem, an autonomous underwater vehicle (AUV) equipped with sonar surveys the bottom of the ocean searching for mines. We use a cellular decomposition to represent the ocean floor by a grid of uniform square cells. The robot scans a fixed number of cells sideways with a varying probability of detection as a function of distance and of seabed type. The goal is to plan a path that achieves the minimal required coverage in each cell while minimizing the total traveled distance and the total number of turns. We propose an off-line hybrid algorithm based on dynamic programming and on a traveling salesman problem reduction. We present experimental results and show that our algorithm's performance is superior to published results in terms of path quality and computational time, which makes it possible to implement the algorithm in an AUV.

This paper presents a method to build large-scale mosaics adapted to underwater sonar imagery. By assuming a simplified imaging model, we propose to address the registrations between images using Fourier-based methods which, unlike feature-based methods, prove well suited to handle the characteristics of forward-looking sonar images, such as low resolution, noise, occlusions and moving shadows. The registration between spatially and temporally distant images resulting from loop-closing situations or registrations in featureless areas are feasible, overcoming the main difficulties of feature-based methods. The problem is cast as a pose-based graph optimization, taking into account the uncertainties of the pairwise registrations and being able to incorporate navigation information. After the optimization, a consistent mosaic from different tracklines is generated with increased resolution and higher signal-to-noise ratio than the original images, while the vehicle motion in x,y and heading is also estimated.

We describe a novel architecture for automotive vision organized on five levels of abstraction, i.e., sensor, data, semantic, reasoning, and resource allocation levels, respectively. Although we implement and evaluate processes to detect and classify other participants within the immediate environment of a moving vehicle, our main emphasis is on the allocation of computational resource and attentive processing by the sensor suite. To that end, an efficient multiobjective resource allocation method is formalized and implemented. This includes a decision-making process dependent upon the environment, the current goal, the available sensors and computational resource, and the time available to make a decision. We evaluate our approach on road traffic test sequences acquired by a test vehicle provided by Audi. This vehicle includes lidar, video, radar, and sonar sensors, in addition to conventional global positioning system (GPS) navigation, but our evaluation is confined to lidar and video data alone.

This paper proposes a semantic world model framework for hierarchical distributed representation of knowledge in autonomous underwater systems. This framework aims to provide a more capable and holistic system, involving semantic interoperability among all involved information sources. This will enhance interoperability, independence of operation, and situation awareness of the embedded service-oriented agents for autonomous platforms. The results obtained specifically affect the mission flexibility, robustness, and autonomy. The presented framework makes use of the idea that heterogeneous real-world data of very different type must be processed by (and run through) several different layers, to be finally available in a suited format and at the right place to be accessible by high-level decision-making agents. In this sense, the presented approach shows how to abstract away from the raw real-world data step by step by means of semantic technologies. The paper concludes by demonstrating the benefits of the framework in a real scenario. A hardware fault is simulated in a REMUS 100 AUV while performing a mission. This triggers a knowledge exchange between the status monitoring agent and the adaptive mission planner embedded agent. By using the proposed framework, both services can interchange information while remaining domain independent during their interaction with the platform. The results of this paper are readily applicable to land and air robotics.

In this paper we present a new method for obtaining situation awareness via the automatic recognition of agent behaviours. In contrast to many other approaches, the presented method models different behaviour durations without using a fixed classification window, and does not require a distribution of behaviour durations. We introduce the Variable Window Layered Hidden Markov Model (VW-LHMM) as an extension of the LHMM to specifically address behaviours with irregular duration. We validate our approach by simulating three high-level behaviours within the harbour and coastline security domain. We compare performance against the LHMM and show that our approach provides a 10% improvement in classification accuracy, in addition to earlier classification.

A method for generating constant envelope (CE) waveforms to realise a given covariance matrix for a closely spaced MIMO radar system is proposed. In contrast to available algorithms, the technique provides closed form solutions for finding the required waveforms and suggests that waveforms can be chosen from finite alphabets such as binary-phase shift keying (BPSK) and quadrature-phase shift keying (QPSK). Gaussian random-variables (RV's) are mapped onto CE non-Gaussian RV's using memoryless non-linear functions. The relationship between the correlation of Gaussian RV's at the input to the nonlinear functions and non-Gaussian RV's at their output is established. Simulation results are presented to demonstrate the effectiveness of the methodology.

Autonomous Underwater Vehicles (AUVs) need positioning systems different than the Global Positioning System (GPS), which does not work in underwater scenarios. A possible solution to this lack of GPS signal are the Simultaneous Localization and Mapping (SLAM) algorithms. SLAM algorithms aim to build a map while simultaneously localize the vehicle within it. These algorithms suffer from several limitations in front of large scale scenarios. For instance, they do not perform consistent maps for large areas, mainly because uncertainties increase with the scenario. In addition, the computational cost increases with the map size. It has been demonstrated that the use of local maps reduces computational cost and improves map consistency. Following this idea, in this paper we propose a new SLAM technique based on using independent local maps, combined with a global level stochastic map. The global level contains the relative transformations between local maps. These local maps are updated once a new loop is detected and the amount of overlapping between local maps is high. Thus, maps sharing a high number of features are updated through fusion, maintaining the correlation between landmarks and vehicle. Experimental results on real data obtained from the REMUS-100 AUV show that our approach is able to obtain large map areas consistently.

This paper proposes a pose-based algorithm to solve the full Simultaneous Localization And Mapping (SLAM) problem for an Autonomous Underwater Vehicle (AUV), navigating in an unknown and possibly unstructured environment. A probabilistic scan matching technique using range scans gathered from a Mechanical Scanning Imaging Sonar (MSIS) is used together with the robot dead-reckoning displacements. The proposed method utilizes two Extended Kalman Filters (EKFs). The first, estimates the local path traveled by the robot while forming the scan as well as its uncertainty, providing position estimates for correcting the distortions that the vehicle motion produces in the acoustic images. The second is an augmented state EKF that estimates and keeps the registered scans poses. The raw data from the sensors are processed and fused in-line. No priory structural information or initial pose are considered. Also, a method of estimating the uncertainty of the scan matching estimation is provided. The algorithm has been tested on an AUV guided along a 600 m path within a marina environment, showing the viability of the proposed approach.

This paper proposes a novel approach for autonomous mission plan recovery for maintaining operability of unmanned underwater vehicles. It combines the benefits of knowledge-based ontology representation, autonomous partial ordering plan repair and robust mission execution. The approach uses the potential of ontology reasoning in order to orient the planning algorithms adapting the mission plan of the vehicle. It can handle uncertainty and action scheduling in order to maximize mission efficiency and minimise mission failures due to external unexpected factors. Its performance is presented in a set of simulated scenarios for different concepts of operations for the underwater domain. The paper concludes by showing the results of a trial demonstration carried out on a real underwater platform. The results of this paper are readily applicable to land and air robotics.

This paper presents a novel technique to align partial 3D reconstructions of the seabed acquired by a stereo camera mounted on an autonomous underwater vehicle. Vehicle localization and seabed mapping is performed simultaneously by means of an Extended Kalman Filter. Passive landmarks are detected on the images and characterized considering 2D and 3D features. Landmarks are re-observed while the robot is navigating and data association becomes easier but robust. Once the survey is completed, vehicle trajectory is smoothed by a Rauch-Tung-Striebel filter obtaining an even better alignment of the 3D views and yet a large-scale acquisition of the seabed.

In this paper we address the problem of human body pose estimation from still images. A multi-view set of images of a person sitting at a table is acquired and the pose estimated. Reliable and efficient pose estimation from still images represents an important part of more complex algorithms, such as tracking human body pose in a video sequence, where it can be used to automatically initialise the tracker on the first frame. The quality of the initialisation influences the performance of the tracker in the subsequent frames. We formulate the body pose estimation as an analysis-by-synthesis optimisation algorithm, where a generic 3D human body model is used to illustrate the pose and the silhouettes extracted from the images are used as constraints. A simple test with gradient descent optimisation run from randomly selected initial positions in the search space shows that a more powerful optimisation method is required. We investigate the suitability of the Particle Swarm Optimisation (PSO) for solving this problem and compare its performance with an equivalent algorithm using Simulated Annealing (SA). Our tests show that the PSO outperforms the SA in terms of accuracy and consistency of the results, as well as speed of convergence.

It has been shown that multiple-input multiple-output (MIMO) radar systems can improve target detection performance significantly by exploiting the spatial diversity gain. We introduce the system model in which the radar target is composed of a finite number of small scatterers and derive the formula to evaluate the theoretical probability of detection for the system having an arbitrary array-target configuration. The results can be used to predict the detection performance of the actual MIMO radar without time-consuming simulations.

This paper presents a computationally efficient approach to estimate translational 3-D motion from range images sequences, that is adapted from a 2-D motion estimation algorithm. An implementation of the algorithm is evaluated for computational efficiency as well as robustness in the presence of noise for both synthetic and real-life range data acquired with a PMD device, a high-speed low-resolution 3-D camera. 1

In this paper, a new method for the estimation of seabed elevation maps from side-scan sonar images is presented. The side-scan image formation process is represented by a Lambertian diffuse model, which is then inverted by a multiresolution optimization procedure inspired by expectation-maximization to account for the characteristics of the imaged seafloor region. On convergence of the model, approximations for seabed reflectivity, side-scan beam pattern, and seabed altitude are obtained. The performance of the system is evaluated against a real structure of known dimensions. Reconstruction results for images acquired by different sonar sensors are presented. Applications to augmented reality for the simulation of targets in sonar imagery are also discussed.

Efficient path-planning algorithms are a crucial issue for modern autonomous underwater vehicles. Classical path-planning algorithms in artificial intelligence are not designed to deal with wide continuous environments prone to currents. We present a novel Fast Marching (FM)-based approach to address the following issues. First, we develop an algorithm we call FM* to efficiently extract a 2-D continuous path from a discrete representation of the environment. Second, we take underwater currents into account thanks to an anisotropic extension of the original FM algorithm. Third, the vehicle turning radius is introduced as a constraint on the optimal path curvature for both isotropic and anisotropic media. Finally, a multiresolution method is introduced to speed up the overall path-planning process.

This paper presents a unified framework for the creation of classified maps of the seafloor from sonar imagery. Significant challenges in photometric correction, classification, navigation and registration, and image fusion are addressed. The techniques described are directly applicable to a range of remote sensing problems. Recent advances in side-scan data correction are incorporated to compensate for the sonar beam pattern and motion of the acquisition platform. The corrected images are segmented using pixel-based textural features and standard classifiers. In parallel, the navigation of the sonar device is processed using Kalman filtering techniques. A simultaneous localization and mapping framework is adopted to improve the navigation accuracy and produce georeferenced mosaics of the segmented side-scan data. These are fused within a Markovian framework and two fusion models are presented. The first uses a voting scheme regularized by an isotropic Markov random field and is applicable when the reliability of each information source is unknown. The Markov model is also used to inpaint regions where no final classification decision can be reached using pixel level fusion. The second model formally introduces the reliability of each information source into a probabilistic model. Evaluation of the two models using both synthetic images and real data from a large scale survey shows significant quantitative and qualitative improvement using the fusion approach.

This paper describes a concurrent mapping and localisation (CML) algorithm suitable for localising an autonomous underwater vehicle (AUV). The proposed CML algorithm uses a standard off-the-shelf sonar for sensing the environment. The returns from the sonar are used to detect targets in the vehicle's vicinity. These targets are used in conjunction with a vehicle model by the CML algorithm to concurrently build an absolute map of the environment and localise the vehicle in absolute coordinates. In order for the algorithm to work, the stored targets must be associated to the sonar returns at each iteration. Given the nature of sonar data, false returns complicate this process. The choice of targets and a suitable data association strategy is, therefore, vital. The chosen targets consist of returns of a significant strength. The segmentation detects these targets and calculates (a) the relative position of their center of mass with respect to the vehicle, (b) the targets' surface size, and (c) the targets' first invariant moment. This information is used by the system to perform the data association. We have chosen to adapt the well known multiple hypothesis tracking filter (MHTF) to the CML structure. This is a measurement oriented approach that finds the probability that an established target gave rise to a certain return. The paper presents results with real sonar data.