This work was developed under the need for an acoustic localization system to monitor marine protected areas (MPAs) with the help of autonomous underwater vehicles (AUVs). Although the use of acoustic signals for underwater localization has been previously studied, most of the solutions rely on filter-based optimization, which is prone to linearization problems in long-term applications. Instead, we implemented a Modular Acoustic Graph Simultaneous Localization and Mapping (SLAM) algorithm that, using a factor graph framework, tracks acoustic beacons with either ranges or bearings. In addition, we developed several novel methods, like a delayed-position update for ultra-short baseline (USBL) position factor integration process, an initialization algorithm for acoustic landmarks, and the creation of a new 3D bearing factor that combines two angles. After developing the algorithm, field experiments were carried out in different areas on the coast of Catalonia. Besides the localization, some monitoring tasks were also tested, such as visual mapping of localized landmarks or optical transmission of data with seafloor stations, which helped validate the accuracy of the acoustic localization system. The results of such experiments are presented and discussed.

Robot localization is a fundamental task in achieving true autonomy. Recently, many graph-based navigators have been proposed that combine an inertial measurement unit (IMU) with an exteroceptive sensor applying IMU preintegration to synchronize both sensors. IMUs are affected by biases that also have to be estimated. To increase the navigator robustness when faults appear on the perception system, IMU preintegration can be complemented with linear velocity measurements obtained from visual odometry, leg odometry, or a Doppler Velocity Log (DVL), depending on the robotic application. Moreover, higher grade IMUs are sensitive to the Earth rotation rate, which must be compensated in the preintegrated measurements. In this article, we propose a general purpose preintegration methodology formulated on a compact Lie group to set motion constraints on graph simultaneous localization and mapping problems considering the Earth rotation effect. We introduce the SE$_{N}(3)$group to jointly preintegrate IMU data and linear velocity measurements to preserve all the existing correlation within the preintegrated quantity. Field experiments using an autonomous underwater vehicle equipped with a DVL and a navigational grade IMU are provided and results are benchmarked against a commercial filter-based inertial navigation system to prove the effectiveness of our methodology.

Acoustic perception in underwater environments is challenging due to the low frequency of the acquisition system and multiple and huge sources of noise. Therefore, point clouds built by profiling sonars mounted on Autonomous Underwater Vehicles (AUV) are sparse and noisy. To solve the mapping task, AUVs need a registration algorithm to prevent maps from inconsistencies. Many scan matching algorithms are available, however, a few of them are specialized in acoustic data. In this paper, a probabilistic scan matching methodology based on Gaussian Mixtures Models (GMM) is presented and, for the first time, the Bayesian-GMM algorithm is applied in this context to model acoustic data. The scan matching problem is properly formulated using Lie groups to define pose. In addition, this methodology can return an uncertainty measure for the matching result, which is fundamental in Pose SLAM applications. This tool is implemented in a public C++library11The library repository can be found in https://bitbucket.org/gmmregistration/gmm_registration. that can process in real-time 2D and 3D scans acquired by a profiling sonar. Theoretical justification and results with real data are provided to benchmark our method against the state-of-the-art Normal Distributions Transforms (NDT) technique.

Safe autonomous navigation is an essential and challenging problem for robots operating in highly unstructured or completely unknown environments. Under these conditions, not only robotic systems must deal with limited localization information but also their maneuverability is constrained by their dynamics and often suffers from uncertainty. In order to cope with these constraints, this article proposes an uncertainty-based framework for mapping and planning feasible motions online with probabilistic safety guarantees. The proposed approach deals with the motion, probabilistic safety, and online computation constraints by: 1) incrementally mapping the surroundings to build an uncertainty-aware representation of the environment and 2) iteratively (re)planning trajectories to goal that is kinodynamically feasible and probabilistically safe through a multilayered sampling-based planner in the belief space. In-depth empirical analyses illustrate some important properties of this approach, namely: 1) the multilayered planning strategy enables rapid exploration of the high-dimensional belief space while preserving asymptotic optimality and completeness guarantees and 2) the proposed routine for probabilistic collision checking results in tighter probability bounds in comparison to other uncertainty-aware planners in the literature. Furthermore, real-world in-water experimental evaluation on a nonholonomic torpedo-shaped autonomous underwater vehicle and simulated trials in an urban environment on an unmanned aerial vehicle demonstrate the efficacy of the method and its suitability for systems with limited onboard computational power. Note to Practitioners—Emergent robotic applications require operating in previously unmapped scenarios. This article presents a unified mapping–planning strategy that enables robots to navigate autonomously and safely in harsh environments.

Underwater robots are subject to complex hydro-dynamic forces. These forces define how the vehicle moves, so it is important to consider them when planning trajectories. However, performing motion planning considering the dynamics on the robot's onboard computer is challenging due to the limited computational resources available. In this paper an efficient motion planning framework for autonomous underwater vehicles (AUVs) is presented. By introducing a loosely coupled multilayered planning design, our framework is able to generate dynamically feasible trajectories while keeping the planning time low enough for online planning. First, a fast path planner operating in a lower-dimensional projected space computes a lead path from the start to the goal configuration. Then, the lead path is used to bias the sampling of a second motion planner, which takes into account all the dynamic constraints. Furthermore, we propose a strategy for online planning that saves computational resources by generating the final trajectory only up to a finite horizon. By using the finite horizon strategy together with the multilayered approach, the sampling of the second planner focuses on regions where good quality solutions are more likely to be found, significantly reducing the planning time. To provide strong safety guarantees our framework also incorporates the conservative approximations of inevitable collision states (icss). finally, we present simulations and experiments using a real underwater robot to demonstrate the capabilities of our framework.

Achieving full autonomous robotic environment exploration in the underwater domain is very challenging, mainly due to noisy acoustic sensors, high localization error, control disturbances of the water and lack of accurate underwater maps. In this work we present a robotic exploration algorithm for underwater vehicles that does not rely on prior information about the environment. Our method has been greatly influenced by many robotic exploration, view planning and path planning algorithms. The proposed method constitutes a significant improvement over our previous work [1]: Firstly, we refine our exploration approach to improve robustness; Secondly, we propose an alternative map representation based on the quadtree data structure that allows different relevant queries to be performed efficiently, reducing the computational cost of the viewpoint generation process; Thirdly, we present an algorithm that is capable of generating consistent maps even when noisy sonar data is used. The aforementioned contributions have increased the reliability of the algorithm, allowing new real experiments performed in artificial structures but also in more challenging natural environments, from which we provide a 3D reconstruction to show that with this algorithm full optical coverage is obtained.

In real-world robotics, motion planning remains to be an open challenge. Not only robotic systems are required to move through unexplored environments, but also their manoeuvrability is constrained by their dynamics and often suffer from uncertainty. One approach to overcome this problem is to incrementally map the surroundings while, simultaneously, planning a safe and feasible path to a desired goal. This is especially critical in underwater environments, where autonomous vehicles must deal with both motion and environment uncertainties. In order to cope with these constraints, this work proposes an uncertainty-based framework for mapping and planning3 feasible motions online with probabilistic safety-guarantees. The proposed approach deals with the motion, probabilistic safety, and online computation constraints by (i) incrementally representing the environment as a collection of local maps, and (ii) iteratively (re)planning kinodynamically-feasible and probabilistically-safe paths to goal. The proposed framework is evaluated on the Sparus II, a nonholonomic torpedo-shaped AUV, by conducting simulated and real-world trials, thus proving the efficacy of the method and its suitability even for systems with limited on-board computational power.

We present a framework for planning collision-free and safe paths online for autonomous underwater vehicles (AUVs) in unknown environments. We build up on our previous work and propose an improved approach. While preserving its main modules (mapping, planning and mission handler), the framework now considers motion constraints to plan feasible paths, i.e., those that meet vehicle's motion capabilities. The new framework also incorporates a risk function to avoid navigating close to nearby obstacles, and reuses the last best known solution to eliminate time-consuming pruning routines. To evaluate this approach, we use the Sparus II AUV, a torpedo-shaped vehicle performing autonomous missions in a 2-dimensional workspace. We validate the framework's new features by solving tasks in both simulation and real-world in-water trials and comparing results with our previous approach.

We present a framework for planning collision-free paths online for autonomous underwater vehicles (AUVs) in unknown environments. It is composed of three main modules (mapping, planning and mission handler) that incrementally explore the environment while solving start-to-goal queries. We use an octree-based representation of the environment and we extend the optimal rapidly-exploring random tree (RRT*) using concepts of anytime algorithms and lazy collision evaluation, thus including the capability to replan paths according to nearby obstacles perceived during the execution of the mission. To validate our approach, we plan paths for the SPARUS-II AUV, a torpedo-shaped vehicle performing autonomous missions in a 2-dimensional workspace. We demonstrate its feasibility with the SPARUS-II AUV in both simulation and real-world in-water trials.

Autonomous manipulation of objects requires reliable information on robot-object contact state. Underwater environments can adversely affect sensing modalities such as vision, making them unreliable. In this paper we investigate underwater robot-object contact perception between an autonomous underwater vehicle and a T-bar valve using a force/torque sensor and the robot's proprioceptive information. We present an approach in which machine learning is used to learn a classifier for different contact states, namely, a contact aligned with the central axis of the valve, an edge contact and no contact. To distinguish between different contact states, the robot performs an exploratory behavior that produces distinct patterns in the force/torque sensor. The sensor output forms a multidimensional time-series. A probabilistic clustering algorithm is used to analyze the time-series. The algorithm dissects the multidimensional time-series into clusters, producing a one-dimensional sequence of symbols. The symbols are used to train a hidden Markov model, which is subsequently used to predict novel contact conditions. We show that the learned classifier can successfully distinguish the three contact states with an accuracy of 72% ± 12 %.

We investigate methods to improve fault-tolerance of Autonomous Underwater Vehicles (AUVs) to increase their reliability and persistent autonomy. We propose a learning-based approach that is able to discover new control policies to overcome thruster failures as they happen. The proposed approach is a model-based direct policy search that learns on an on-board simulated model of the AUV. The model is adapted to a new condition when a fault is detected and isolated. Since the approach generates an optimal trajectory, the learned fault-tolerant policy is able to navigate the AUV towards a specified target with minimum cost. Finally, the learned policy is executed on the real robot in a closed-loop using the state feedback of the AUV. Unlike most existing methods which rely on the redundancy of thrusters, our approach is also applicable when the AUV becomes under-actuated in the presence of a fault. To validate the feasibility and efficiency of the presented approach, we evaluate it with three learning algorithms and three policy representations with increasing complexity. The proposed method is tested on a real AUV, Girona500.

We present a novel method for planning 3D coverage paths for inspection of complex structures on the ocean floor (such as seamounts or coral reefs) using an autonomous underwater vehicle (AUV). Our method initially uses an a priori map to plan a nominal coverage path that allows the AUV to pass its sensors over all points on the target structure. We then go beyond previous approaches in the literature by considering the vehicle's position uncertainty rather than relying on the unrealistic assumption of an idealized path execution. To this aim, we present a replanning algorithm based on stochastic trajectory optimization that reshapes the nominal path to cope with the actual target structure perceived in situ. The replanning algorithm runs onboard the AUV in realtime during the inspection mission, adapting the path according to the measurements provided by the vehicle's range sensing sonars. We demonstrate the efficacy of our method in experiments at sea using the GIRONA 500 AUV where we cover a concrete block of a breakwater structure in a harbor and an underwater boulder rising from 40 m up to 27 m depth. Moreover, we apply state-of-the-art surface reconstruction techniques to the data acquired by the AUV and obtain 3D models of the inspected structures that show the benefits of our planning method for 3D mapping.

This paper describes the design and implementation of a robust position tracking control scheme for an Autonomous Underwater Vehicle (AUV). The proposed controller does not require knowledge of the vehicle's dynamic parameters and guarantees prescribed transient and steady state performance despite the presence of external disturbances acting on the vehicle. The resulting scheme is of low complexity and computational cost and thus can be easily integrated to an embedded control platform of an AUV. The proposed control scheme has analytically guaranteed stability and convergence properties, while its applicability and performance are experimentally verified using the Girona500 AUV into two different missions: a) navigation and stabilization to a specific configuration, b) meandrus-like trajectory tracking. In both cases the vehicle was under the influence of time-varying external disturbances caused by a high-pressure water jet installed on the Girona500 manipulator.

Tracking an underwater chain using an autonomous vehicle can be a first step towards more efficient solutions for cleaning and inspecting mooring chains. We propose to use a forward looking sonar as a primary perception sensor to enable the vehicle operation in limited visibility conditions and overcome the turbidity arisen during marine growth removal. Despite its advantages, working with acoustic imagery raises additional challenges to the involved image processing and control methodologies. In this paper we present a robust framework to perform chain following, combining perception, planning and control disciplines. We first introduce a detection system that exploits the sonar's high frame rate and applies local pattern matching to handle the complexity of detecting link chains in acoustic images. Then, a planning system deals with the dispersed detections and determines the link waypoints that the vehicle should reach. Finally, the vehicle is guided through these waypoints using a high level controller that has been tailored to simultaneously traverse the chain and keep track of upcoming links. Experiments on real data demonstrate the capability of autonomously follow a chain with sufficient accuracy to perform subsequent cleaning or inspection tasks.

This paper proposes a coverage path planning (CPP) method for inspection of 3D natural structures on the ocean floor charted as 2.5D bathymetric maps. This task is integral to many marine robotics applications, such as microbathymetry mapping and image photo-mosaicing. We consider an autonomous underwater vehicle (AUV) with hovering capabilities imaging the ocean floor with an orientable sensor, such as a camera or a sonar. While standard lawnmower-type surveys at constant altitude are well-suited for covering effectively planar areas, two major problems arise when tracing such paths over high-relief terrain. First, the sudden depth changes required by such paths imply very costly motions, as moving in the vertical axis is expensive for most AUVs. Second, some time is required to adjust the vehicle depth after a sudden change in the relief, resulting in a varying distance from the target surface which deteriorates the overall quality of the collected imaging data. The method proposed in this paper accounts for these facts and generates different coverage patterns according to terrain's relief, resulting in a well-suited coverage path for imaging tasks. The proposed CPP method is fast and easy to implement, and provides a valuable tool for planning coverage paths in marine environments. We tested the proposed method on a real-world bathymetric dataset of a lava tongue obtained during recent sea trials in the Santorini caldera in Greece and compares favorably to a standard lawnmower-type survey path.

This paper describes the design and implementation of a visual servo control scheme for an Autonomous Underwater Vehicle (AUV). The purpose of the control scheme is to navigate and stabilize the vehicle towards a visual target. The controller does not utilize the vehicle's dynamic model parameters and guarantees prescribed transient and steady state performance despite the presence of external disturbances representing ocean currents and waves. The proposed control scheme is of low complexity and can be easily integrated to an embedded control platform of an Autonomous Underwater Vehicle (AUV) with limited power and computational resources. Moreover, through the appropriate selection of certain performance functions, the proposed scheme guarantees that the target lies inside the onboard camera's field of view for all time. The resulting control scheme has analytically guaranteed stability and convergence properties, while its applicability and performance are experimentally verified using the Girona 500 AUV.

We present a novel survey path planning technique which minimizes the robot's position uncertainty along the planned path while taking into account area coverage performance. The proposed technique especially targets bathymetric mapping applications and respects application constraints such as the desire to survey in parallel tracks and to avoid turns in the target area to maximize sonar measurements quality. While accounting for uncertainty in the survey planning process can lead to more accurate data products, existing survey planning tools typically ignore it. Our method bridges this gap using the saliency on an a priori map to predict how the terrain will affect the robot's belief at every point on the target area. Based on this magnitude, we provide an algorithm that computes the order in which to trace parallel tracks to cover the target area minimizing the overall uncertainty along the path. A particle filter keeps track of the robot's position uncertainty during the planning process and, in order to find useful loop-closures for mapping, crossing tracks that visit salient locations are added when the uncertainty surpasses a user-provided threshold. We test our method on real-world datasets collected off the coasts of Spain, Greece and Australia. We evaluate the expected robot's position uncertainty along the planned paths and assess their associated mapping performance using a bathymetric mapping algorithm. Results show that our method offers benefits over a standard lawnmower-type path both in terms of position uncertainty and map quality.

Coverage path planning is the problem of moving an effector (e.g. a robot, a sensor) over all points in a given region. In marine robotics, a number of applications require to cover a region on the seafloor while navigating above it at a constant depth. This is the case of Autonomous Surface Vehicles, that always navigate at the water surface level, but also of several Autonomous Underwater Vehicle tasks as well. Most existing coverage algorithms sweep the free space in the target region using lawnmower-like back-and-forth motions, and the inter-lap spacing between these back-and-forth laps is determined by the robot's sensor coverage range. However, while covering the seafloor surface by navigating above it at a constant depth, the sensor's field of view varies depending on the seafloor height. Therefore, to ensure full coverage one would need to use the inter-lap spacing determined by the shallowest point on the target surface, resulting in undesired coverage overlapping among the back-and-forth laps. In this work, we propose a novel method to generate a coverage path that completely covers a surface of interest on the seafloor by navigating in a constant-depth plane above it. The proposed method uses environment information to minimize the coverage overlapping by segmenting the target surface in regions of similar depth features and addressing them as individual coverage path planning problems. A cell decomposition coverage method is applied to each region. The surface gradient is used to determine the best sweep orientation in each cell, and the inter-lap spacing in the lawnmower-like paths used to cover each cell is maximized on a lap-by-lap basis, hence obtaining a shorter, more efficient coverage path. The proposal is validated in simulation experiments conducted with a real-world bathymetric dataset that show a significant increase on path efficiency in comparison with a standard boustrophedon coverage path.

The paper proposes a method that uses topological information to guide the path search in any 2D workspace. As a first contribution, we have extended the method proposed by Jenkins to generate the topological information by taking into consideration the constraints of the workspace during the construction of the topological environment, which is then used to compute topological paths. As a second contribution, a planner based on the Rapidly-exploring Random Tree (RRT), called Homotopic RRT (HRRT) is proposed to use the topological information to guide the path search in the workspace. Finally, simulated and real results with an Autonomous Underwater Vehicle (AUV) are presented showing the feasibility of the proposal. Comparison with well-known path planning algorithms has also been included.

This paper proposes a field application of a high-level Reinforcement Learning (RL) control system for solving the action selection problem of an autonomous robot in a cable tracking task. The underwater vehicle ICTINEUAUVlearns to perform a visual based cable tracking task in a two step learning process. First, a policy is computed by means of simulation where a hydrodynamic model of the vehicle simulates the cable following task. Once the simulated results are accurate enough, in a second step, the learned-in-simulation policy is transferred to the vehicle where the learning procedure continues in a real environment, improving the initial policy. The natural actor-critic (NAC) algorithm has been selected to solve the problem in both steps. This algorithm aims to take advantage of policy gradient and value function techniques for fast convergence. Actor's policy gradient gives convergence guarantees under function approximation and partial observability while critic's value function reduces variance of the estimates update improving the convergence process.

This paper presents the design and implementation of a mission control system (MCS) for an autonomous underwater vehicle (AUV) based on Petri nets. In the proposed approach the Petri nets are used to specify as well as to execute the desired autonomous vehicle mission. The mission is easily described using an imperative programming language called mission control language (MCL) that formally describes the mission execution thread. A mission control language compiler (MCL-C) able to automatically translate the MCL into a Petri net is described and a real-time Petri net player that allows to execute the resulting Petri net onboard an AUV are also presented.

This paper proposes a field application of a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot in cable tracking task. The learning system is characterized by using a direct policy search method for learning the internal state/action mapping. Policy only algorithms may suffer from long convergence times when dealing with real robotics. In order to speed up the process, the learning phase has been carried out in a simulated environment and, in a second step, the policy has been transferred and tested successfully on a real robot. Future steps plan to continue the learning process on-line while on the real robot while performing the mentioned task. We demonstrate its feasibility with real experiments on the underwater robotICTINEUAUV.

A pioneer team of students of the University of Girona decided to design and develop an autonomous underwater vehicle (AUV) called ICTINEUAUVto face the Student Autonomous Underwater Challenge-Europe (SAUC-E). The prototype has evolved from the initial computer aided design (CAD) model to become an operative AUV in the short period of seven months. The open frame and modular design principles together with the compatibility with other robots previously developed at the lab have provided the main design philosophy. Hence, at the robot's core, two networked computers give access to a wide set of sensors and actuators. The Gentoo/Linux distribution was chosen as the onboard operating system. A software architecture based on a set of distributed objects with soft real time capabilities was developed and a hybrid control architecture including mission control, a behavioural layer and a robust map-based localization algorithm made ICTINEUAUVthe winning entry.

This paper proposes a high-level Reinforcement Learning (RL) control system for solving the action selection problem of an autonomous robot. Although the dominant approach, when using RL, has been to apply value function based algorithms, the system here detailed is characterized by the use of Direct Policy Search methods. Rather than approximating a value function, these methodologies approximate a policy using an independent function approximator with its own parameters, trying to maximize the future expected reward. The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior. In this preliminary work, we demonstrate its feasibility with simulated experiments using the underwater robot GARBI in a target reaching task.

This paper presents a complete control architecture that has been designed to fulfill predefined missions with an autonomous underwater vehicle (AUV). The control architecture has three levels of control: mission level, task level and vehicle level. The novelty of the work resides in the mission level, which is built with a Petri network that defines the sequence of tasks that are executed depending on the unpredictable situations that may occur. The task control system is composed of a set of active behaviours and a coordinator that selects the most appropriate vehicle action at each moment. The paper focuses on the design of the mission controller and its interaction with the task controller. Simulations, inspired on an industrial underwater inspection of a dam grate, show the effectiveness of the control architecture

This paper presents a vision-based localization approach for an underwater robot in a structured environment. The system is based on a coded pattern placed on the bottom of a water tank and an onboard down looking camera. Main features are, absolute and map-based localization, landmark detection and tracking, and real-time computation (12.5 Hz). The proposed system provides three-dimensional position and orientation of the vehicle along with its velocity. Accuracy of the drift-free estimates is very high, allowing them to be used as feedback measures of a velocity-based low-level controller. The paper details the localization algorithm, by showing some graphical results, and the accuracy of the system.

This paper describes the improvements achieved in our mosaicking system to assist unmanned underwater vehicle navigation. A major advance has been attained in the processing of images of the ocean floor when light absorption effects are evident. Due to the absorption of natural light, underwater vehicles often require artificial light sources attached to them to provide the adequate illumination for processing underwater images. Unfortunately, these flashlights tend to illuminate the scene in a nonuniform fashion. In this paper a technique to correct non-uniform lighting is proposed. The acquired frames are compensated through a point-by-point division of the image by an estimation of the illumination field. Then, the gray-levels of the obtained image remapped to enhance image contrast. Experiments with real images are presented.

Reinforcement learning (RL) is a very suitable technique for robot learning, as it can learn in unknown environments and in real-time computation. The main difficulties in adapting classic RL algorithms to robotic systems are the generalization problem and the correct observation of the Markovian state. This paper attempts to solve the generalization problem by proposing the semi-online neural-Q/spl I.bar/learning algorithm (SONQL). The algorithm uses the classic Q/spl I.bar/learning technique with two modifications. First, a neural network (NN) approximates the Q/spl I.bar/function allowing the use of continuous states and actions. Second, a database of the most representative learning samples accelerates and stabilizes the convergence. The term semi-online is referred to the fact that the algorithm uses the current but also past learning samples. However, the algorithm is able to learn in real-time while the robot is interacting with the environment. The paper shows simulated results with the "mountain-car" benchmark and, also, real results with an underwater robot in a target following behavior.

The purpose of this paper is to propose a Neural-Q/spl I.bar/learning approach designed for online learning of simple and reactive robot behaviors. In this approach, the Q/spl I.bar/function is generalized by a multi-layer neural network allowing the use of continuous states and actions. The algorithm uses a database of the most recent learning samples to accelerate and guarantee the convergence. Each Neural-Q/spl I.bar/learning function represents an independent, reactive and adaptive behavior which maps sensorial states to robot control actions. A group of these behaviors constitutes a reactive control scheme designed to fulfill simple missions. The paper centers on the description of the Neural-Q/spl I.bar/learning based behaviors showing their performance with an underwater robot in a target following task. Real experiments demonstrate the convergence and stability of the learning system, pointing out its suitability for online robot learning. Advantages and limitations are discussed.

This paper presents the distributed environment for virtual and/or real experiments for underwater robots (DEVRE). This environment is composed of a set of processes running on a local area network composed of three sites: 1) the onboard AUV computer; 2) a surface computer used as human-machine interface (HMI); and 3) a computer used for simulating the vehicle dynamics and representing the virtual world. The HMI can be transparently linked to the real sensors and actuators dealing with a real mission. It can also be linked with virtual sensors and virtual actuators, dealing with a virtual mission. The aim of DEVRE is to assist engineers during the software development and testing in the lab prior to real experiments.

This paper proposes a hybrid coordination method for behavior-based control architectures. The hybrid method takes advantages of the robustness and modularity in competitive approaches as well as optimized trajectories in cooperative ones. This paper shows the feasibility of applying this hybrid method with a 3D-navigation to an autonomous underwater vehicle (AUV). The behaviors are learnt online by means of reinforcement learning. A continuous Q-learning implemented with a feed-forward neural network is employed. Realistic simulations were carried out. The results obtained show the good performance of the hybrid method on behavior coordination as well as the convergence of the behaviors.

When underwater vehicles perform navigation close to the ocean floor, computer vision techniques can be applied to obtain quite accurate motion estimates. The most crucial step in the vision-based estimation of the vehicle motion consists on detecting matchings between image pairs. Here we propose the extensive use of texture analysis as a tool to ameliorate the correspondence problem in underwater images. Once a robust set of correspondences has been found, the three-dimensional motion of the vehicle can be computed with respect to the bed of the sea. Finally, motion estimates allow the construction of a map that could aid to the navigation of the robot.