In this work, we address the problem of real-time dense depth estimation from monocular images for mobile underwater vehicles. We formulate a deep learning model that fuses sparse depth measurements from triangulated features to improve the depth predictions and solve the problem of scale ambiguity. To allow prior inputs of arbitrary sparsity, we apply a dense parameterization method. Our model extends recent state-of-the-art approaches to monocular image based depth estimation, using an efficient encoder-decoder backbone and modern lightweight transformer optimization stage to encode global context. The network is trained in a supervised fashion on the forward-looking underwater dataset, FLSea. Evaluation results on this dataset demonstrate significant improvement in depth prediction accuracy by the fusion of the sparse feature priors. In addition, without any retraining, our method achieves similar depth prediction accuracy on a downward looking dataset we collected with a diver operated camera rig, conducting a survey of a coral reef. The method achieves real-time performance, running at 24 FPS on a NVIDIA Jetson Xavier NX, 160 FPS on a NVIDIA RTX 2080 GPU and 7 FPS on a single Intel i9-9900K CPU core, making it suitable for direct deployment on embedded GPU systems. The implementation of this work is made publicly available at https://github.com/ebnerluca/uw_depth.

A known challenge for computer vision methods applied to the underwater domain is that nonlinear attenuation of light in underwater environments distorts the color signal in captured imagery, resulting in inconsistent color and contrast at varying distances to an imaged target. While surface reflectance can provide a useful cue for classifying imagery of the seafloor by object or substrate types, color inconsistency makes robust classification challenging. We introduce a method that leverages hyperspectral imagery with an underwater light formation model and structure from motion to estimate the intrinsic optical properties of the underwater environment and correct seafloor reflectance estimates from radiance measurements. We show that our method enables consistent surface reflectance estimates under both artificial and ambient lighting conditions and is readily integrated on small underwater vehicle platforms, such as a BlueROV.

Conventional underwater intervention operations using robotic vehicles require expert teleoperators and limit interaction with remote scientists. We present the SHared Autonomy for Remote Collaboration (SHARC) framework that enables novice operators to cooperatively conduct underwater sampling and manipulation tasks. With SHARC, operators can plan and complete manipulation tasks using natural language or hand gestures through a virtual reality (SHARC-VR) interface. The interface provides remote operators with a contextual 3D scene understanding that is updated according to bandwidth availability. Evaluation of the SHARC framework through controlled lab experiments demonstrates that SHARC-VR enables novice operators to complete manipulation tasks in framerate-limited conditions (i.e., 0.1–0.5 frames per second) faster than expert pilots using a conventional topside controller. For both novice and expert users, the SHARC-VR interface also increases the task completion rate and improves sampling precision. The SHARC framework is readily extensible to other hardware architectures, including terrestrial and space systems.

Autonomous underwater gliders (AUGs) are capable of traversing basin scale distances but lack sufficient localization accuracy to operate without periodic surfacing to obtain GPS fixes and constrain localization drift. Conventionally, AUGs use a dynamic flight model with depth averaged current correction (DACC) to dead-reckon their position while subsurface. However, these flight models become unstable at shallow pitch angles and DACC is inaccurate in dynamic and highly sheared water column currents. We present a method and preliminary results from field trials for improved real-time AUG localization using a Doppler Velocity Logger to estimate vehicle velocity and dynamically profile water column currents. This improved localization reduces the need for periodic surfacing, while independence from a dynamic flight model makes it particularly suited for shallow-yo profile missions.