High resolution underwater 3D scene reconstruction is crucial for various applications, including construction, infrastructure maintenance, monitoring, exploration, and scientific investigation. Prior work has leveraged the complementary sensing modalities of imaging sonars and optical cameras for opti-acoustic 3D scene reconstruction, demonstrating improved results over methods which rely solely on either sensor. However, while most existing approaches focus on offline reconstruction, real-time spatial awareness is essential for both autonomous and piloted underwater vehicle operations. This paper presents OASIS, an opti-acoustic fusion method that integrates data from optical images with voxel carving techniques to achieve real-time 3D reconstruction unstructured underwater workspaces. Our approach utilizes an "eye-in-hand" configuration, which leverages the dexterity of robotic manipulator arms to capture multiple workspace views across a short baseline. We validate OASIS through tank-based experiments and present qualitative and quantitative results that highlight its utility for underwater manipulation tasks.

Conventional underwater intervention operations using robotic vehicles require expert teleoperators and limit interaction with remote scientists. We present the SHared Autonomy for Remote Collaboration (SHARC) framework that enables novice operators to cooperatively conduct underwater sampling and manipulation tasks. With SHARC, operators can plan and complete manipulation tasks using natural language or hand gestures through a virtual reality (SHARC-VR) interface. The interface provides remote operators with a contextual 3D scene understanding that is updated according to bandwidth availability. Evaluation of the SHARC framework through controlled lab experiments demonstrates that SHARC-VR enables novice operators to complete manipulation tasks in framerate-limited conditions (i.e., 0.1–0.5 frames per second) faster than expert pilots using a conventional topside controller. For both novice and expert users, the SHARC-VR interface also increases the task completion rate and improves sampling precision. The SHARC framework is readily extensible to other hardware architectures, including terrestrial and space systems.

Autonomous underwater gliders (AUGs) are capable of traversing basin scale distances but lack sufficient localization accuracy to operate without periodic surfacing to obtain GPS fixes and constrain localization drift. Conventionally, AUGs use a dynamic flight model with depth averaged current correction (DACC) to dead-reckon their position while subsurface. However, these flight models become unstable at shallow pitch angles and DACC is inaccurate in dynamic and highly sheared water column currents. We present a method and preliminary results from field trials for improved real-time AUG localization using a Doppler Velocity Logger to estimate vehicle velocity and dynamically profile water column currents. This improved localization reduces the need for periodic surfacing, while independence from a dynamic flight model makes it particularly suited for shallow-yo profile missions.

Remotely Operated Vehicles (ROVs) are essential to human-operated underwater expeditions in the deep sea. However, piloting an ROV to safely interact with live ecosystems is an expensive and cognitively demanding task, requiring extensive maneuvering and situational awareness. Immersive Virtual Reality (VR) Head-Mounted Displays (HMDs) could address some of these challenges. This paper investigates how VR HMDs influence operator performance through a novel telepresence system for piloting ROVs in real-time. We present an empirical user study [N=12] that examines common midwater creature capture tasks, comparing Stereoscopic-VR, Monoscopic-VR, and Desktop teleoperation conditions. Our findings indicate that Stereoscopic-VR can outperform Monoscopic-VR and Desktop ROV capture tasks, effectively doubling the efficacy of operators. We also found significant differences in presence, task load, usability, intrinsic motivation, and cybersickness. Our research points to new opportunities towards VR with ROVs.