Gesture-based communication is a standard underwater communication strategy that is taught to divers as part of their regular diver training and it would seem a natural mechanism to leverage for diver to robot communication underwater. Enabling an unmanned underwater vehicle (UUV) to understand such sequences would involve having the robot learn the large set of gestures that divers use and the way they are combined. As perfect transcription of gestures is unlikely, the communication process also requires an error-correcting framework to ensure that communication is clear and correct. Here we describe an interactive process that provides this infrastructure. A weakly supervised transfer learning approach is used to recognize standard SCUBA gestures in individual video frames and within a Sim2Real process to train a LSTM to recognize gesture sequences. This process is placed within a per-gesture and per-sequence interaction process to assist and confirm the recognition of individual gestures and to confirm entire gesture sequences. Individual aspects of this process and complete end-to-end operation are demonstrated using an unmanned underwater vehicle.

The underwater environment provides a range of interesting applications for human-robot teams. A critical issue for such teams is the development of an appropriate communication mechanism between humans and robots operating at depth. Humans operating at depth have developed an applied gesture-based communication language that can be leveraged to enable this communication, but it would be expensive and perhaps impractical to develop a hand-labelled dataset of these gestures to support a machine learning-based approach to the task. To avoid the cost of hand labelling such a large dataset, here we automate the process of collecting a labelled dataset through the use of a simple model trained on a hand-labelled dataset that only identifies salient objects (divers, their heads and hands), and then use a weakly supervised learning process to label a complex set of diver gestures. The result of this process is a system that can recognize a large number of diver hand gestures. Performance of the resulting system is compared against a hand-labelled set of diver gestures.

Current methods for human robot interaction in the underwater domain seem antiquated in comparison to their terrestrial counterparts. Visual tags and custom built wired remotes are commonplace underwater, but such approaches have numerous drawbacks. Here we describe a method for human robot interaction underwater that borrows from the long standing history of diver communication using hand signals; a three stage approach for diver-robot communication using a series of neural networks.

Robot-diver communication underwater is complicated by the attenuation of RF signals, the complexities of the environment in terms of deploying interaction devices, and issues related to the cognitive loading of human operators. Humans operating underwater have developed a simple yet effective strategy for diver-diver communication based on the visual recognition of gestures. Can a similar approach be effective for diver-robot communication? Here we present experiments with SCUBANet, an underwater detection dataset of body parts associated with diver-robot communication. Given the nature of standard diver gestures, here we concentrate on diver recognition and in particular on diver body-head-hand localization and examine the feasibility of using a CNN-based approach to address this problem. Such data-driven approaches typically require an appropriately annotated dataset. The SCUBANet dataset contains images of object classes commonly encountered during human-robot communication underwater. Object classes are labeled using per-instance bounding boxes. Annotations were created through crowd sourcing via a web-based interface to ease deployment. We provide baseline performance on diver and diver component recognition and localization using transfer learning on three widely available pre-trained models.

Simultaneous Localization and Mapping (SLAM) is a key stepping stone on the road to truly autonomous robots. SLAM is of particular importance to robots with large motion estimation problems, such as robots operating on the surface of aquatic GPS-denied environments where a paucity of local landmarks complicates SLAM and accurate navigation. Visual sensors have proven to be an effective tool for SLAM generally and have wide applicability, but is vision enough to solve SLAM in this environment, and how important are other sensors including a compass and water column depth to solve SLAM for an aquatic surface vehicle? Here we show that more sensors are almost always helpful in terms of improving SLAM performance in such a situation but that a compass is a particularly useful sensor for SLAM for autonomous surface vehicles; suggesting that a compass is a worthwhile investment for such a robot, and that compass alternatives should be considered when operating an autonomous vehicle in environments that are both GPS and compass-denied.

The development of effective user interfaces for an autonomous system can be quite difficult, especially for devices that are to be operated in the field where access to standard computer platforms may be difficult or impossible. One approach in this type of environment is to utilize tablet or phone devices, which when coupled with an appropriate tool such as ROSBridge can be used to connect with standard robot middleware. This has proven to be a successful approach for devices with mature user interface requirements but may require significant software development for experimental systems. Here we describe RCON, a software tool that allows user interfaces on iOS devices to be configured on the device itself, in real time, in response to changes in the robot software infrastructure or the needs of the operator. The system is described in detail along with the accompanying communication framework and the process of building a user interface for a simple autonomous device.

Although many commercially available robots ship with a version of ROS this is not as true for many external sensors. There is a lack of ROS support for many devices and sensors one might use to extend the capabilities of a robot. As robots are deployed in more complex environments there is the need for more specialized sensors. In particular in the aquatic domain there is the need for support for depth sounders. This paper describes the design and construction process for building a ROS node for a NMEA 0183 compliant depth and temperature transducer and a strategy for extending this design to other NMEA devices.

Virtual reality has been proposed as a training regime for a large number of tasks from surgery rehearsal (cf. [Robb et al. 1996], to combat simulation (cf. [U. S. Congress, Office of Technology Assessment 1994]) to assiting in basic design (cf. [Fa et al. 1992]). Virtual reality provides a novel and effective training medium for applications in which training "in the real world" is dangerous or expensive. Here we describe the C2SM simulator system -- a virtual reality-based training system that provides an accurate simulation of the CBRNE Crime Scene Modeller System (see [Topol et al. 2008]). The training system provides a simulation of both the underlying robotic platform and the C2SM sensor suite, and allows training of the system to take place without physically deploying the robot or the simulation of chemical and radiological agents that might be present. This paper describes the basic structure of the C2SM simulator and the software components that were used to construct it.