We present a novel system which blends multiple distinct sensing modalities in audio-visual surveys to assist marine biologists in collecting datasets for understanding the ecological relationship of fish and other organisms with their habitats on and around coral reefs. Our system, designed for the CUREE AUV, uses four hydrophones to determine the bearing to biological sound sources through beamforming. These observations are merged in a Bayesian Occupancy Grid to produce a 2D map of the acoustic activity of a coral reef. Simultaneously, the AUV uses unsupervised topic modeling to identify different benthic habitats. Combining these maps allows us to determine the level of acoustic activity within each habitat. We demonstrated the system in field trials on reefs in the U.S. Virgin Islands, where it was able to autonomously discover the favored habitats of snapping shrimp (genus Alpheus).

The current approach to exploring and monitoring complex underwater ecosystems, such as coral reefs, is to conduct surveys using diver-held or static cameras, or deploying sensor buoys. These approaches often fail to capture the full variation and complexity of interactions between different reef organisms and their habitat. The CUREE platform presented in this paper provides a unique set of capabilities in the form of robot behaviors and perception algorithms to enable scientists to explore different aspects of an ecosystem. Examples of these capabilities include low-altitude visual surveys, soundscape surveys, habitat characterization, and animal following. We demonstrate these capabilities by describing two field deployments on coral reefs in the US Virgin Islands. In the first deployment, we show that CUREE can identify the preferred habitat type of snapping shrimp in a reef through a combination of a visual survey, habitat characterization, and a soundscape survey. In the second deployment, we demonstrate CUREE's ability to follow arbitrary animals by separately following a barracuda and stingray for several minutes each in midwater and benthic environments, respectively.

Abstract In-situ visual observations of marine organisms is crucial to developing behavioural understandings and their relations to their surrounding ecosystem. Typically, these observations are collected via divers, tags, and remotely-operated or human-piloted vehicles. Recently, however, autonomous underwater vehicles equipped with cameras and embedded computers with GPU capabilities are being developed for a variety of applications, and in particular, can be used to supplement these existing data collection mechanisms where human operation or tags are more difficult. Existing approaches have focused on using fully-supervised tracking methods, but labelled data for many underwater species are severely lacking. Semi-supervised trackers may offer alternative tracking solutions because they require less data than fully-supervised counterparts. However, because there are not existing realistic underwater tracking datasets, the performance of semi-supervised tracking algorithms in the marine domain is not well understood. To better evaluate their performance and utility, in this paper we provide (1) a novel dataset specific to marine animals located at http://warp.whoi.edu/vmat/ , (2) an evaluation of state-of-the-art semi-supervised algorithms in the context of underwater animal tracking, and (3) an evaluation of real-world performance through demonstrations using a semi-supervised algorithm on-board an autonomous underwater vehicle to track marine animals in the wild.

In this paper, we present an approach that enables long-term monitoring of biological activity on coral reefs by extending mission time and adaptively focusing sensing resources on high-value periods. Coral reefs are one of the most biodiverse ecosystems on the planet; yet they are also among the most imperiled: facing bleaching, ecological community collapses due to global climate change, and degradation from human activities. Our proposed method improves the ability of scientists to monitor biological activity and abundance using passive acoustic sensors. We accomplish this by extracting periodicities from the observed abundance, and using them to predict future abundance. This predictive model is then used with a Monte Carlo Tree Search planning algorithm to schedule sampling at periods of high biological activity, and power down the sensor during periods of low activity. In simulated experiments using long-term acoustic datasets collected in the US Virgin Islands, our adaptive Online Sensor Scheduling algorithm is able to double the lifetime of a sensor while simultaneously increasing the average observed acoustic activity by 21%.