Ocean scientists studying diverse organisms and phenomena increasingly rely on imaging devices for their research. These scientists have many tools to collect their data, but few resources for automated analysis. In this paper, we report on discussions with diverse stakeholders to identify community needs and develop a set of functional requirements for the ongoing development of ocean science-specific analysis tools. We conducted 36 in-depth interviews with individuals working in the Blue Economy space, revealing four central issues inhibiting the development of effective imaging analysis monitoring tools for marine science. We also identified twelve user archetypes that will engage with these services. Additionally, we held a workshop with 246 participants from 35 countries centered around FathomNet, a web-based open-source annotated image database for marine research. Findings from these discussions are being used to define the feature set and interface design of Ocean Vision AI, a suite of tools and services to advance observational capabilities of life in the ocean.

The ocean is a vast three-dimensional space that is poorly explored and understood, and harbors unobserved life and processes that are vital to ecosystem function. To fully interrogate the space, novel algorithms and robotic platforms are required to scale up observations. Locating animals of interest and extended visual observations in the water column are particularly challenging objectives. Towards that end, we present a novel Machine Learning-integrated Tracking (or ML-Tracking) algorithm for underwater vehicle control that builds on the class of algorithms known as tracking-by-detection. By coupling a multi-object detector (trained on in situ underwater image data), a 3D stereo tracker, and a supervisor module to oversee the mission, we show how ML-Tracking can create robust tracks needed for long duration observations, as well as enable fully automated acquisition of objects for targeted sampling. Using a remotely operated vehicle as a proxy for an autonomous underwater vehicle, we demonstrate continuous input from the ML-Tracking algorithm to the vehicle controller during a record, 5+ hr continuous observation of a midwater gelatinous animal known as a siphonophore. These efforts clearly demonstrate the potential that tracking-by-detection algorithms can have on exploration in unexplored environments and discovery of undiscovered life in our ocean.