Coral reefs play a crucial role in marine ecosystems, offering a nutrient-rich environment and safe shelter for numerous marine species. Automated coral image recognition aids in monitoring ocean health at a scale without experts' manual effort. Recently, large vision-language models like CLIP have greatly enhanced zero-shot and low-shot classification capabilities for various visual tasks. However, these models struggle with fine-grained coral-related tasks due to a lack of specific knowledge. To bridge this gap, we compile a fine-grained coral image dataset consisting of 16,659 images with taxonomy labels (from Kingdom to Species), accompanied by morphology-specific text descriptions for each species. Based on the dataset, we propose CORAL-Adapter, integrating two complementary kinds of coral-specific knowledge (biological taxonomy and coral morphology) with general knowledge learned by CLIP. CORAL-Adapter is a simple yet powerful extension of CLIP with only a few parameter updates and can be used as a plug-and-play module with various CLIP-based methods. We show improvements in accuracy across diverse coral recognition tasks, e.g., recognizing corals unseen during training that are prone to bleaching or originate from different oceans.

Coral reefs are vital yet vulnerable ecosystems that require continuous monitoring to support conservation. While coral reef images provide essential information in coral monitoring, interpreting such images remains challenging due to the need for domain expertise. Visual Question Answering (VQA), powered by Large Vision-Language Models (LVLMs), has great potential in user-friendly interaction with coral reef images. However, applying VQA to coral imagery demands a dedicated dataset that addresses two key challenges: domain-specific annotations and multidimensional questions. In this work, we introduce CoralVQA, the first large-scale VQA dataset for coral reef analysis. It contains 12,805 real-world coral images from 67 coral genera collected from 3 oceans, along with 277,653 question-answer pairs that comprehensively assess ecological and health-related conditions. To construct this dataset, we develop a semi-automatic data construction pipeline in collaboration with marine biologists to ensure both scalability and professional-grade data quality. CoralVQA presents novel challenges and provides a comprehensive benchmark for studying vision-language reasoning in the context of coral reef images. By evaluating several state-of-the-art LVLMs, we reveal key limitations and opportunities. These insights form a foundation for future LVLM development, with a particular emphasis on supporting coral conservation efforts.

Although coral reefs are special and vital marine ecosystems, massive coral degradation began to occur due to the increase in global temperatures and the intensification of human industrial activities. Coral reef protection requires accurate coral recognition because it is the foundation for learning the distribution, disease, and growth of coral reefs, hereby informing the proper ways for further action. Recently, CNNs have been applied in automated coral image classification. These classifier models, however, are difficult to be generalized from the trained coral images in a marine region (source domain) to the coral images in a different marine region (target domain) since the corals have significant within-species morphological variability among the different geographic location domains. In this paper, a novel coral recognition algorithm is introduced via knowledge transfer across domains and its advantages lie in the following aspects. (1) It simultaneously transfers corals’ texture and structure features across domains thus providing useful knowledge to assist the coral recognition tasks in the target marine domain. (2) To overcome the difficulty that the confusing coral images (e.g., bleached corals) are prone to be misclassified and transfer useless or even negative information, our algorithm is equipped with the reject option for the confusing corals while adapting. These corals can be sent to an expert or a more expensive but accurate system, resulting in strengthened transferability and reliability. Furthermore, we develop a new cross-domain coral image dataset to enhance coral research. Without the label information from the target marine region, our method significantly reduces the distribution gap and domain shift among the different marine regions. In addition, CR-Cross goes a step further in tackling the challenges of missing coral data, maximizing the utilization of available coral datasets, and enhancing the reusability of both coral data and coral recognition models. A series of empirical studies show that our method remarkably outperforms a broad range of baselines.

Graph embedding aims at learning vertex representations in a low-dimensional space by distilling information from a complex-structured graph. Recent efforts in graph embedding have been devoted to generalizing the representations from the trained graph in a source domain to the new graph in a different target domain based on information transfer. However, when the graphs are contaminated by unpredictable and complex noise in practice, this transfer problem is quite challenging because of the need to extract helpful knowledge from the source graph and to reliably transfer knowledge to the target graph. This paper puts forward a two-step correntropy-induced Wasserstein GCN (graph convolutional network, or CW-GCN for short) architecture to facilitate the robustness in cross-graph embedding. In the first step, CW-GCN originally investigates correntropy-induced loss in GCN, which places bounded and smooth losses on the noisy nodes with incorrect edges or attributes. Consequently, helpful information are extracted only from clean nodes in the source graph. In the second step, a novel Wasserstein distance is introduced to measure the difference in marginal distributions between graphs, avoiding the negative influence of noise. Afterwards, CW-GCN maps the target graph to the same embedding space as the source graph by minimizing the Wasserstein distance, and thus the knowledge preserved in the first step is expected to be reliably transferred to assist the target graph analysis tasks. Extensive experiments demonstrate the significant superiority of CW-GCN over state-of-the-art methods in different noisy environments.