Confusable structure segmentation (CSS) is a type of semantic segmentation applied in remote sensing sea fog detection, medical image segmentation, camouflaged object detection, etc. Structural similarity and visual ambiguity are two critical issues in CSS that pose difficulties in distinguishing foreground objects from the background. Current methods focus primarily on enhancing visual representations and do not often incorporate multimodal information, which leads to performance bottlenecks. Inspired by recent achievements in vision-language models, we proposeVision-LanguageMutualPrompting (VLMP), a novel and unified language-guided framework that leverages text prompts to enhance CSS. Specifically, VLMP consists of vision-to-language prompting and language-to-vision prompting, which bidirectionally model the interactions between visual and linguistic features, thereby facilitating cross-modal complementary information flow. To prevent the predominance of one modality over another, we design a feature integration modulator that modulates and balances feature weights for adaptive multimodal fusion. Our framework is designed to be modular and flexible, allowing for integration with any backbone, including CNNs and transformers. We evaluate VLMP with three diverse datasets: SFDD-H8, QaTa-COV19, and CAMO-COD10K. Extensive experiments demonstrate the effectiveness and superiority of the proposed framework over those of state-of-the-art methods across these datasets. This shift from basicsightto deeperinsightin CSS through vision-language integration represents a significant advancement in the field.

Deep learning approaches for marine fog detection and forecasting have outperformed traditional methods, demonstrating significant scientific and practical importance. However, the limited availability of open-source datasets remains a major challenge. Existing datasets, often focused on a single region or satellite, restrict the ability to evaluate model performance across diverse conditions and hinder the exploration of intrinsic marine fog characteristics. To address these limitations, we introduce MFogHub, the first multi-regional and multi-satellite dataset to integrate annotated marine fog observations from 15 coastal fog-prone regions and six geostationary satellites, comprising over 68,000 high-resolution samples. By encompassing diverse regions and satellite perspectives, MFogHub facilitates rigorous evaluation of both detection and forecasting methods under varying conditions. Extensive experiments with 16 baseline models demonstrate that MFogHub can reveal generalization fluctuations due to regional and satellite discrepancy, while also serving as a valuable resource for the development of targeted and scalable fog prediction techniques. Through MFogHub, we aim to advance both the practical monitoring and scientific understanding of marine fog dynamics on a global scale. The dataset and code are at https://github.com/kaka0910/MFogHub.

The utilization of ground observation stations for land fog monitoring is constrained by station distribution, which results in incomplete and nonuniform coverage of actual conditions. This problem can be overcome by using geostationary meteorological satellite multichannel observation images and semantic segmentation to recognize fog areas. Existing methods, however, rely on a large number of annotated satellite images and fail to make full use of other auxiliary information, such as the ground observation station data. The segment anything model (SAM) proposed by Meta AI can use prompts to guide segmentation, making it possible to use data from multiple sources to undertake land fog recognition tasks. In this letter, we proposed LF-SAM, a method designed to automatically generate point prompts derived from ground observation station data and extract high-frequency features, thereby aiding in the segmentation of satellite images. We created a dedicated dataset named FY-OBS for land fog recognition, including 300 training sets and 60 test sets, and validated our method on it. Our method reduces the amount of fully annotated data required and achieves significantly better performance than prompt-free methods. The mean intersection over union (mIoU) and average accuracy (aAcc) of LF-SAM reach 73.99% and 95.24%, which were increased by 3.05% and 0.93%, respectively, compared with the method without prompts.

Seafog detection is a complex challenge in meteorology, primarily due to the extensive distribution of seafog and the limited availability of maritime observation stations. Remote sensing meteorological satellites, with their wide-area observation capabilities and abundant data resources, hold significant potential for seafog detection. However, current seafog detection approaches—whether based on traditional thresholding techniques or deep learning methods—do not fully exploit the multichannel information provided by remote sensing data, nor do they effectively capture the physical motion differences between seafog and other cloud types. Deep learning has demonstrated notable performance in seafog detection tasks, yet traditional threshold-based methods for seafog detection still hold the advantage of strong physical interpretability. Furthermore, challenges such as fuzzy seafog boundaries and overlap with clouds lead to discontinuous and incomplete detection regions, posing significant challenges for practical applications. To address these issues, we propose a deep learning approach that integrates the spectral and motion physical characteristics of seafog and various cloud types, enhancing the feature distinctions between seafog and other clouds to enable more accurate and interpretable feature extraction. To further optimize feature extraction, we propose a contrastive learning mechanism for seafog, aimed at amplifying the distinctions between seafog and other cloud types while simultaneously reducing intraclass variability within seafog across different conditions. Finally, we propose a probabilistic mask representation, which effectively mitigates the issue of regional discontinuities in detection, thereby enhancing the overall performance of the detection process. Our extensive experiments on the FY-4A satellite dataset demonstrate a CSI of 64.98%, which is 7.41% higher than the current best-performing method.

Model customization mitigates the issues of inadequate performance, resource wastage, and privacy risks associated with using general-purpose models in specialized domains and well-defined tasks. However, achieving customization at a low annotation cost still poses a challenge. Existing domain adaptation research has addressed cases where all customized classes are present in the labeled database, yet scenarios involving customer-specific classes are still unresolved. Therefore, this paper proposes a novel Class-Customized Domain Adaptation (CCDA) method, addressing the latter scenario with just one additional annotation for each customer-specific class. CCDA adopts the classic adaptation training framework and comprises two innovative techniques. Firstly, to ensure the shared class knowledge from the database and the private class knowledge from additional annotations are transferred and propagated to the correct regions within the target domain, we design the partial-feature alignment strategy, based on the mechanical properties of feature alignment. Second, we propose soft-balanced sampling to tackle the long-tail distribution problem in labeled data, preventing the model from overfitting to the labeled samples of customer-specific classes. The effectiveness of CCDA has been validated across 48 tasks simulated on domain adaptation benchmarks and two real-world customization scenarios, consistently showing excellent performance. Additionally, extensive analytical experiments illustrate the contributions of two innovative techniques. The code is available at https://github.com/CHEN-kx/ClassCustomizedDA.

Sea fog detection is a challenging and significant task in the field of remote sensing. Deep learning-based methods have shown promising potential, but require a large amount of pixel-level labeled data that are time-consuming and labor-intensive to acquire. To scale up the dataset and overcome the limitations of pixel-level annotation, we attempt to explore the existing knowledge from historical statistics for label efficient sea fog detection. In this paper, we propose an image-level Weakly Supervised Sea Fog Detection Dataset (WS-SFDD) and a novel weakly supervised sea fog detection framework via prototype learning, named ProCAM. According to the sea fog events recorded by the Marine Weather Review published quarterly by the National Meteorological Center of China, we collect the sea fog images from Himawari-8 satellite data and obtain free image-level labels to construct the dataset. However, with image-level annotations, existing weakly supervised semantic segmentation methods mainly rely on class activation maps (CAMs) and have limitations when applied to such a specific scenario: 1) the pseudo labels mainly cover the most discriminative part of object regions that are incomplete; 2) the background is complex with varying atmospheric conditions and it is difficult to distinguish sea fog from low clouds due to their high similarity in spectral characteristics; 3) the co-occurring context like ‘sea’ distracts the model and thus degrades the performance. To address the above issues, in our proposed ProCAM, we first design a prototype re-activation (PRA) module that reactivates self-similar sea fog regions by pixel-to-prototype feature matching to improve the robustness and completeness of CAMs. Then, we develop a pixel-to-prototype contrastive (PPC) learning method to increase the distance between sea fog and background in the embedding space for learning more discriminative dense features. Finally, a self-augmented regularization (SAR) strategy is presented to decouple sea fog from its co-occurring context and thus avoid background interference. Extensive experiments on the WS-SFDD dataset demonstrate our proposed method ProCAM achieves superior performance with an F1-score of 77.59% and a critical success index of 63.39%. To the best of our knowledge, this is the first work to perform image-level weakly supervised sea fog detection in remote sensing images. The dataset and code are available at https://github.com/yixianghuang/ProCAM.

Optical coherence tomography angiography (OCTA) is a new non-invasive imaging technique that can generate high-resolution volumetric blood flow information in a few seconds for retinal vessel imaging. However, retinal vessel segmentation from OCTA images is a challenging cross-dimensional task which requires efficiently acquiring spatial feature from voxels and projecting it into a plane. Considering that using 3D convolution to extract features causes exponential computational resource consumption, we simplify the design of the projection module and improve the decoding capability of the segmentation network. In this paper, we propose a lightweight Linear Projection Module (LPM) for dimensionality reduction and transformation of spatial features, and a general Multiple Attention Decoder (MADecoder) for fusing and enhancing the representation of multi-level semantic information. Our proposed cross-dimensional end-to-end segmentation network achieves SOTA results on the OCTA-500 dataset. Compared with previous method, our method reduces GPU memory usage by 55.18% and arithmetic complexity by 95.29%, dice score improved by 0.25% while reducing the number of parameters by 22.25%.

Remote sensing images serve a significant role in earth observation to tackle climate change and post-disaster reconstruction concerns. However, optical images are obscured by clouds or haze, preventing precise earth observation; hence, cloud removal has been a hot topic among concerned scholars. The objective of this article is to make cloud removal more efficient and explicable by proposing three principles: identifying clouds, guessing objects beneath the clouds, and reconstructing the cloudy area. In addition, a modified dual contrastive learning Generative Adversarial Network is proposed based on these three principles by adding cloud detection and weight sharing strategy to obtain cloud semantics. In particular, we align two datasets by forming a quaternary sample pair that includes not only optical pictures and SAR images, but also region information for a more precise reconstruction. Our experiment results on the integrated dataset reveal the superiority of proposed method over previous cloud removal methods and the effectiveness of added modules through ablation experiments, with PSNR and SSIM values of 26.2 and 0.728, respectively.

Sea fog recognition is a challenging and significant semantic segmentation task in remote sensing images. The fully supervised learning method relies on the pixel-level label, which is labor-intensive and time-consuming. Moreover, it is impossible to accurately annotate all pixels of the sea fog region due to the limited ability of the human eye to distinguish between low clouds and sea fog. In this paper, we propose a novel approach of point-based annotation for weakly supervised semantic segmentation with the auxiliary information of International Comprehensive Ocean-Atmosphere Data Set (ICOADS) visibility data. It only needs several definite points for both foreground and background, which significantly reduces the annotation cost of manpower. We conduct extensive experiments on Himawari-8 satellite remote sensing images to demonstrate the effectiveness of our annotation method. The mean intersection over union (mIoU) and overall recognition accuracy of our annotation method reach 82.72% and 95.18 %, respectively. Compared with the fully supervised learning method, the accuracy and the recognition rate of sea fog area are improved with a maximum increase of 7.69% and 9.69 %, respectively.

Sea fog detection is a challenging and essential issue in satellite remote sensing. Although conventional threshold methods and deep learning methods can achieve pixel-level classification, it is difficult to distinguish ambiguous boundaries and thin structures from the background. Considering the correlations between neighbor pixels and the affinities between superpixels, a correlation context-driven method for sea fog detection is proposed in this letter, which mainly consists of a two-stage superpixel-based fully convolutional network (SFCNet), named SFCNet. A fully connected Conditional Random Field (CRF) is utilized to model the dependencies between pixels. To alleviate the problem of high cloud occlusion, an attentive Generative Adversarial Network (GAN) is implemented for image enhancement by exploiting contextual information. Experimental results demonstrate that our proposed method achieves 91.65% mIoU and obtains more refined segmentation results, performing well in detecting fogs in small, broken bits and weak contrast thin structures, as well as detects more obscured parts.