Li Zheng, Xin Zhang, Shuyi He, Fei Li, Chong Teng, Jiang-Ming Yang, Donghong Ji, Zhuang Li. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Weakly Supervised temporal Article Grounding (WSAG) is an important task in the field of video understanding and retrieval, which aims to ground sentences from the article with the corresponding video clips (segments), using only video-text pairwise annotations during training. Existing methods typically rely on a proposal-based approach that utilizes contrastive learning to score candidate video clips. However, these approaches often face two main drawbacks: (1) They mainly focus on global video-level alignment with the article while neglecting the multi-level relationships between video clips and text hierarchy; (2) They overlook intra-modal feature learning, failing to adequately distinguish salient patterns. To address these issues, we propose a novel multi-granularity game-theoretic method, namely Hierarchical Cooperative Network (HCN), which models video clips and article units (i.e., at word, sentence, and paragraph layers) as players in a game. Specifically, our proposed method introduces two mechanisms: (1) Intra-modal feature cooperation: Within each modality, we encourage cooperation among the features to obtain a salient feature set as the effective representations, enabling a sharper distinction between semantically overlapping video clips and enhancing discriminative textual cues; (2) Layered cross-modal cooperation: For different semantic layers of the article, we employ the Shapley interaction index to quantify and promote synergistic effects to further enhance cross-modal matching. By decomposing multi-modal interactions into intra-modal and cross-modal cooperations with multi-layer semantics, HCN achieves fine-grained alignment and strengthens salient feature representations for the WSAG task. Experimental results demonstrate that HCN achieves the state-of-the-art performances on representative WSAG benchmarks, significantly outperforming existing weakly supervised methods and competitive video large language models.

The rail surface defects directly impact the safety and efficiency of heavy-haul train operations. Timely assessment of these defects is crucial for informed maintenance decisions, with precise defect detection at its core. In recent years, the accumulation of extensive rail inspection images has led to the application of numerous computer vision-based methods for pixel-level detection of rail surface defects. However, given the constraint of a limited number of labeled defect samples, ensuring the generalization and robustness of existing methods remains challenging, particularly across varying track conditions and complex heavy-haul scenarios. Thus, this paper introduces a Segment-Anything-Model (SAM)-enhanced method for the detection of rail surface defects. First, a shadow-detection-based algorithm is developed to extract the rail regions and mitigate background interference. Then a student-teacher-Simi-network (S-T-Simi)-based unsupervised method is designed to generate prompt information for SAM. Utilizing this prompt information, we develop a task-specified SAM for precise rail defect detection. Finally, comprehensive validation is performed using inspection data collected from diverse heavy-haul tracks. Experimental results indicate that the proposed method achieves highly accurate segmentation of rail defects.

Cloud detection plays a significant role in remote sensing image applications. Existing deep learning-based cloud detection methods rely on massive precise pixel-wise annotations, which are time-consuming and expensive. To alleviate this problem, we propose a weakly supervised cloud detection framework that leverages physical rules to generate weak supervision for cloud detection in remote sensing images. Specifically, a rule-based adaptive pseudo labeling (RAPL) algorithm is devised to adaptively annotate potential cloud pixels based on cloud spectral properties without manual intervention. Unlike existing physical annotations using fixed thresholds, RAPL employs the bidirectional threshold segmentation and adaptive gating mechanism to annotate cloud and boundary masks with more explicit semantic categories and spatial structures separately. Subsequently, these pseudo masks are treated as weak supervision to optimize the heuristic cloud detection network for pixel-wise segmentation. Considering that clouds appear as complex geometric structures and nonuniform spectral reflectance, a deformable boundary refining module is designed to enhance the modeling ability of spatial transformation and activate sharp boundaries from translucent cloud regions. Moreover, a harmonic loss is employed to recognize clouds with nonuniform spectral reflectance and suppress the interference of bright backgrounds. Extensive experiments on the GF-1, L8 Biome, and WDCD datasets demonstrate that the proposed method achieves state-of-the-art results. A public reference implementation of this work in PyTorch is available at https://github.com/NiAn-creator/HeuristicCloudDetection.

Semantic segmentation of remote sensing imageries plays a crucial role in resource exploration, urban planning, weather forecasting, etc. For this task, deep learning-based methods have shown significant achievement, typically trained with large-scale labeled data. However, these methods often suffer the performance deterioration facing limited labeled data in real-world applications. To address this problem, a novel self-supervised semantic segmentation framework is proposed for remote sensing imageries with limited labeled data. Specifically, image inpainting is acted as pixel-level pretext task for learning dense feature representations suitable for semantic segmentation. Further, rather than trivially leveraging the conventional random inpainting strategy, a novel adversarial training scheme is proposed to drive the pretext task to adaptively mask and restore salient local regions. The adversarial training scheme consists of instructor network and inpainting network, the instructor network increasingly predicts meaningful salient regions as erased regions, and meanwhile the inpainting network seeks for restoring the corrupted image as pretext task to learn its intrinsic representation. Moreover, the structural similarity (SSIM) is applied as a patch-level loss function for semantic segmentation considering that remote sensing images are highly structured. The experimental results on the ISPRS Potsdam dataset demonstrate that our method outperforms state-of-the-art self-supervised methods and the ImageNet pre-training methods. The source code is available at https://github.com/JasmineBJTU/self-supervised_RSSS.