Composed Video Retrieval (CVR) is designed to retrieve a target video that matches a reference video modified by a modification text. While existing methods explore cross-modal correspondences, they often assume modified objects appear directly in videos. However, modification texts frequently describe concepts not explicitly presented but implicitly expressed through semantically related visual cues (e.g., “cake” implying “birthday party”). Current approaches typically rely on aligning explicit feature representations within the concrete space, neglecting critical latent associations. To address this, we propose an adaptIve scheMa-ImAGery enhanced composItional NEtwork (IMAGINE). Unlike standard explicit matching, IMAGINE materializes implicit semantics (termed schema imagery) via dynamic multimodal prototypes. These prototypes capture shared latent concepts to adaptively modulate visual features, effectively injecting implicit guidance into the retrieval process. By bridging the gap between explicit visual contents and implicit retrieval intentions, IMAGINE achieves state-of-the-art performance in both CVR and Composed Image Retrieval (CIR) across three widely used benchmarks.

Composed Image Retrieval (CIR) constitutes a pivotal paradigm requiring models to perform joint reasoning on reference images and modification texts. However, the prevalence of Noisy Triplet Correspondence (NTC) in large-scale datasets severely constrains model performance. Existing denoising methods either target binary mismatches or rely on scalar-based point-wise estimation, neglecting rich global structural correlations among sample populations and dynamic value variations during training, thereby yielding suboptimal results. This paper identifies two critical unresolved challenges: Global Structural Inconsistency of Semantic Correlations and Hard Sample Discrimination Uncertainty. To address these, we propose RankVR, a framework designed to construct a robust CIR model via global structure consistency and dynamic value perception. Specifically, we introduce the Global Structure Consistency Perception (GSCP) module, which utilizes the Effective Rank of the Correlation Matrix to decouple clean samples from structural noise. By measuring rank difference, GSCP identifies samples disrupting macroscopic semantic symmetry. Furthermore, we develop the Adaptive Semantic Value Calibration (ASVC) module to distinguish high-value hard clean samples. By integrating training potential and reliability, it dynamically quantifies the semantic value of each triplet, ensuring effective utilization of hard samples while suppressing noise characterized by logical conflicts. Extensive experiments on the FashionIQ and CIRR benchmark datasets demonstrate that RankVR significantly outperforms existing state-of-the-art methods, validating its superior robustness in noisy environments.

Video Moment Retrieval (VMR) aims to retrieve target video moments that correspond to natural language queries. Most existing methods rely on a positive-only assumption that the queried moment always exists within the video, which limits their reliability in practical scenarios. Departing from this restrictive setting, we study Video Moment Retrieval with Relevance Feedback (VMR-RF), which requires models to both retrieve relevant moments and reject irrelevant queries. This task remains challenging due to the following issues: 1) Intrinsic Semantic Interference caused by visually similar but irrelevant moments, and 2) Propagative Decision Irreversibility induced by unidirectional relevance prediction. In light of these, we introduce AdversaRial Reliability cOrrection netWork (ARROW) for VMR-RF. ARROW adopts an active discriminative strategy through two synergetic components: a Gradient-induced Semantic Adversary (GSA) that probes model vulnerabilities by actively amplifying semantic interference, and an Adversarial Reliability Predictor (ARP) that quantifies prediction stability under such interference to effectively suppress unreliable decisions. Extensive experiments validate the effectiveness of ARROW.

Continual Text-to-Video Retrieval (CTVR) is a challenging multimodal continual learning setting, where models must incrementally learn new semantic categories while maintaining accurate text-video alignment for previously learned ones, thus making it particularly prone to catastrophic forgetting. A key challenge in CTVR is feature drift, which manifests in two forms: intra-modal feature drift caused by continual learning within each modality, and non-cooperative feature drift across modalities that leads to modality misalignment. To mitigate these issues, we propose StructAlign, a structured cross-modal alignment method for CTVR. First, StructAlign introduces a simplex Equiangular Tight Frame (ETF) geometry as a unified geometric prior to mitigate modality misalignment. Building upon this geometric prior, we design a cross-modal ETF alignment loss that aligns text and video features with category-level ETF prototypes, encouraging the learned representations to form an approximate simplex ETF geometry. In addition, to suppress intra-modal feature drift, we design a Cross-modal Relation Preserving loss, which leverages complementary modalities to preserve cross-modal similarity relations, providing stable relational supervision for feature updates. By jointly addressing non-cooperative feature drift across modalities and intra-modal feature drift, StructAlign effectively alleviates catastrophic forgetting in CTVR. Extensive experiments on benchmark datasets demonstrate that our method shows competitive advantages over state-of-the-art continual retrieval approaches.

Hybrid Approximate Nearest Neighbor Search (Hybrid ANNS) is a foundational search technology for large-scale heterogeneous data and has gained significant attention in both academia and industry. However, current approaches overlook the heterogeneity in data distribution, thus ignoring two major challenges: the Compatibility Barrier for Similarity Magnitude Heterogeneity and the Tolerance Bottleneck to Attribute Cardinality. To overcome these issues, we propose the robuSt he Terogeneity-Aware hyBrid retrievaL framEwork, STABLE, designed for accurate, efficient, and robust hybrid ANNS under datasets with various distributions. Specifically, we introduce an enhAnced heterogeneoUs semanTic perceptiOn (AUTO) metric to achieve a joint measurement of feature similarity and attribute consistency, addressing similarity magnitude heterogeneity and improving robustness to datasets with various attribute cardinalities. Thereafter, we construct our Heterogeneous Emanticre Lation graPh (HELP) index based on AUTO to organize heterogeneous semantic relations. Finally, we employ a novel Dynamic Heterogeneity Routing method to ensure an efficient search. Extensive experiments on five feature vector benchmarks with various attribute cardinalities demonstrate the superior performance of STABLE.

Video Moment Retrieval (VMR) aims to retrieve temporal moments in videos that align with natural language queries, a task requiring cross-modal reasoning between video and text. Among various supervision paradigms, point-supervised VMR has emerged as a practical solution, leveraging single-frame annotations to significantly reduce annotation costs while maintaining competitive retrieval performance. However, this sparse supervision approach induces cross-modal representation shift. This shift complicates the model’s ability to accurately capture action sequences and associate text with visual content. To tackle this, we propose a novel framework called pseuDo fRame-based tempOral and semaNtic rEfinement (DRONE) with two key modules: (1) Pseudo-Frame Temporal Alignment (PTA), which embeds textual queries as pseudo-frames to enhance temporal coherence, and (2) Curriculum-Guided Semantic Refinement (CSR), which uses a progressive contrastive learning strategy to refine semantic representations from easy to hard cases. Extensive experiments show that DRONE achieves effective retrieval performance while keeping annotation costs low.

Current knowledge distillation methods for semantic segmentation are primarily designed for knowledge transfer within homogeneous networks, and are less effective for heterogeneous networks. The feature information output by heterogeneous networks faces several challenges, including differences in feature scales and varying capabilities to represent local and global contextual information. To address these issues, we propose a novel Heterogeneous Model Knowledge Distillation (HMKD) framework using a dual alignment method to improve distillation performance between CNN-based and Transformer-based semantic segmentation models. Specifically, we introduce the Patch-based Self-attention Alignment Module (PSAM), which computes and aligns patch-level self-attention across distinct feature map spaces, enabling the transfer of local or global contextual information between heterogeneous models. Additionally, the Heterogeneous Scale Alignment Module (HSAM) is designed to ensure consistency across heterogeneous feature scales and enrich the semantic content. We have also conducted extensive experiments on two benchmark datasets (Cityscapes and CamVid) to validate the effectiveness and superiority of our approach compared with several recent state-of-the-art (SOTA) methods. Our code is deposited at https://github.com/xumingzhu989/HMKD-ICMR.

Due to their strong generalizable multimodal processing and reasoning capabilities, Multimodal Large Language Models (MLLMs) have demonstrated significant potential as universal image retrievers, effectively addressing diverse real-world image retrieval tasks.Nevertheless, pioneering studies, while promising, overlook the potential of fine-grained context modeling and disentangled fine-tuning objectives in enhancing MLLMs' retrieval performance, particularly for complex tasks such as long-text-to-image retrieval, visual dialog retrieval, and composed image retrieval (CIR).Therefore, in this work, we propose an automated fine-grained multimodal quintuple dataset construction pipeline and a novel two-stage fine-grained multimodal fine-tuning strategy.The dataset generation pipeline produces a comprehensive CIR dataset with fine-grained image captions and modification text, facilitating fine-grained context modeling.Beyond the previously entangled fine-tuning paradigm, our approach separates the fine-tuning process into two distinct stages: (1) fine-grained context reasoning-oriented fine-tuning and (2) fine-grained retrieval-oriented fine-tuning.These stages aim to sequentially enhance the model's context understanding and query-target alignment capabilities, thereby improving retrieval performance.Extensive experiments across five datasets encompassing diverse and complex image retrieval tasks demonstrate the remarkable superiority of our method over existing approaches in *Xuemeng Song (sxmustc

Video moment retrieval aims to determine the temporal boundaries of moments within a video that are most relevant to textual queries. Unlike fully-supervised and weakly-supervised methods, frame-supervised methods use a single annotated frame to model the similarities between the target moment and queries. This task is still in its infancy due to the following challenges: 1) indiscernible intra-modal information and 2) inflexible inter-modal information interaction. In light of these challenges, we introduce the Gaming fOr elAstic Localization (GOAL) method for frame-supervised video moment retrieval. It enables target moment boundary localization from a novel strategic game perspective. GOAL encompasses two core components: a game-based paradigm to find the most reliable moment and a Dynamic Updating Technique (DUT) to continuously optimize moment retrieval through dynamic gradients, thereby refining boundary predictions with different feedback. Extensive experiments on Charades-STA, ActivityNet Captions, and TACoS have validated the effectiveness of GOAL.

Multivariate time series classification (MTSC) is a significant research topic in the realm of data mining, with broad applications in different industries, including healthcare, finance, meteorology, and traffic. While existing studies have designed many classifiers based on LSTMs, CNNs, and Transformer, the sophisticated architectures raise concerns regarding efficiency in computation. Additionally, most methods concentrate on a single dimension, typically temporal patterns, without fully considering multi-dimensional information such as the independence and interactions across variables that are essential in multivariate settings. To address these challenges, this article introduces FreConvNet, a lightweight convolutional network integrated with frequency adaptive learning. Inheriting the modular design paradigm of Transformer to achieve multi-view modeling of multivariate time series. FreConvNet consists of two key components: the frequency adaptive block (FAB) and the convolutional feed-forward network (ConvFFN). The FAB leverages the Fourier Transform in conjunction with adaptive filters to capture both long-term and short-term dependencies in the temporal dimension. Following that, ConvFFN captures cross-variable and cross-feature interactions by controlling inter-channel information flow through grouped pointwise convolutions, while introducing non-linearity to enhance representational capacity. Extensive experiments conducted on the well-known UEA archive validate that FreConvNet outperforms existing convolution-based, Transformer-based, and hybrid methods in classification performance and offers a computationally efficient solution.

Traffic forecasting plays a crucial role in establishing an Intelligent Transportation System (ITS) by providing essential insights. Existing traffic forecasting relies on the assumption that there is a hidden invariant spatial-temporal pattern in the large-scale dataset. However, the traffic patterns are easily influenced by many unpredictable external factors, such as policy interventions and climate changes. Due to the dynamic nature of these exogenous factors, the traffic network's spatial-temporal patterns are also changed, thus impacting the performance of traffic forecasting models. Thus, there is an urgent need to rethink the traffic forecasting model in a fast-adaptive manner. To solve this challenge, this paper proposes an Adaptive Spatio-Temporal Context Learning framework named ASTCL, which achieves desired forecasting accuracy using daily basis traffic data collected from dozens of sensors. ASTCL constructs adaptive spatio-temporal contexts for target locations in the traffic network and generates dynamic sequence graphs based on semantic similarities. The adaptive contexts aggregate valuable information from available data, while the graphs reveal dynamic trends in traffic properties. Further, ASTCL introduces a joint convolution and attention mechanism to model intricate spatio-temporal relationships from multiple perspectives. Extensive experiments conducted on four real-world datasets demonstrate that ASTCL achieves remarkable fast adaptability and outperforms other state-of-the-art methods by a significant margin.

Composed Image Retrieval (CIR) allows users to search target images with a multimodal query, comprising a reference image and a modification text that describes the user's modification demand over the reference image. Nevertheless, due to the expensive labor cost of training data annotation, recent researchers have shifted to the challenging task of zero-shot CIR (ZS-CIR), which targets fulfilling CIR without annotated triplets. The pioneer ZS-CIR studies focus on converting the CIR task into a standard text-to-image retrieval task by pre-training a textual inversion network that can map a given image into a single pseudo-word token. Despite their significant progress, their coarse-grained textual inversion may be insufficient to capture the full content of the image accurately. To overcome this issue, in this work, we propose a novel Fine-grained Textual Inversion Network for ZS-CIR, named FTI4CIR. In particular, FTI4CIR comprises two main components: fine-grained pseudo-word token mapping and tri-wise caption-based semantic regularization. The former maps the image into a subject-oriented pseudo-word token and several attribute-oriented pseudo-word tokens to comprehensively express the image in the textual form, while the latter works on jointly aligning the fine-grained pseudo-word tokens to the real-word token embedding space based on a BLIP-generated image caption template. Extensive experiments conducted on three benchmark datasets demonstrate the superiority of our proposed method.

Localizing a desired moment within an untrimmed video via a given natural language query, i.e., cross-modal moment localization, has attracted widespread research attention recently. However, it is a challenging task because it requires not only accurately understanding intra-modal semantic information, but also explicitly capturing inter-modal semantic correlations (consistency and complementarity). Existing efforts mainly focus on intra-modal semantic understanding and inter-modal semantic alignment, while ignoring necessary semantic supplement. Consequently, we present a cross-modal semantic perception network for more effective intra-modal semantic understanding and inter-modal semantic collaboration. Concretely, we design a dual-path representation network for intra-modal semantic modeling. Meanwhile, we develop a semantic collaborative network to achieve multi-granularity semantic alignment and hierarchical semantic supplement. Thereby, effective moment localization can be achieved based on sufficient semantic collaborative learning. Extensive comparison experiments demonstrate the promising performance of our model compared with existing state-of-the-art competitors.

Unleashing the power of image-text matching in real-world applications is hampered by noisy correspondence. Manually curating high-quality datasets is expensive and time-consuming, and datasets generated using diffusion models are not adequately well-aligned. The most promising way is to collect image-text pairs from the Internet, but it will inevitably introduce noisy correspondence. To reduce the negative impact of noisy correspondence, we propose a novel model that first transforms the noisy correspondence filtering problem into a similarity distribution modeling problem by exploiting the powerful capabilities of pre-trained models. Specifically, we use the Gaussian Mixture model to model the similarity obtained by CLIP as clean distribution and noisy distribution, to filter out most of the noisy correspondence in the dataset. Afterward, we used relatively clean data to fine-tune the model. To further reduce the negative impact of unfiltered noisy correspondence, i.e., a minimal part where two distributions intersect during the fine-tuning process, we propose a distribution-sensitive dynamic margin ranking loss, further increasing the distance between the two distributions. Through continuous iteration, the noisy correspondence gradually decreases and the model performance gradually improves. Our extensive experiments demonstrate the effectiveness and robustness of our model even under high noise rates.