Da Li, Yuxiao Luo, Keping Bi, Jiafeng Guo, Wei Yuan, Biao Yang, Yan Wang, Fan Yang, Tingting Gao, Guorui Zhou. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Multimodal embedding models, rooted in multimodal large language models (MLLMs), have yielded significant performance improvements across diverse tasks such as retrieval and classification. However, most existing approaches rely heavily on large-scale contrastive learning and offer limited exploration of how the architectural and training paradigms of MLLMs affect embedding quality. While effective for generation, the causal attention and next-token prediction paradigm of MLLMs does not explicitly encourage the formation of globally compact representations, limiting their effectiveness as multimodal embedding backbones. To address this, we propose CoCoA, a Content reconstruction pre-training paradigm based on Collaborative Attention for universal multimodal representation learning. Specifically, we restructure the attention flow and introduce an EOS-based reconstruction task, encouraging the model to reconstruct input from the corresponding (EOS) embeddings. This drives the multimodal model to compress the semantic information of the input into the (EOS) token, laying the foundations for subsequent contrastive learning. Extensive experiments on MMEB-V1 demonstrate that CoCoA built upon Qwen2-VL and Qwen2.5-VL significantly improves embedding quality. Results validate that content reconstruction serves as an effective strategy to maximize the value of existing data, enabling multimodal embedding models to generate compact and informative representations, raising their performance ceiling. Our project is available at https://github.com/Trustworthy-Information-Access/CoCoA.

Open-domain table retrieval plays a vital role in accessing information from structured formats on the web, yet it remains less explored than text retrieval. Table cells primarily consist of phrases and words, which include numerous entities, such as times, locations, persons, and organizations. While emphasizing entities in text retrieval has been extensively studied, there is a significant lack of research on their applications in table retrieval. In this work, we explore how to leverage entities in tables to improve retrieval performance. We investigate the important role of entities in table retrieval from a statistical perspective and propose an Entity-Centric Alignment framework for Table retrieval (ECAT). Specifically, we use entity types to highlight entities appearing in queries and tables. Then, we propose an entity-driven late interaction paradigm based on entity representations for dense and sparse retrievers, respectively. Our proposed framework is plug-and-play and flexible, making it easy to integrate into existing table retrievers. Empirical results on table retrieval benchmarks, NQ-TABLES and OTT-QA, show that our proposed ECAT is effective in enhancing existing retrievers. Extensive analyses confirm the efficacy of ECAT's different components. Our code and dataset are available at https://github.com/Trustworthy-Information-Access/ECAT.

Table retrieval, essential for accessing information through tabular data, is less explored compared to text retrieval.The row/column structure and distinct fields of tables (including titles, headers, and cells) present unique challenges.For example, different table fields have varying matching preferences: cells may favor finer-grained (word/phrase level) matching over broader (sentence/passage level) matching due to their fragmented and detailed nature, unlike titles.This necessitates a table-specific retriever to accommodate the various matching needs of each table field.Therefore, we introduce a Table-tailored HYbrid Matching rEtriever (THYME), which approaches table retrieval from a field-aware hybrid matching perspective.Empirical results on two table retrieval benchmarks, NQ-TABLES and OTT-QA, show that THYME significantly outperforms state-of-the-art baselines.Comprehensive analyses have confirmed the differing matching preferences across table fields and validated the efficacy of THYME.

N-ary Knowledge Graphs (NKGs) are a specialized type of knowledge graph designed to efficiently represent complex real-world facts.Unlike traditional knowledge graphs, where a fact typically involves two entities, NKGs can capture n-ary facts containing more than two entities.Link prediction in NKGs aims to predict missing elements within these n-ary facts, which is essential for completing NKGs and improving the performance of downstream applications.This task has recently gained significant attention.In this paper, we present the first comprehensive survey of link prediction in NKGs, providing an overview of the field, systematically categorizing existing methods, and analyzing their performance and application scenarios.We also outline promising directions for future research.

Fine-grained ship detection in remote sensing images (RSIs) depends heavily on numerous training data with expensive manual annotations. Learning novel ship categories from very few labeled samples and without forgetting the learned knowledge of seen categories is important to real-world applications. In this letter, we formulate fine-grained ship detection in RSIs as a problem of generalized few-shot object detection (G-FSOD). Existing methods often neglect the structured information in ship taxonomy, and thus result in mutually exclusive representations between base and novel classes and hinder the transfer of the learned knowledge to the novel concepts under the few-shot settings. To handle this problem, we propose to incorporate the inherent hierarchical taxonomy in ship classes into the generalized few-shot ship detection to leverage the shared knowledge among base and novel classes. In particular, a ship detector is trained based on the coarsest class labels and a multitask classification network is built to distinguish various ships at both coarse and fine-grained levels on base classes, which leads to a generalized ship representation between base classes to novel classes. To build the classifier of novel classes, a prototype bank is constructed with the few-shot samples of novel classes, without the wreck of the feature extractor so as to maintain the performance on base classes. Extensive experiments on two large-scale ship detection datasets demonstrate the effectiveness of our method against state-of-the-art methods.

Pedestrian attribute recognition (PAR) aims at predicting the visual attributes of a pedestrian image. PAR has been used as soft biometrics for visual surveillance and IoT security. Most of the current PAR methods are developed based on discrete images. However, it is challenging for the image-based method to handle the occlusion and action-related attributes in real-world applications. Recently, video-based PAR has attracted much attention in order to exploit the temporal cues in the video sequences for better PAR. Unfortunately, existing methods usually ignore the correlations among different attributes and the relations between attributes and spatio regions. To address this problem, we propose a novel method for video-based PAR by exploring the relationships among different attributes in both the spatio and temporal domains. More specifically, a spatio-temporal saliency module (STSM) is introduced to capture the key visual patterns from the video sequences, and a module for spatio-temporal attribute relationship learning (STARL) is proposed to mine the correlations among these patterns. Meanwhile, a large-scale benchmark for video-based PAR, RAP-Video, is built by extending the image-based dataset RAP-2, which contains 83,216 tracklets with 25 scenes. To the best of our knowledge, this is the largest dataset for video-based PAR. Extensive experiments are performed on the proposed benchmark as well as on MARS Attribute and DukeMTMC-Video Attribute. The superior performance demonstrates the effectiveness of the proposed method.

Incremental pedestrian attribute recognition (IncPAR) aims to learn novel person attributes continuously and avoid the catastrophic forgetting, which is an essential problem for image forensic and security applications, e.g., suspect search. Different from the conventional continual learning for visual classification, we formulate the IncPAR as a problem of multi-label continual learning with incomplete labels (MCL-IL), where the training samples in a novel task are annotated with only a few categories of interest but may implicitly contain other attributes of previous tasks. The incomplete label assignments is a challenging and frequently-encountered issue in real-world multi-label classification applications due to a number of reasons, e.g., incomplete data collection, moderate budget for annotations, etc. To tackle the MCL-IL problem, we propose a self-training based approach via dual uncertainty-aware pseudo-labeling (DUAPL) to transfer the knowledge learned in previous tasks to novel tasks. Specially, both kinds of uncertainties, i.e., aleatoric uncertainty and epistemic uncertainty, are modeled to mitigate the negative influences of noisy pseudo labels induced by low quality samples and immature models learned by inadequate training in early tasks. Based on the DUAPL, more reliable supervision signals can be estimated to prevent the model evolution from forgetting attributes seen in previous tasks. For standard evaluations of MCL-IL methods, two benchmarks on IncPAR, termed RAP-CL and PETA-CL, are constructed by re-organizing public human attribute datasets. Extensive experiments have been performed on these benchmarks to compare the proposed method with multiple baselines. The superior performance in terms of both recognition accuracies and forgetting ratios demonstrate the effectiveness of the proposed DUAPL for IncPAR.

Generalizable person Re-Identification (ReID) aims to learn ready-to-use cross-domain representations for direct cross-data evaluation, which has attracted growing attention in the recent computer vision (CV) community. In this work, we construct a structural causal model (SCM) among identity labels, identity-specific factors (clothing/shoes color etc.), and domain-specific factors (background, viewpoints etc.). According to the causal analysis, we propose a novel Domain Invariant Representation Learning for generalizable person Re-Identification (DIR-ReID) framework. Specifically, we propose to disentangle the identity-specific and domain-specific factors into two independent feature spaces, based on which an effective backdoor adjustment approximate implementation is proposed for serving as a causal intervention towards the SCM. Extensive experiments have been conducted, showing that DIR-ReID outperforms state-of-the-art (SOTA) methods on large-scale domain generalization (DG) ReID benchmarks.

Person retrieval is a hot research topic due to its important application potential for public security. Though existing algorithms have achieved impressive progresses on current public datasets, it is still a challenging task in the real surveillance scenarios due to the various viewpoints, pose variations and occlusions. Moreover, few of the existing works study the problem of person retrieval on large-scale gallery set, where lots of distractions may deteriorate the retrieval results heavily. To have a deep understanding on the above challenges, we perform a comprehensive study on current state-of-the-art person retrieval algorithms with a large-scale benchmark in real surveillance scenarios. In the study, two kinds of techniques, i.e., attribute recognition and person re-identification, including eight algorithms, are evaluated at both algorithm level and system level. Here, the system-level evaluations investigate the effects of the combinations of the above algorithms with the module of person detection, where lots of distractions in person detection results pose a big challenge for person retrieval in real scenes. Extensive evaluations with large gallery sizes (up to 243k) and comprehensive analyses are presented in the study, which will guide researchers to develop more advanced algorithms in future.

Although existing person Re-IDentification (ReID) methods have achieved great progress with large-scale labeled data, it is still hard to generalize to unseen scenarios without any la-beled person identities. To alleviate this problem, this paper proposes a new framework to take full advantage of the label information of source domain and the data distribution geometry of unlabeled target domain to improve the ReID performance in the unlabeled target domain. Instead of direct model transfer, the data transfer is first adopted where the identity preserving samples are generated from the labeled source domain to unlabeled target domain. Accordingly, a better initialized target domain adapted ReID model could be obtained with the generated samples. The fine-grained part-level features are then learned instead of global features to better mine new persons in the unlabeled target domain. Finally, the proposed framework iteratively updates the ReID model with the generated persons and the mined persons in last iteration, and explores new persons from the unlabeled target domain. The state-of-the-art experimental results are achieved on Market1501 and DukeMTMC-reID in terms of unsupervised cross-domain person ReID.

Intelligent video surveillance (IVS) is always an interesting research topic to utilize visual analysis algorithms for exploring richly structured information from big surveillance data. However, existing IVS systems either struggle to utilize computing resources adequately to improve the efficiency of large-scale video analysis, or present a customized system for specific video analytic functions. It still lacks of a comprehensive computing architecture to enhance efficiency, extensibility and flexibility of IVS system. Moreover, it is also an open problem to study the effect of the combinations of multiple vision modules on the final performance of end applications of IVS system. Motivated by these challenges, we develop an Intelligent Scene Exploration and Evaluation (ISEE) platform based on a heterogeneous CPU-GPU cluster and some distributed computing tools, where Spark Streaming serves as the computing engine for efficient large-scale video processing and Kafka is adopted as a middle-ware message center to decouple different analysis modules flexibly. To validate the efficiency of the ISEE and study the evaluation problem on composable systems, we instantiate the ISEE for an end application on person retrieval with three visual analysis modules, including pedestrian detection with tracking, attribute recognition and re-identification. Extensive experiments are performed on a large-scale surveillance video dataset involving 25 camera scenes, totally 587 hours 720p synchronous videos, where a two-stage question-answering procedure is proposed to measure the performance of execution pipelines composed of multiple visual analysis algorithms based on millions of attribute-based and relationship-based queries. The case study of system-level evaluations may inspire researchers to improve visual analysis algorithms and combining strategies from the view of a scalable and composable system in the future.