Bundle recommender system is divided into two main stages: bundle editing and bundle recommendation. While substantial research progress has been made in each stage, in practical application scenarios, bundle compositions and the final recommended bundles mutually influence each other: the continuously editing bundle compositions affect the recommendation results, while user feedback on recommended bundles in turn guides the refinement of bundle compositions. This paper presents the first comprehensive reproducibility study of the complete bundle recommendation pipeline. We implement eight bundle-level editing methods, nine item-level editing methods, and seven state-of-the-art bundle recommendation models, and evaluate their performance across six real-world datasets. Our empirical analysis reveals several key findings. First, bundle-level editing faces the challenge of generating high-quality bundles. Second, in the item-level editing, the replacement operation emerges as a universal bottleneck across all methods. Third, in the recommendation stage, recommendation models exhibit varying performance across different interaction density scenarios (e.g., cold-start). Finally, bundle recommendation suffers degraded performance when integrating item-level editing and bundle recommendation within a unified pipeline. Overall, there is the systemic limitation of bundle recommendation: prior work has focused on optimizing individual stages independently, disregarding the interdependencies throughout the entire recommendation system. These findings highlight the urgent need to develop end-to-end solutions that can holistically address the bundle editing and recommendation workflow. Our repository is now available for public access via https://github.com/anyr123/Bundle_Edit_Rec_SIGIR26.

Composed Image Retrieval (CIR) enables image retrieval by combining multiple query modalities, but existing benchmarks predominantly focus on general-domain imagery and rely on reference images with short textual modifications. As a result, they provide limited support for retrieval scenarios that require fine-grained semantic reasoning, structured visual understanding, and domain-specific knowledge. In this work, we introduce CIRThan, a sketch+text composed image retrieval dataset for Thangka imagery, a culturally grounded and knowledge-specific visual domain characterized by complex structures, dense symbolic elements, and domain-dependent semantic conventions. CIRThan contains 2,287 high-quality Thangka images, each paired with a human-drawn sketch and hierarchical textual descriptions at three semantic levels, enabling composed queries that jointly express structural intent and multi-level semantic specification. We provide standardized data splits, comprehensive dataset analysis, and benchmark evaluations of representative supervised and zero-shot CIR methods. Experimental results reveal that existing CIR approaches, largely developed for general-domain imagery, struggle to effectively align sketch-based abstractions and hierarchical textual semantics with fine-grained Thangka images, particularly without in-domain supervision. We believe CIRThan offers a valuable benchmark for advancing sketch+text CIR, hierarchical semantic modeling, and multimodal retrieval in cultural heritage and other knowledge-specific visual domains. The dataset is publicly available at https://github.com/jinyuxu-whut/CIRThan.

Composed Image Retrieval (CIR) aims to retrieve a target image from a query composed of a reference image and modification text. Recent training-free zero-shot methods often employ Multimodal Large Language Models (MLLMs) with Chain-of-Thought (CoT) to compose a target image description for retrieval. However, due to the fuzzy matching nature of ZS-CIR, the generated description is prone to semantic bias relative to the target image. We propose SDR-CIR, a training-free Semantic Debias Ranking method based on CoT reasoning. First, Selective CoT guides the MLLM to extract visual content relevant to the modification text during image understanding, thereby reducing visual noise at the source. We then introduce a Semantic Debias Ranking with two steps, Anchor and Debias, to mitigate semantic bias. In the Anchor step, we fuse reference image features with target description features to reinforce useful semantics and supplement omitted cues. In the Debias step, we explicitly model the visual semantic contribution of the reference image to the description and incorporate it into the similarity score as a penalty term. By supplementing omitted cues while suppressing redundancy, SDR-CIR mitigates semantic bias and improves retrieval performance. Experiments on three standard CIR benchmarks show that SDR-CIR achieves state-of-the-art results among one-stage methods while maintaining high efficiency. The code is publicly available at https://github.com/suny105/SDR-CIR.

ABSTRACT Detecting ritual implements in Thangka paintings—such as swords and scriptures—remains challenging due to their intricate visual composition and symbolic complexity. Existing object detection models, typically trained on natural scenes, tend to perform poorly in this domain. To address this limitation, we summarize the principles of composition in Thangka and identify key spatial and co‐occurrence priors specific to ritual implements. Based on these insights, we propose GPCDet: a guided by principles of composition detector that integrates domain‐specific priors into the detection process. Specifically, we introduce a spatial coordinate attention module to emphasize critical spatial regions where implements frequently appear. In addition, we design a graph convolution network‐auxiliary detection module to model inter‐category co‐occurrence, thereby enhancing feature representation and improving classification performance. Experiments on the newly curated ritual implements in Thangka (RITK) dataset show that GPCDet achieves substantial improvements over existing methods, establishing a new state‐of‐the‐art baseline for this challenging task.

Zero-shot referring image segmentation aims to locate and segment the target region based on a referring expression, with the primary challenge of aligning and matching semantics across visual and textual modalities without training. Previous works address this challenge by utilizing Vision-Language Models and mask proposal networks for region-text matching. However, this paradigm may lead to incorrect target localization due to the inherent ambiguity and diversity of free-form referring expressions. To alleviate this issue, we present LGD (Leveraging Generative Descriptions), a framework that utilizes the advanced language generation capabilities of Multi-Modal Large Language Models to enhance region-text matching performance in Vision-Language Models. Specifically, we first design two kinds of prompts, the attribute prompt and the surrounding prompt, to guide the Multi-Modal Large Language Models in generating descriptions related to the crucial attributes of the referent object and the details of surrounding objects, referred to as attribute description and surrounding description, respectively. Secondly, three visual-text matching scores are introduced to evaluate the similarity between instance-level visual features and textual features, which determines the mask most associated with the referring expression. The proposed method achieves new state-of-the-art performance on three public datasets RefCOCO, RefCOCO+ and RefCOCOg, with maximum improvements of 9.97 % in oIoU and 11.29 % in mIoU compared to previous methods

Fine-grained sketch-based image retrieval aims to retrieve matched natural photos at instance level based on hand-drawn sketches. Understanding sketches and aligning them with photos in the joint embedding space are the primary challenges. Previous works have attempted to improve the understanding and alignment of fine-grained semantics by leveraging a selected subset of local features based on the local matching strategy. However, these approaches rely heavily on local features to capture regional visual details, which introduces unavoidable uncertainty and noise in the matching process. In addition, existing works lack the ability to effectively separate unmatched pairs with similar overall semantics in the common space, which may disrupt fine-grained semantic alignment. In this paper, we presentCIGSPA: aContextual Information Guided Sketch-Photo Alignmentframework that leverages latent contextual information to enhance sketch understanding and sketch-photo matching without additional knowledge and prior supervision. Specifically, we first design anAdaptive Local Feature Filtermodule to refine feature similarity based on the context of local features and select more appropriate features, which ensure the semantic fidelity and spatial compactness of salient features. Secondly, we design aContextual Weighted Contrastiveloss to guide global sketch-photo matching by leveraging discriminative contextual information from photos, and hence aligns the sketch and photo at the instance level. Our comprehensive experiments on six public datasets demonstrate the effectiveness of our proposed CIGSPA. The code is available athttps://github.com/jinyuxu-whut/CIGSPA4FG-SBIR.

Proxy hashing methods have attracted increasing attention in cross-modal retrieval, because they are able to learn the mapping of different modalities into a common low-dimensional hash space by leveraging global proxies. However, existing approaches typically suffer from two limitations: (1) They solely utilize prior labels to capture the global semantic information, and hence lack the ability to explore necessary fine-grained semantic information to bridge the modality gap effectively. (2) They seldom consider the guidance information of intrinsic semantic similarity on proxy-centered space, and thus fail to leverage the similarity relations among instances sufficiently. To mitigate these limitations, we propose RAPH, a novel Relation-Aware Proxy Hashing framework that learns the semantic relations between different modalities and different semantic levels to enhance the discriminative capability of hash codes. Specifically, we first propose a Local Semantic Interaction (LSI) module based on masked language modeling to achieve the interaction of multi-modal fine-grained semantic features. Second, a relation-aware hashing learning scheme is designed to simultaneously explore the intrinsic semantic relationships and the global semantic information based on proxies. This is achieved by minimizing the reconstruction error between the multi-modal affinity matrices derived from learned features and the cross-modal similarity matrix of the hash codes. The proposed framework is able to learn more discriminative hash codes and achieves superior performance to many baselines on three public datasets.

As a key problem in the field of recommender systems, Click-Through Rate (CTR) prediction has garnered significant attention due to its pivotal role in industrial applications. In recent years, numerous CTR prediction models have emerged, mainly focusing on feature interactions and user interest modeling. However, existing approaches are one-sided and tend to ignore the respective effects of users’ relatively stable discrete preferences and continuous dynamic interests. In addition, current models usually overlook the decision-making process behind users’ clicks, making the predicted results difficult to interpret. To address these issues, this paper introduces the Decision Evaluation Network driven by User Preferences and Dynamic Interests (UPDI-DEN), which innovatively reframes the CTR prediction task as a problem of evaluating user click decisions. The proposed model exhibits an advanced capability to distinctly capture the discrete preferences and dynamic interests embedded within users’ history sequences, and explicitly model their respective impacts on the decision-making process governing final click behavior. Experimental results demonstrate the effectiveness and strong competitiveness of the proposed method on three datasets.

Recent research has revealed that object detectors are highly susceptible to backdoor attacks, which can introduce detection errors during inference, such as detecting non-existent objects or failing to detect existing objects. Though several backdoor attacks targeting object detection have been proposed to achieve high attack success rates, these methods often involve visible triggers, which can be detected by human inspection or backdoor defenses. To enhance the attack stealthiness, we introduce a stealthy backdoor attack for object detection. Specifically, it employs a uniform shift on each pixel within images as the trigger. The particle swarm optimization is utilized to effectively find the optimal uniform shift to accomplish different attack targets in object detection, including object disappearance, object generation, and object misclassification. To achieve these targets and preserve stealthy, we design corresponding objective functions to maintain a balance between attack stealthiness and attack effectiveness. We have conducted comprehensive experiments to demonstrate the effectiveness of our proposed attack across the three attack targets in object detection, as well as its robustness against existing defense methods.

Digital watermarking has become a critical technology for copyright protection of digital images. However, the removal of such watermarks to restore original image content presents significant challenges. Current deep learning methods, which rely on GAN or multi-stage encoder-decoder networks, struggle to restore the clean and apparent texture of background images accurately, leading to visual artifacts and degradation in color and texture. Additionally, these methods require precise prediction of watermark positions, limiting their flexibility in handling inaccurate mask predictions.In this paper, we introduce a novel framework, Diffusion Watermark Removal (DiffWR), which employs a diffusion-based approach to address the limitations of existing techniques. DiffWR utilizes Denoising Diffusion Probabilistic Models (DDPM) to generate high-quality, artifact-free restored images. It incorporates an Initial Prediction Module (IPM) to provide preliminary image predictions and guide the diffusion process within accurate boundaries using a binary mask. This approach allows for adaptive inpainting beyond the initially predicted regions, enhancing the flexibility of the restoration process.Our model demonstrates superior performance in terms of visual quality and structural similarity without introducing artifacts. It outperforms state-of-the-art methods in perceptual metrics such as FID and LPIPS, as well as in traditional metrics like SSIM and PSNR. The results showcase the capability of DiffWR to effectively remove watermarks while preserving the original image’s visual fidelity and texture.

Thangka art, part of the UNESCO Intangible Cultural Heritage of Humanity, is visually characterized by complex junctions and intricate corners, demand high-fidelity vectorization to preserve its structural integrity and smooth curvilinear aesthetics. Conventional line art vectorization algorithms applied to Thangka element line art face challenges: (1) hard to fit complex junctions that leads to spurious spikes and discontinuous strokes; and (2) unnatural distortions in long curves due to insufficient smoothness constraints. To address these challenges, we propose a skeleton-guided vectorization framework to optimize the topology of vectorized Thangka element line art, and a multilayer perceptual loss as a smoothness regulation to improve curve continuity. Experimental results on manually annotated Thangka element line art dataset demonstrate that our method surpasses state-of-the-art approaches in preserving topological integrity and achieving visual smoothness, offering a robust foundation for digitizing cultural heritage artworks with complex topologies and similar aesthetic requirements.

Text-to-Image person re-identification (TIReID) aims to retrieve pedestrian images from a gallery based on textual descriptions, thus bridging vision and language modalities for practical retrieval scenarios. Despite recent advances leveraging various cross-modal alignment strategies, existing methods typically assume all image-text pairs in training datasets are correctly matched, overlooking the pervasive Noisy Correspondence (NC) problem—erroneous image-text associations that degrade model robustness. Prior approaches either lack noise identification mechanisms or rely on direct filtering of detected noisy samples, which only partially mitigates the adverse effects of noise and cannot fully prevent overfitting to incorrect correspondences during training. Addressing this challenge, we propose Robust Dual Embedding Contrastive Learning (RDECL), which consists of two main components: 1) A Dual-View Cumulative Trust Division (DCTD) progressively constructs a high-confidence clean sample repository via adaptive sample selection, ensuring reliable image-text correspondence learning under uncertain noise detection.2) A Robust Generalized Contrastive Loss (RGCL) further enhances robustness by leveraging all negative samples and maximizing the loss distribution discrepancy between clean and noisy samples, thereby suppressing overfitting to noisy labels. We conduct extensive experiments on three public benchmark datasets, namely CUHK-PEDES, ICFG-PEDES, and RSTPReID, to evaluate the performance and robustness of our RDECL.

Shaopeng Tang, Lin Li, Xiaohui Tao, Leqi Zhong, Qing Xie. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.

Cross-domain recommendation (CDR) aims to alleviate the data sparsity by transferring knowledge across domains.Disentangled representation learning provides an effective solution to model complex user preferences by separating intra-domain features (domainshared and domain-specific features), thereby enhancing robustness and interpretability.However, disentanglement-based CDR methods employing generative modeling or GNNs with contrastive objectives face two key challenges: (i) pre-separation strategies decouple features before extracting collaborative signals, disrupting intra-domain interactions and introducing noise; (ii) unsupervised disentanglement objectives lack explicit task-specific guidance, resulting in limited consistency and suboptimal alignment.To address these challenges, we propose DGCDR, a GNN-enhanced encoder-decoder framework.To handle challenge (i), DGCDR first applies GNN to extract high-order collaborative signals, providing enriched representations as a robust foundation for disentanglement.The encoder then dynamically disentangles features into domain-shared and -specific spaces, preserving collaborative information during the separation process.To handle challenge (ii), the

AI-based video generation is rapidly advancing field with significant research focus and broad applications across industries like entertainment, education, and healthcare. To gain a structured understanding of the field, this paper systematically reviews the research dynamics and application trends in video generation from 2020 to April 2025. Utilizing a combined approach of content analysis and bibliometric analysis on 422 research publications selected from the Web of Science Core Collection database, we investigate key developments. The content analysis examines state-of-the-art algorithmic innovations, generation strategies, and prominent application domains. Bibliometric analysis, employing tools like CiteSpace and VOSviewer, maps publication trends, identifies influential authors, institutions, and sources, visualizes collaboration networks, and analyzes the evolution of research hotspots through keywords. Our findings the rise of specific architectures, pinpoint high-activity application areas, reveal evolving thematic clusters, and also remarks persistent challenges, including technical hurdles and critical ethical issues such as deepfake detection, algorithmic transparency, and data privacy. By synthesizing these analyses, this study offers a structured overview of the recent landscape, and highlights areas that may warrant further exploration in AI video generation.

Panoptic Scene Graph Generation (PSG) aims to predict a variety of relations between pairs of objects within an image, and indicate the objects by panoptic segmentation masks instead of bounding boxes . Existing PSG methods attempt to straightforwardly fuse the object tokens for relation prediction, thus failing to fully utilize the interaction between the pairwise objects. To address this problem, we propose a novel framework named Group R elation F ormer (GroupRF) to capture the fine-grained inter-dependency among all instances. Our method introduce a set of learnable tokens termed group rln tokens, which exploit fine-grained contextual interaction between object tokens with multiple attentive relations. In the process of relation prediction, we adopt multiple triplets to take advantage of the fine-grained interaction included in group rln tokens. We conduct comprehensive experiments on OpenPSG dataset, which show that our method outperforms the previous state-of-the-art method. Furthermore, we also show the effectiveness of our framework by ablation studies. Our code is available at https://github.com/WHY-student/GroupRF .

The rise of social media platforms has greatly amplified the spread of hate speech, which poses serious societal risks. The automated detection of hate speech on social media, especially in low-resource Arabic language, presents unique challenges owing to linguistic diversity, dialectal differences, and regional nuances. However, most current research efforts primarily focus on a single social media platform, which hinders the ability to address dialect differences, as residents of Arab regions often favor one platform over another. This study provides an in-depth analysis of Arabic hate speech detection using advanced transformer-based models across three datasets collected from diverse social media platforms. Our models include multilingual, monolingual models pretrained in Arabic, and models that employ transfer learning from rich-resource English. To provide a thorough evaluation, we also compared the performance of our transformer-based models with two baseline models: LR and NBSVM, highlighting their relative effectiveness in detecting hate speech across multi-dialect and multi-platforms. Our analysis includes the effects of oversampling, data augmentation, and model interpretability using the LIME method. The monolingual transformer-based models, in particular, demonstrated significant performance improvements, setting new benchmarks for F1-scores and surpassing traditional models, our best classifier achieved an F1-score of 98.82%. Additionally, we conducted thorough cross-validation across datasets to evaluate the models’ generalization capabilities. Our research significantly advances Arabic hate speech detection by tackling these complexities and laying a solid foundation for future research. However, challenges remain in detecting subtle forms of hate speech, such as implicit hate speech and fine-grained distinctions between offensive content.

This is an erratum for the article "A Dual Perspective Framework of Knowledge-correlation for Cross-domain Recommendation" published in ACM Trans. Knowl. Discov. Data 18(6): 152:1-152:28 (2024).

Recently, GNN(Graph Neural Network) recommender systems have benefitted from contrastive learning as an auxiliary task of recommendation and have employed data augmentation to overcome the data sparsity problem. However, we find that the training process of contrastive learning is affected by the popularity bias due to the longtail distribution of interaction data, resulting in the inadequate feature training of low-degree nodes. To address this problem, we propose DCLGCF (Debiased Contrastive Learning For Graph Collaborative Filtering). More specifically, we propose two data augmentation methods with respect to popularity reduction and longtail enhancement. In addition, we propose Mixed-InfoNCE, which designs a novel mixed sampling strategy and introduce a new contrastive learning loss function by considering a frequency penalty term, aiming at increasing the contribution of longtail items to the gradient calculation, and enhancing the training of longtail item features. To validate the effectiveness of our proposed DCLGCF, we conduct thorough experiments on four real-world datasets. The results clearly demonstrate that DCLGCF outperforms existing models in terms of recommendation accuracy, and remarkable improvements are achieved especially when recommending longtail items.

In this paper, we explore the impact of the spatial properties of point clouds on 3D object detection in autonomous driving scenarios. To reduce the memory and computational costs, existing point-based models typically use random sampling or the farthest point sampling strategy to retain the foreground points or points closer to the center of the object. However, they treated points with different spatial properties equally, which led to the loss of potential relative geometric information in the point cloud. To this end, we design a two-stage detector that fully exploits the spatial properties of point clouds, termed Dual Position Aware 3D Object Detector(DPA-RCNN). Specifically, we first sample more points close to the object center by exploiting the relative position information between points and the object center. These sampling points can generate more precisely positioned proposals, which can reduce the difficulty of subsequent bounding box regression stages. In addition, we use the distance from the point to the edge of the bounding box to learn edge features in proposals, which further improves the regression accuracy of the bounding box. Experiments on KITTI and ONCE datasets validate the superiority and universality of our approach. Our method outperforms all point-based detectors, and the proposed Edge Point-aware Segmentation Module(EPAS) can effectively improve the detection accuracy of two-stage detectors.

In the task of image dehazing, it has been proven that high-quality codebook priors can be used to compensate for the distribution differences between real-world hazy images and synthetic hazy images, thereby helping the model improve its performance. However, because the concentration and distribution of haze in the image are irregular, the manners those simply replacing or blending the prior information in the codebook with the original image features are inconsistent with this irregularity, which leads to a non-ideal dehazing performance. To this end, we propose a haze concentration aware network (HcaNet), its haze-concentration-aware module (HcaM) can reduce the information loss in the vector quantization stage and achieve an adaptive domain transfer for regions with different degrees of degradation. To further capture the detailed texture information, we develop a frequency selective fusion module (FSFM) to facilitate the transmission of shallow information retained in haze areas to deeper layers, thereby enhancing the fusion with high-quality feature priors. Extensive evaluations demonstrate that the proposed model can be merely trained on synthetic hazy-clean pairs and effectively generalize to real-world data. Several experimental results confirm that the proposed dehazing model outperforms state-of-the-art methods significantly on real-world images.

Cross-domain recommendation (CDR) aims to utilize the information from relevant domains to guide the recommendation task in the target domain, and shows great potential in alleviating the data sparsity and cold-start problems of recommender systems. Most existing methods utilize the interaction information (e.g., ratings and clicks) or consider auxiliary information (e.g., tags and comments) to analyze the users' cross-domain preferences, but such kinds of information ignore the intrinsic semantic relationship of different domains. In order to effectively explore the inter-domain correlations, encyclopedic knowledge graphs (KG) involving different domains are highly desired in cross-domain recommendation tasks because they contain general information covering various domains with structured data format. However, there are few datasets containing KG information for CDR tasks, so in order to enrich the available data resource, we build a KG-enhanced cross-domain recommendation dataset, named Amazon-KG, based on the widely used Amazon dataset for CDR and the well-known KG DBpedia. In this work, we analyze the potential of KG applying in cross-domain recommendations, and describe the construction process of our dataset in detail. Finally, we perform quantitative statistical analysis on the dataset. We believe that datasets like Amazon-KG contribute to the development of knowledge-aware cross-domain recommender systems. Our dataset has been released at https://github.com/WangYuhan-0520/Amazon-KG-v2.0-dataset.

Food computing, a newly emerging topic, is closely linked to human life through computational methodologies. Meal recommendation, a food-related study about human health, aims to provide users a meal with courses constrained from specific categories (e.g., appetizers, main dishes) that can be enjoyed as a service. Historical interaction data, important user information, is often used by existing models to learn user preferences. However, if a user’s preferences favor less healthy meals, the model will follow that preference and make similar recommendations, potentially negatively impacting the user’s long-term health. This emphasizes the necessity for health-oriented and responsible meal recommendation systems. In this article, we propose a healthiness-aware and category-wise meal recommendation model called CateRec, which boosts healthiness exposure by using nutritional standards as knowledge to guide the model training. Two fundamental questions are raised and answered: (1) How can the healthiness of meals be evaluated? Two well-known nutritional standards from the World Health Organization and the United Kingdom Food Standards Agency are used to calculate the healthiness score of the meal. (2) How can the model training be guided in a health-oriented manner? We construct category-wise personalization partial rankings and category-wise healthiness partial rankings, and theoretically analyze that they meet the necessary properties and assumptions required to be trained by the maximum posterior estimator under Bayesian probability. The data analysis confirms the existence of user preferences leaning towards less healthy meals in two public datasets. A comprehensive experiment demonstrates that our CateRec effectively boosts healthiness exposure in terms of mean healthiness score and ranking exposure while being comparable to the state-of-the-art model in terms of recommendation accuracy.

Recommender System provides users with online services in a personalized way. The performance of traditional recommender systems may deteriorate because of problems such as cold-start and data sparsity. Cross-domain Recommendation System utilizes the richer information from auxiliary domains to guide the task in the target domain. However, direct knowledge transfer may lead to a negative impact due to data heterogeneity and feature mismatch between domains. In this article, we innovatively explore the cross-domain correlation from the perspectives of content semanticity and structural connectivity to fully exploit the information of Knowledge Graph. First, we adopt domain adaptation that automatically extracts transferable features to capture cross-domain semantic relations. Second, we devise a knowledge-aware graph neural network to explicitly model the high-order connectivity across domains. Third, we develop feature fusion strategies to combine the advantages of semantic and structural information. By simulating the cold-start scenario on two real-world datasets, the experimental results show that our proposed method has superior performance in accuracy and diversity compared with the SOTA methods. It demonstrates that our method can accurately predict users’ expressed preferences while exploring their potential diverse interests.

Sequential prediction has great value for resource allocation due to its capability in analyzing intents for next prediction. A fundamental challenge arises from real-world interaction dynamics where similar sequences involving multiple intents may exhibit different next items. More importantly, the character of volume candidate items in sequential prediction may amplify such dynamics, making deep networks hard to capture comprehensive intents. This article presents a sequential prediction framework with Decoupled Progressive Distillation (DePoD), drawing on the progressive nature of human cognition. We redefine target and non-target item distillation according to their different effects in the decoupled formulation. This can be achieved through two aspects: (1) Regarding how to learn, our target item distillation with progressive difficulty increases the contribution of low-confidence samples in the later training phase while keeping high-confidence samples in the earlier phase. And, the non-target item distillation starts from a small subset of non-target items from which size increases according to the item frequency. (2) Regarding whom to learn from, a difference evaluator is utilized to progressively select an expert that provides informative knowledge among items from the cohort of peers. Extensive experiments on four public datasets show DePoD outperforms state-of-the-art methods in terms of accuracy-based metrics.

Referring image segmentation aims to locate and segment the target region based on a given textual expression query. The primary challenge is to understand semantics from visual and textual modalities and achieve alignment and matching. Prior works have attempted to address this challenge by leveraging separately pretrained unimodal models to extract global visual and textual features and perform straightforward fusion to establish cross-modal semantic associations. However, these methods often concentrate solely on the global semantics, disregarding the hierarchical semantics of expression and image and struggling with complex and open real scenarios, thus failing to capture critical cross-modal information. To address these limitations, this article introduces an innovative mutually-guided hierarchical multi-modal feature learning scheme. By leveraging the guidance of global visual features, the model mines hierarchical text features from different stages of the text encoder. Simultaneously, the guidance of global textual features is leveraged to aggregate multi-scale visual features. This mutually guided hierarchical feature learning effectively addresses the semantically inaccurate cause by free-form text and naturally occurring scale variations. Furthermore, a Segment Detail Refinement (SDR) module is designed to enhance the model’s spatial detail awareness through attention mapping of low-level visual features and cross-modal features. To evaluate the effectiveness of the proposed approach, extensive experiments are conducted on three widely used referring image object segmentation datasets. The results demonstrate the superiority of the presented method in accurately locating and segmenting objects in images.

Current sentence-level evidence extraction based methods may lose the discourse coherence of legal articles since they tend to make the extracted sentences scattered over the article. To solve the problem, this paper proposes a Cascaded Answer-guided key segment learning framework for long Legal article Question Answering, namely CALQA. The framework consists of three cascaded modules: Sifter, Reader, and Responder. The Sifter transfers a long legal article into several segments and works in an answer-guided way by automatically sifting out key fact segments in a coarse-to-fine approach through multiple iterations. The Reader utilizes a set of attention mechanisms to obtain semantic representations of the question and key fact segments. Finally, considering it a multi-label classification task the Responder predicts final answers in a cascaded manner. CALQA outperforms state-of-the-art methods in CAIL 2021 Law dataset.

Tag recommendation in software information sites is a significant task to help developers make distinctions among software objects. Most existing methods usually ignore the semantic information of code snippets in software information sites. To tackle this issue, we regard the code as a semantic enhancement signal, and propose a novel Code-Enhanced fine-grained semantic matching method for Tag Recommendation in software information sites (CETR) to learn the matching score between tags and software objects. In our CETR, code-enhanced semantic interaction is designed to capture fine-grained semantic relevance between tags and software objects. Specifically, code snippets and text descriptions in software objects are first encoded in two specialized ways to capture semantic information from the perspective of natural language and structural language respectively. The semantic interaction is then learned between code snippets and text descriptions. Besides, a hierarchy-aware semantic learning block attentively combines the features from different layers as the final feature to capture semantic information of different levels. Lastly, a pairwise sentence matching computation block is exploited to predict matching probability. Experimental results on four software information site datasets have demonstrated the effectiveness of our proposed CETR for tag recommendation compared with the state-of-the-art methods.

Thangka cultural elements detection aims to locate and identify instances in Thangka. However, as a unique form of pictorial art, Thangka exhibits distinct spatial structures that deviate significantly from general images in scale and density. Therefore, it is challenging for most state-of-the-art detectors designed for natural scenes to handle Thangka cultural elements detection effectively. To overcome this issue, we propose a multi-scale and dense object detector referred as MDDet. It embeds a multi-scale receptive field fusion module (MRF) that enlarges the receptive field while capturing the spatial and channel relationships at different scales, which significantly enriches the multi-scale features extracted from the backbone. In addition, we introduce a threshold-slicing aided hyper inference (T-SAHI) scheme, which adaptively slices images in dense scenarios to aid with dense object detection in the test time. We thoroughly evaluate our method, and MDDet outperforms the prior art by a clear margin on the Thangka dataset, achieving an absolute improvement of 1.9% in average precision (AP). For the challenging medium and small objects in Thangka, MDDet obtains wide margins of 12% and 3.7% in accuracy improvement, respectively. It also shows strong generalization ability when evaluated on general scenarios, e.g., Pascal VOC 2007 and MS COCO, validating the role of MDDet in object detection.

Hard interaction learning between source sequences and their next targets is challenging, which exists in a myriad of sequential prediction tasks. During the training process, most existing methods focus on explicitly hard interactions caused by wrong responses. However, a model might conduct correct responses by capturing a subset of learnable patterns, which results in implicitly hard interactions with some unlearned patterns. As such, its generalization performance is weakened. The problem gets more serious in sequential prediction due to the interference of substantial similar candidate targets.

Cross-modal retrieval technology can help people quickly achieve mutual information between cooking recipes and food images. Both the embeddings of the image and the recipe consist of multiple representation subspaces. We argue that multiple aspects in the recipe are related to multiple regions in the food image. It is challenging to improve the cross-modal retrieval quality by making full use of the implicit connection between multiple subspaces of recipes and images. In this paper, we propose a multi-subspace implicit alignment cross-modal retrieval framework of recipes and images. Our framework learns multi-subspace information about cooking recipes and food images with multi-head attention networks; the implicit alignment at the subspace level promotes narrowing the semantic gap between recipe embeddings and food image embeddings; triple loss and adversarial loss are combined to help our framework for cross-modal learning. The experimental results show that our framework significantly outperforms to state-of-the-art methods in terms of MedR and [email protected] on Recipe 1M.

Multi-Label hash methods have achieved excellent performance in multi-label image retrieval, but how to leverage the semantic information of label to improve retrieval quality is still a challenge in this field. This paper proposes GAT-Assisted Deep Hashing (DHGAT). Our model uses Convolutional Neural Network (CNN) to extract image-level features, along with graph attention network (GAT) to extract label-level features. Assisted by GAT, DHGAT is able to pay more attention on the co-occurrence of label. In order to solve the problem of feature fusion, we propose Multi-modal Max-Pooling Bilinear (MMB) mechanism, which MMB fuses image-level feature and label-level feature to generate abundant semantic features, so that the model can output discriminative hash code. Extensive experiments demonstrate that the proposed method can generate hash codes which achieve better retrieval performance on two benchmark datasets, NUS-WIDE and MS-COCO.

Person re-identification (re-id) has received ever-increasing research focus, because of its important role in video surveillance applications. This paper addresses the re-id problem between visible images of color cameras and infrared images of infrared cameras, which is significant in case that the appearance information is insufficient in poor illumination conditions. In this field, there are two key challenges, i.e., the difficulty to locate the discriminative information to re-identify the same person between visible and infrared images, and the difficulty to learn a robust metric for such large-scale cross-modality retrieval. In this paper, we propose a novel human body parts assistance network (BANet) to tackle the two challenges above. BANet mainly focuses on extracting discriminative information and learning robust features by leveraging the human body part cues. Extensive experiments demonstrate that the proposed approach outperforms the baseline and the state-of-the-art methods.

This paper proposes a novel approach for pose transfer, which aims at transferring the pose of a given person to a target pose. Unlike previous works directly simulating the target pose, we emphasize the pose geometric constraint in our approach to tackle this task. To capture the geometric constraints among the condition pose and target pose, we generate a sequence of different intermediate pose skeleton. Moreover, we introduced the 3D convolution into the generator of our Generative Adversarial Network (GAN) in order to learn spatiotemporal features from the generated pose skeleton sequence. The final target pose will be estimated and refined by a pose transfer block and a series of residual blocks based on the spatiotemporal features. Compared with the existing works, our generated person pose images achieved better performance in terms of Inception Score and Structure Similarity. Extensive experiments on dataset Market-1501 demonstrate the effectiveness of the pose skeleton sequence on pose transfer.

As a cross modality retrieval task, RGB-infrared person re-identification(Re-ID) is an important and challenging task, because of its important role in video surveillance applications and large cross-modality variations between visible and infrared images. Most previous works addressed the problem of cross-modality gap with feature alignment by original feature representation learning straightly. In this paper, different from existing works, we propose a novel network(CE2L) to tackle the cross-modality gap with feature alignment. CE2L mainly focuses on adding discriminative information and learning robust features by converting modality between visible and infrared images. Its merits are highlighted in two aspects: 1)Using CycleGAN to convert infrared images into color images can not only increase the recognition characteristics of images, but also allow the network to better learn the two modal image features; 2)Our novel method can serve as data augmentation. Specifically, it can increase data diversity and total data against over-fitting by converting labeled training images to another modal images. Extensive experimental results on two datasets demonstrate superior performance compared to the baseline and the state-of-the-art methods.

Along with the fast growing web-based applications, the recommender system is now attracting much attention due to its core function that matches the target users’ interest with the potential resources from the massive online information. Since the recommender system is a user centric application, in this work, we propose a recommendation framework based on user interaction, so as to explore the user’s real-time interest from the instant feedback. Naturally, we utilize the tag information assigned to different resources as the medium for user interaction. During the interaction, the most effective tags will be provided for users to choose, and the chosen tag words will be considered as the personalized preference and utilized to dynamically adjust the recommendation list during the process. However, the interaction procedure may cause the problem of potential false dismissal during the candidate filtering. In this work, we propose to analyze the association between different tags, and utilize the tag co-occurrence to refine the recommendation candidate, so as to avoid false dismissal. To generate the recommendation list from the filtered candidates, we design the representation of user and resource characteristics based on tag information and user historical behavior. We distinguish the significance of each tag word for the corresponding resource item, so as to precisely describe the item feature. Probabilistic matrix factorization is employed in our work to overcome the rating sparsity, which is enhanced by embedding the similar user and resource information. The experiments on real-world datasets demonstrate that the proposed algorithm can achieve more accurate predictions and higher recommendation efficiency.

The problem we aim to address is the optimization of cost management for executing multiple continuous queries on data streams, where each query is defined by several filters, each of which monitors certain status of the data stream. Specially, the filter can be shared by different queries and expensive to evaluate. The conventional objective for such a problem is to minimize the overall execution cost to solve all queries, by planning the order of filter evaluation in shared strategy. However, in the streaming scenario, the characteristics of data items may change in process, which can bring some uncertainty to the outcome of individual filter evaluation, and affect the plan of query execution as well as the overall execution cost. In our work, considering the influence of the uncertain variation of data characteristics, we propose a framework to deal with the dynamic adjustment of filter ordering for query execution on data stream, and focus on the issues of cost management. By incrementally monitoring and analyzing the results of filter evaluation, our proposed approach can be effectively adaptive to the varied stream behavior and adjust the optimal ordering of filter evaluation, so as to optimize the execution cost. In order to achieve satisfactory performance and efficiency, we also discuss the trade-off between the adaptivity of our framework and the overhead incurred by filter adaption. The experimental results on synthetic and two real data sets (traffic and multimedia) show that our framework can effectively reduce and balance the overall query execution cost and keep high adaptivity in streaming scenario.

Web Harvesting enables the enrichment of incomplete data sets by retrieving required information from the Web. However, the ambiguity of instances may greatly decrease the quality of the harvested data, given that any instance in the local data set may become ambiguous when attempting to identify it on the Web. Although plenty of disambiguation methods have been proposed to deal with the ambiguity problems in various settings, none of them are able to handle the instance ambiguity problem in Web Harvesting. In this paper, we propose to do instance disambiguation in Web Harvesting with a novel disambiguation method inspired by the idea of collaborative identity recognition. In particular, we expect to find some common properties in forms of latent shared attribute values among instances in the list, such that these shared attribute values can differentiate instances within the list against those ambiguous ones on the Web. Our extensive experimental evaluation illustrates the utility of collaborative disambiguation for a popular Web Harvesting application, and shows that it substantially improves the accuracy of the harvested data.

This paper addresses the challenges in detecting the potential correlation between numerical data streams, which facilitates the research of data stream mining and pattern discovery. We focus on local correlation with delay, which may occur in burst at different time in different streams, and last for a limited period. The uncertainty on the correlation occurrence and the time delay make it difficult to monitor the correlation online. Furthermore, the conventional correlation measure lacks the ability of reflecting visual linearity, which is more desirable in reality. This paper proposes effective methods to continuously detect the correlation between data streams. Our approach is based on the Discrete Fourier Transform to make rapid cross-correlation calculation with time delay allowed. In addition, we introduce a shape-based similarity measure into the framework, which refines the results by representative trend patterns to enhance the significance of linearity. The similarity of proposed linear representations can quickly estimate the correlation, and the window sliding strategy in segment level improves the efficiency for online detection. The empirical study demonstrates the accuracy of our detection approach, as well as more than $30\%$ improvement of efficiency.

Piecewise Linear Representation (PLR) has been a widely used method for approximating data streams in the form of compact line segments. The buffer-based approach to PLR enables a semi-global approximation which relies on the aggregated processing of batches of streamed data so that to adjust and improve the approximation results. However, one challenge towards applying the buffer-based approach is allocating the necessary memory resources for stream buffering. This challenge is further complicated in a multi-stream environment where multiple data streams are competing for the available memory resources, especially in resource-constrained systems such as sensors and mobile devices.

Online video steam data is surging to an unprecedented level. Massive video publishing and sharing impose heavy demands on continuous video near-duplicate detection for many novel video applications. This paper presents an accurate and accelerated system for video near-duplicate detection over continuous video streams. We propose to transform a high-dimensional video stream into a one-dimensional Video Trend Stream (VTS) to monitor the continuous luminance changes of consecutive frames, based on which video similarity is derived. In order to do fast comparison and effective early pruning, a compact auxiliary signature named CutSig is proposed to approximate the video structure. CutSig explores cut distribution feature of the video structure and contributes to filter candidates quickly. To scan along a video stream in a rapid way, shot cuts with local maximum AI (average information) in a query video are used as reference cuts, and a skipping approach based on reference cut alignment is embedded for efficient acceleration. Extensive experimental results on detecting diverse near-duplicates in real video streams show the effectiveness and efficiency of our method.