Cognitive Diagnosis (CD) models are constrained by the data quality of students' response logs. Recent advancements in Large Language Model (LLM) based data augmentation show promise for enhancing CD. However, ensuring the reliability and accuracy of LLM-generated annotations remains a significant challenge. In this paper, we propose Multi-Agent based Concept Augmentation for Cognitive Diagnosis (MACA-CD), a novel approach that enhances CD by generating and fusing reliable concept descriptions and relations based solely on concept names. MACA-CD consists of two main components: (1) a Multi-Agent Debate (MAD) based concept augmentation process that generates diverse and reliable concept descriptions and relations, reducing reliance on behavioral data. For concept descriptions, two agents generate outputs that include definitions, core features, and real-world applications, and continue debating until a judge agent determines that consensus has been reached. Concept relations are then identified using a Breadth-First Search approach to efficiently and progressively uncover relationships based on concept descriptions, with each step carried out by MAD. (2) a concept augmentation-enhanced CD model that refines concept embeddings using a graph self-supervised learning fusion layer and a pairwise comparator-based Description Fusion Layer, leading to more reliable and accurate concept embeddings. Experimental results on three real-world datasets show that MACA-CD consistently outperforms existing methods under various real-world scenarios.

As Large Language Models (LLMs) increasingly shape online content, how to remove targeted information from well-trained LLMs (also known as LLM unlearning) has become increasingly critical for web governance. A key challenge in LLM unlearning lies in the sample-wise imbalance within the forget set: different samples exhibit widely varying unlearning difficulty, leading to asynchronous forgetting speeds where some knowledge remains insufficiently erased while others become over-forgotten. To address this challenge, we propose BalDRO, a novel and efficient framework for balanced LLM unlearning. BalDRO formulates unlearning as a min–sup process, where the inner process identifies a worst-case data distribution that adaptively emphasizes hard-to-unlearn samples, while the outer process updates model parameters based on the worst-case data distribution. We instantiate this formulation through two efficient variants: BalDRO-G, a discrete GroupDRO-based approximation that focuses on high-loss subsets, and BalDRO-DV, a continuous Donsker–Varadhan dual method that enables smooth, adaptive weighting within standard LLM training pipelines. Extensive experiments on the TOFU and MUSE benchmarks demonstrate the effectiveness of our proposed BalDRO, yielding significant improvements in both forgetting quality and model utility over existing methods. For reproducibility, we have released the code for BalDRO. https://github.com/nxZhai/BalDRO.

Depressive disorders represent a major global public health challenge. As an increasing number of individuals share their emotional experiences and concerns on social media, researchers have shown growing interest in leveraging such data for early depression screening. However, most existing methods rely on a fixed model and a singular reasoning paradigm, which constrains their adaptability to depression detection. The limited availability of mental health-related data and variability in training data distributions across different LLMs hinder their consistent and comprehensive understanding of diverse psychological symptoms. In this paper, we propose AdaDepression, a framework that enables explainable depression screening through a two-hop retrieval algorithm to identify symptom-relevant posts and a two-stage adaptive routing mechanism for selecting appropriate reasoning strategies and LLMs. Specifically, we first collect representative posts from the training dataset to capture the real-world symptom expressions, and then utilize these posts to retrieve symptom-relevant posts from the user's posting history. Subsequently, we employ the Mixture of Routers (MoR), which integrates the Mixture of Experts (MoE) into the routing mechanism to select the optimal reasoning strategies and LLMs in a cascaded manner. Finally, we complete the standardized psychological questionnaire using the selected LLMs and reasoning strategies. Experimental results on the Reddit-based benchmarks demonstrate the effectiveness of the proposed method, outperforming existing studies on various metrics. Our code is released at https://github.com/MindIntLab-HFUT/AdaDepression.

Knowledge Distillation (KD) is a widely popular model compression technique that can effectively transfer knowledge from a pre-trained, large-scale teacher model to a more compact and lightweight student model. Traditional KD methods aim to improve the student’s representation capability by mimicking the teacher’s features, e.g., minimizing the \(\mathcal{L}_{2}\) distance between their intermediate features. However, due to the capacity gap between the student and the teacher, student often struggles to precisely mimic the features of the teacher. To address this challenge, we propose to boost the knowledge distillation for the visual recognition tasks via Direction Alignment and Discrepancy Standardization ( DADS) , which exploits the feature scaling technique to distill from both the feature direction and feature discrepancy. To this end, we devise an efficient feature alignment module to align the dimensions of teacher and student features. Moreover, we align the direction of student features and teacher features, which are pre-processed by normalization. Furthermore, we leverage the Kullback–Leibler (KL) divergence to refine the features alignment, minimizing discrepancy in the distribution of features across samples, which is pre-processed by \(\mathcal{Z}\) -score standardization. In this way, our proposed approach can effectively transfer the knowledge from the teacher to the student, facilitating the downstream visual recognition applications, such as image classification and semantic segmentation. Extensive experimental analyses clearly validate the effectiveness of DADS . Compared with previous KD methods, our approach sets a new benchmark, achieving state-of-the-art results on visual recognition tasks.

Cognitive Diagnosis Modeling is a fundamental task in intelligent education, intending to assess students’ mastery levels on knowledge concepts through interactions. Previous methodologies prioritized enhancing average diagnostic accuracy. However, they often neglected the Long-Tailed issue in interactions. To address this issue, we propose an Adaptive Self-Supervised Graph Learning for Cognitive Diagnosis (ASCD) framework that leverages relatively balanced sparse views to forcing the graph network to focus on long-tailed nodes, aiming to tackle the long-tailed problem in graph-based cognitive diagnosis. Additionally, we employ self-supervised manners to mitigate the impact of dropped information on other nodes. Our approach leverages the adaptive graph confusion method to create sparse views of the original student-exercise interaction graph. In these sparse views, both long-tailed and head students carry similar weights, pushing the graph network to allocate attention impartially across all students. We integrate two different graph confusion techniques into adaptive graph confusion to accommodate varying degrees of data sparsity: edge dropout and feature masking. ASCD can serve as a plug-and-play module integrated into any graph-based cognitive diagnosis model, enhancing its performance regarding long-tailed scenarios. Extensive experiments on real-world datasets show the effectiveness of our approach, especially on the students with much sparser interaction records.

Large language models (LLMs) enable diverse forms of AI-assisted creation, yet they often struggle to bridge the preference-articulation gap: users may provide incomplete or vague intentions or lack the vocabulary to specify what they want, yielding outputs misaligned with true preferences. To address this gap and facilitate music creation in a vibe-centric environment, we introduce VibeMus, a proactive agentic system built on open-source components. The system engages in multi-turn dialogue to progressively determine the music’s emotion, genre, lyrics, and other aspects before generation. Simulated evaluations show that proactive clarification improves alignment with users’ intended nuances. Our approach is training-free, leveraging an open-source music model, an open-source agentic framework, and publicly available LLM APIs. We release our code, showcase several demos, and provide additional details at https://github.com/tuteng0915/VibeMus.

Achieving ID-preserving text-to-video (T2V) generation remains challenging despite recent advances in diffusion-based models. Existing approaches often fail to capture fine-grained facial dynamics or maintain temporal identity coherence. To address these limitations, we propose MoCA, a novel Video Diffusion Model built on a Diffusion Transformer (DiT) backbone, incorporating a Mixture of Cross-Attention mechanism inspired by the Mixture-of-Experts paradigm. Our framework improves inter-frame identity consistency by embedding MoCA layers into each DiT block, where Hierarchical Temporal Pooling captures identity features over varying timescales, and Temporal-Aware Cross-Attention Experts dynamically model spatiotemporal relationships. We further incorporate a Latent Video Perceptual Loss to enhance identity coherence and fine-grained details across video frames. To train this model, we collect CelebIPVid, a dataset of 10,000 high-resolution videos from 1,000 diverse individuals, promoting cross-ethnic generalization. Extensive experiments on CelebIPVid show that MoCA outperforms existing T2V methods by over 5% across facial similarity.

Recent advancements in sequential recommendation have underscored the potential of Large Language Models (LLMs) for enhancing item embeddings. However, existing approaches face three key limitations: 1) the degradation of the semantic space when high-dimensional language embeddings are mapped to lower-dimensional ID embeddings, 2) the underutilization of language embeddings, and 3) the reliance on additional trainable parameters, such as an adapter, to bridge the gap between the semantic and behavior spaces. In this paper, we introduce AlphaFuse, a simple but effective language-guided learning strategy that addresses these challenges by learning ID embeddings within the null space of language embeddings. Specifically, we decompose the semantic space of language embeddings via Singular Value Decomposition (SVD), distinguishing it into a semantic-rich row space and a semantic-sparse null space. Collaborative signals are then injected into the null space, while preserving the rich semantics of the row space. AlphaFuse prevents degradation of the semantic space, integrates the retained language embeddings into the final item embeddings, and eliminates the need for auxiliary trainable modules, enabling seamless adaptation to any sequential recommendation framework. We validate the effectiveness and flexibility of AlphaFuse through extensive experiments on three benchmark datasets, including cold-start user and long-tail settings, showcasing significant improvements in both discriminative and diffusion-based generative sequential recommenders.

Deep neural networks (DNNs) have achieved remarkable success in various computer vision tasks, e.g., image classification. However, most of the existing models depend heavily on annotated data, where label noise is inevitable. Training with such noisy data negatively impacts the generalization performance of DNNs. To this end, recent advances in learning with noisy labels (LNL) adopt the sample selection strategy that identifies clean samples from the noisy dataset to update DNNs, using semi-supervised learning where rejected samples are treated as unlabeled data. However, existing LNL methods often overlook the varying fitting difficulties of different classes, resulting in suboptimal sample selection and confirmation bias, and consequently, the errors accumulate during semi-supervised training. In this article, we propose a novel method, TNCollab, which aims at alleviating confirmation bias in both sample selection and semi-supervised training stages via two-network collaboration. Specifically, we introduce a class-adaptive threshold for sample selection to address the varying fitting difficulties across different classes. Additionally, we construct a hard set consisting of samples where the two networks disagree and introduce a noise-robust loss to extract potentially useful information while maintaining robustness against label noise. Furthermore, we propose a dual consistency loss to ensure consistent predictions between the networks across different augmented views of the same sample, facilitating mutual learning. Extensive experiments demonstrate that TNCollab achieves state-of-the-art performance on image classification and facial expression recognition tasks, particularly on CIFAR-10, CIFAR-100, WebVision, Clothing1M, Tiny-ImageNet, and RAF-DB datasets, showing improved visual understanding and generalization capabilities. Our codes are available at https://github.com/Delete12137/TNCollab .

Knowledge tracing has been widely used in online learning systems to guide the students’ future learning. However, most existing KT models primarily focus on extracting abundant information from the question sets and explore the relationships between them, but ignore the personalized student behavioral information in the learning process. This will limit the model’s ability to accurately capture the personalized knowledge states of students and reasonably predict their performances. To alleviate this limitation, we explicitly models the personalized learning process by incorporating the emotions, a representative personalized behavior in the learning process, into KT framework. Specifically, we present a novel Dual-State Personalized Knowledge Tracing with Emotional Incorporation model to achieve this goal: First, we incorporate emotional information into the modeling process of knowledge state, resulting in the Knowledge State Boosting Module. Second, we design an Emotional State Tracing Module to monitor students’ personalized emotional states, and propose an emotion prediction method based on personalized emotional states. Finally, we apply the predicted emotions to enhance students’ response prediction. Furthermore, to extend the generalization capability of our model across different datasets, we design a transferred version of DEKT, named Transfer Learning-based Self-loop model (T-DEKT). Extensive experiments show our method achieves the state-of-the-art performance.

Vision-and-language pre-training models have achieved impressive performance for image captioning. But most of them are trained with millions of paired image-text data and require huge memory and computing overhead. To alleviate this, we try to stand on the shoulders of large-scale pre-trained language models (PLM) and pre-trained vision models (PVM) and efficiently connect them for image captioning. There are two major challenges: one is that language and vision modalities have different semantic granularity (e.g., a noun may cover many pixels), and the other is that the semantic gap still exists between the pre-trained language and vision models. To this end, we design a lightweight and efficient connector to glue PVM and PLM, which holds a criterion of selection-then-transformation . Specifically, in the selection phase, we treat each image as a set of patches instead of pixels. We select salient image patches and cluster them into visual regions to align with text. Then, to effectively reduce the semantic gap, we propose to map the selected image patches into text space through spatial and channel transformations. With training on image captioning datasets, the connector learns to bridge the semantic granularity and semantic gap via backpropagation, preparing for the PLM to generate descriptions. Experimental results on the MSCOCO and Flickr30k datasets demonstrate that our method yields comparable performance to existing works. By solely training the small connector, we achieve a CIDEr performance of 132.2% on the MSCOCO Karpathy test split. Moreover, our findings reveal that fine-tuning the PLM can further enhance performance potential, resulting in a CIDEr score of 140.6%. Code and models are available at https://github.com/YuanEZhou/PrefixCap .

Cross-lingual cross-modal retrieval aims at leveraging human-labeled annotations in a source language to construct cross-modal retrieval models for a new target language, due to the lack of manually-annotated dataset in low-resource languages (target languages). Contrary to the growing developments in the field of monolingual cross-modal retrieval, there has been less research focusing on cross-modal retrieval in the cross-lingual scenario. A straightforward method to obtain target-language labeled data is translating source-language datasets utilizing Machine Translations (MT). However, as MT is not perfect, it tends to introduce noise during translation, rendering textual embeddings corrupted and thereby compromising the retrieval performance. To alleviate this, we propose Noise-Robust Fine-tuning (NRF) which tries to extract clean textual information from a possibly noisy target-language input with the guidance of its source-language counterpart. Besides, contrastive learning involving different modalities are performed to strengthen the noise-robustness of our model. Different from traditional cross-modal retrieval methods which only employ image/video-text paired data for fine-tuning, in NRF, selected parallel data plays a key role in improving the noise-filtering ability of our model. Extensive experiments are conducted on three video-text and image-text retrieval benchmarks across different target languages, and the results demonstrate that our method significantly improves the overall performance without using any image/video-text paired data on target languages.

Recommender systems suffer from confounding biases when there exist confounders affecting both item features and user feedback (e.g.like or not). Existing causal recommendation methods typically assume confounders are fully observed and measured, forgoing the possible existence of hidden confounders in real applications. For instance, product quality is a confounder since it affects both item prices and user ratings, but is hidden for the third-party e-commerce platform due to the difficulty of large-scale quality inspection; ignoring it could result in the bias effect of over-recommending high-price items. This work analyzes and addresses the problem from a causal perspective. The key lies in modeling the causal effect of item features on a user's feedback. To mitigate hidden confounding effects, it is compulsory but challenging to estimate the causal effect without measuring the confounder. Towards this goal, we propose a Hidden Confounder Removal (HCR) framework that leverages front-door adjustment to decompose the causal effect into two partial effects, according to the mediators between item features and user feedback. The partial effects are independent from the hidden confounder and identifiable. During training, HCR performs multi-task learning to infer the partial effects from historical interactions. We instantiate HCR for two scenarios and conduct experiments on three real-world datasets. Empirical results show that the HCR framework provides more accurate recommendations, especially for less-active users. We will release the code once accepted.

Multi-behavioral recommender systems have emerged as a solution to address data sparsity and cold-start issues by incorporating auxiliary behaviors alongside target behaviors. However, existing models struggle to accurately capture varying user preferences across different behaviors and fail to account for diverse item preferences within behaviors. Various user preference factors (such as price or quality) entangled in the behavior may lead to sub-optimization problems. Furthermore, these models overlook the personalized nature of user behavioral preferences by employing uniform transformation networks for all users and items. To tackle these challenges, we propose the Disentangled Cascaded Graph Convolutional Network (Disen-CGCN), a novel multi-behavior recommendation model. Disen-CGCN employs disentangled representation techniques to effectively separate factors within user and item representations, ensuring their independence. In addition, it incorporates a multi-behavioral meta-network, enabling personalized feature transformation across user and item behaviors. Furthermore, an attention mechanism captures user preferences for different item factors within each behavior. By leveraging attention weights, we aggregate user and item embeddings separately for each behavior, computing preference scores that predict overall user preferences for items. Our evaluation of benchmark datasets demonstrates the superiority of Disen-CGCN over state-of-the-art models, showcasing an average performance improvement of 7.07% and 9.00% on respective datasets. These results highlight Disen-CGCN’s ability to effectively leverage multi-behavioral data, leading to more accurate recommendations.

Attribute-specific fashion retrieval (ASFR) is a challenging information retrieval task, which has attracted increasing attention in recent years. Different from traditional fashion retrieval which mainly focuses on optimizing holistic similarity, the ASFR task concentrates on attribute-specific similarity, resulting in more fine-grained and interpretable retrieval results. As the attribute-specific similarity typically corresponds to the specific subtle regions of images, we propose a Region-to-Patch Framework (RPF) that consists of a region-aware branch and a patch-aware branch to extract fine-grained attribute-related visual features for precise retrieval in a coarse-to-fine manner. In particular, the region-aware branch is first to be utilized to locate the potential regions related to the semantic of the given attribute. Then, considering that the located region is coarse and still contains the background visual contents, the patch-aware branch is proposed to capture patch-wise attribute-related details from the previous amplified region. Such a hybrid architecture strikes a proper balance between region localization and feature extraction. Besides, different from previous works that solely focus on discriminating the attribute-relevant foreground visual features, we argue that the attribute-irrelevant background features are also crucial for distinguishing the detailed visual contexts in a contrastive manner. Therefore, a novel E-InfoNCE loss based on the foreground and background representations is further proposed to improve the discrimination of attribute-specific representation. Extensive experiments on three datasets demonstrate the effectiveness of our proposed framework, and also show a decent generalization of our RPF on out-of-domain fashion images. Our source code is available at https://github.com/HuiGuanLab/RPF.

Conversational systems have recently attracted significant attention. Both the research community and industry believe that it will exert huge impact on human-computer interaction, and specifically, the IR/RecSys community has begun to explore Conversational Recommendation. In real-life scenarios, such systems are often urgently needed in helping users accomplishing different tasks under various situations. However, existing works still face several shortcomings: (1) Most efforts are largely confined in single task setting. They fall short of hands in handling tasks across domains. (2) Aside from soliciting user preference from dialogue history, a conversational recommender naturally has access to the back-end data structure which should be fully leveraged to yield good recommendations. In this paper, we thus present a Topic-guided Conversational Recommender (TCR) which is specifically designed for the multi-domain setting. It augments the sequence-to-sequence (seq2seq) models with a neural latent topic component to better guide the response generation. To better leverage the dialogue history and the back-end data structure, we adopt a graph convolutional network (GCN) to model the relationships between different recommendation candidates while also capture the match between candidates and the dialogue history. We then seamlessly combine these two parts with the idea of pointer networks. We perform extensive evaluation on a large-scale task-oriented multi-domain dialogue dataset and the results show that our method achieves superior performance as compared to a wide range of baselines.

This companion paper supports the replication of the fashion trend forecasting experiments with the KERN (Knowledge Enhanced Recurrent Network) method that we presented in the ICMR 2020. We provide an artifact that allows the replication of the experiments using a Python implementation. The artifact is easy to deploy with simple installation, training and evaluation. We reproduce the experiments conducted in the original paper and obtain similar performance as previously reported. The replication results of the experiments support the main claims in the original paper.

We tackle the task of video moment retrieval (VMR), which aims to localize a specific moment in a video according to a textual query. Existing methods primarily model the matching relationship between query and moment by complex cross-modal interactions. Despite their effectiveness, current models mostly exploit dataset biases while ignoring the video content, thus leading to poor generalizability. We argue that the issue is caused by the hidden confounder in VMR, i.e., temporal location of moments, that spuriously correlates the model input and prediction. How to design robust matching models against the temporal location biases is crucial but, as far as we know, has not been studied yet for VMR.

Fashion trend forecasting is a crucial task for both academia andindustry. Although some efforts have been devoted to tackling this challenging task, they only studied limited fashion elements with highly seasonal or simple patterns, which could hardly reveal thereal fashion trends. Towards insightful fashion trend forecasting,this work focuses on investigating fine-grained fashion element trends for specific user groups. We first contribute a large-scale fashion trend dataset (FIT) collected from Instagram with extracted time series fashion element records and user information. Furthermore, to effectively model the time series data of fashion elements with rather complex patterns, we propose a Knowledge Enhanced Recurrent Network model (KERN) which takes advantage of the capability of deep recurrent neural networks in modeling time series data. Moreover, it leverages internal and external knowledgein fashion domain that affects the time-series patterns of fashion element trends. Such incorporation of domain knowledge further enhances the deep learning model in capturing the patterns of specific fashion elements and predicting the future trends. Extensive experiments demonstrate that the proposed KERN model can effectively capture the complicated patterns of objective fashion elements, therefore making preferable fashion trend forecast.

The rapid growth of user-generated videos on the Internet has intensified the need for text-based video retrieval systems. Traditional methods mainly favor the concept-based paradigm on retrieval with simple queries, which are usually ineffective for complex queries that carry far more complex semantics. Recently, embedding-based paradigm has emerged as a popular approach. It aims to map the queries and videos into a shared embedding space where semantically-similar texts and videos are much closer to each other. Despite its simplicity, it forgoes the exploitation of the syntactic structure of text queries, making it suboptimal to model the complex queries.

Learning effective visual similarity is an essential problem in multimedia research. Despite the promising progress made in recent years, most existing approaches learn visual features and similarities in two separate stages, which inevitably limits their performance. Once useful information has been lost in the feature extraction stage, it can hardly be recovered later. This article proposes a novel end-to-end approach for visual similarity modeling, calleddeep neighborhood component analysis, which discriminatively trains deep neural networks to jointly learn visual features and similarities. Specifically, we first formulate a metric learning objective that maximizes the intra-class correlations and minimizes the inter-class correlations under the neighborhood component analysis criterion, and then train deep convolutional neural networks to learn a nonlinear mapping that projects visual instances from original feature space to a discriminative and neighborhood-structure-preserving embedding space, thus resulting in better performance. We conducted extensive evaluations on several widely used and challenging datasets, and the impressive results demonstrate the effectiveness of our proposed approach.

Most existing image enhancement methods function like a black box, which cannot clearly reveal the procedure behind each image enhancement operation. To overcome this limitation, in this paper, we design a progressive image enhancement framework, which generates an expected "good" retouched image with a group of self-interpretable image filters under the guidance of an aesthetic assessment model. The introduced aesthetic network effectively alleviates the shortage of paired training samples by providing extra supervision, and eliminate the bias caused by human subjective preferences. The self-interpretable image filters designed in our image enhancement framework, make the overall image enhancing procedure easy-to-understand. Extensive experiments demonstrate the effectiveness of our proposed framework.

With the rapid evolution of social networks, the increasing user intention gap and visual semantic gap both bring great challenge for users to access satisfied contents. It becomes promising to investigate users' customized multimedia recommendation. In this paper, we propose cross-modal collaborative manifold propagation (CMP) for image recommendation. CMP leverages users' interest distribution to propagate images' user records, which lets users know the trend from others and produces interest-aware image candidates upon users' interests. Visual distribution is investigated simultaneously to propagate users' visual records along dense semantic visual manifold. Visual manifold propagation helps to estimate semantic accurate user-image correlations for the candidate images in recommendation ranking. Experimental performance demonstrate the collaborative user-image inferring ability of CMP with effective user interest manifold propagation and semantic visual manifold propagation in personalized image recommendation.

Understanding the objects and relations between them is indispensable to fine-grained video content analysis, which is widely studied in recent research works in multimedia and computer vision. However, existing works are limited to evaluating with either small datasets or indirect metrics, such as the performance over images. The underlying reason is that the construction of a large-scale video dataset with dense annotation is tricky and costly. In this paper, we address several main issues in annotating objects and relations in user-generated videos, and propose an annotation pipeline that can be executed at a modest cost. As a result, we present a new dataset, named VidOR, consisting of 10k videos (84 hours) together with dense annotations that localize 80 categories of objects and 50 categories of predicates in each video. We have made the training and validation set public and extendable for more tasks to facilitate future research on video object and relation recognition.

Understanding the mix-and-match relationships of fashion items receives increasing attention in fashion industry. Existing methods have primarily utilized the visual content to learn the visual compatibility and performed matching in a latent space. Despite their effectiveness, these methods work like a black box and cannot reveal the reasons that two items match well. The rich attributes associated with fashion items, e.g.,off-shoulder dress and black skinny jean, which describe the semantics of items in a human-interpretable way, have largely been ignored.