Hateful videos have become prevalent on online platforms, highlighting an urgent need for effective detection. However, existing studies primarily focus on binary classification and fail to provide contextual rationales that reveal the implicit meanings behind these judgments, significantly undermining model explainability. To fill this gap, we aim to achieve explainable hateful video detection, enabling models to provide contextual rationales that integrate relevant evidence and logical reasoning alongside decisions. This approach can comprehensively enhance the understanding of video content and the explainability of the decision-making process. We first introduce two datasets, Ex-HateMM and Ex-ImpliHateVid, for explainable hateful video detection. Each dataset provides fine-grained annotations of multimodal harmful elements, along with contextual rationales. We then propose an Information Augmentation and Reasoning Enhancement (IARE) framework designed for explainable detection. The framework employs an information augmentation phase that leverages the multimodal chain-of-thought to integrate harmful elements, thereby enriching rationale evidence. Additionally, IARE incorporates a reasoning enhancement phase, in which Direct Preference Optimization guides the model toward correct reasoning paths and away from incorrect ones, thereby improving the logical coherence of its justifications. We conduct extensive experiments on the two datasets, comparing multiple baselines with our proposed IARE framework. The results demonstrate that IARE achieves state-of-the-art performance while also generating accurate rationales.

Generative retrieval enhances retrieval effectiveness by generating natural language represented document identifiers. However, current methods often struggle with two major challenges: limited identifier quality and insufficient query-document interaction, leading to limited retrieval performance. To tackle these challenges, we propose a novel generative retrieval framework integrated with semantic-aligned multi-layer identifiers and reinforcement learning. To improve identifier quality, we design a prompt-driven multi-task learning strategy to generate three types of hierarchical identifiers: summary, keyword, and pseudo-query, to capture multi-granularity document semantics. Furthermore, we adopt supervised fine-tuning to integrate these identifiers. To improve query-document interaction, we devise a multi-view ranking fusion mechanism that combines retrieval results across multi-layer identifiers. We further employ a GRPO-based reinforcement learning based on dense similarity rewards and a difficulty-aware negative sampling strategy to optimize the generated identifiers. Experiments on multiple benchmark datasets show that our framework significantly outperforms existing generative retrieval methods, offering a promising solution for building more effective and semantically aligned retrieval systems. The code for our model is publicly available at https://github.com/yicentian02/GRAM-RL.

Relation reasoning in temporal knowledge graph infers unknown or emerging relational dependencies from historical structured data. Traditional approaches face inherent limitations in capturing complex semantic correlations and structural patterns among relations. To tackle this problem, we propose the Relational Multi-path Enhancement network (RME), which primarily focuses on relation modeling to enrich relation representations through comprehensive multi-path analysis. RME consists of five key components: (1) Controlled random walk module creates multi-hop head-to-tail paths using an adaptive stopping rule that balances short- and long-term connections. (2) Shared path extraction module identifies both shared-head paths and shared-tail paths. (3) Time-decayed path encoding module processes these paths differently. (4) Gated information aggregation module combines path information to determine which parts matter most. (5) Attention decoding module makes the final prediction by focusing on the most relevant path features. Experiments on multiple TKG benchmark datasets demonstrate that RME outperforms the state-of-the-art methods in relation multi-path reasoning.

Current large vision-language models (VLMs) exhibit remarkable performance in basic video understanding tasks. However, existing VLMs are still limited to surface-level perception and lack fine-grained spatio-temporal understanding and combinatorial reasoning capabilities. Existing methods typically rely on expensive human annotations or subtitle extraction, yet they struggle to effectively model temporal relations between frames. This paper proposes a structured representation based on temporal triplets to address two major challenges in traditional approaches: temporal fragmentation and entity reference ambiguity. By modeling objects, attributes, and relationships within the video and incorporating temporal information, we convert semantic content from keyframes into a sequence of temporal triplets. This structured representation is then used as input for zero-shot video question answering (VideoQA). Experiments were conducted on four benchmark VideoQA datasets: NExT-QA, STAR, MSVD-QA, and MSRVTT-QA, showing that our method achieves competitive performance without requiring fine-tuning, validating its generality and effectiveness.

News recommender systems aim to provide personalized news reading experiences for users based on their reading history. Behavioral science studies suggest that screen-based news reading contains three successive steps: scanning, title reading, and then clicking. Adhering to these steps, we find that intra-news entity interest dominates the scanning stage, while the inter-news entity interest guides title reading and influences click decisions. Unfortunately, current methods overlook the unique utility of entities in news recommendation. To this end, we propose a novel method called IP2 to probe entity-guided reading interest at both intra- and inter-news levels. At the intra-news level, a Transformer-based entity encoder is devised to aggregate mentioned entities in the news title into one signature entity. Then, a signature entity-title contrastive pre-training is adopted to initialize entities with proper meanings using the news story context, which in the meantime facilitates us to probe for intra-news entity interest. As for the inter-news level, a dual tower user encoder is presented to capture inter-news reading interest from both the title meaning and entity sides. In addition to highlighting the contribution of inter-news entity guidance, a cross-tower attention link is adopted to calibrate title reading interest using inter-news entity interest, thus further aligning with real-world behavior. Extensive experiments on two real-world datasets demonstrate that our IP2 achieves state-of-the-art performance in news recommendation.

Hateful memes are prevalent on the Internet, raising the urgent need for effective detection.Given their implicit nature, incorporating rationales with background knowledge is crucial for enhancing model understanding.However, existing methods often suffer from limited quality of external rationales and misalignment with original meme information.These challenges hinder model comprehension, leading to reduced accuracy and explainability.To address these challenges, we propose a Multimodal Multi-agent Knowledge Enhanced (M2KE) framework for hateful meme detection.M2KE introduces a multi-agent rationale discovery mechanism to extract high-quality rationales relevant to meme content and an adaptive knowledge interaction mechanism to ensure alignment between original meme information and external rationales.Specifically, multi-agent rationale discovery mechanism improves the reliability of rationales by collaboratively verifying and refining them with multiple agents, supported by large language models (LLMs) due to their extensive knowledge.And adaptive knowledge interaction mechanism uses information entropy to dynamically balance the model's attention between original meme information and external rationales, preventing over-reliance on rationales and enabling a more comprehensive understanding.Experimental results on three datasets demonstrate that M2KE significantly outperforms existing models.Further analysis underscores the importance of effectively integrating accurate rationales to enhance model performance.Disclaimer: Samples in this paper may be considered offensive.

Session-based recommendation is gaining increasing attention due to its practical value in predicting the intents of anonymous users based on limited behaviors. Emerging efforts incorporate various side information to alleviate inherent data scarcity issues in this task, leading to impressive performance improvements. The core of side information-driven session-based recommendation is the discovery and utilization of diverse data. In this survey, we provide a comprehensive review of this task from a data-centric perspective. Specifically, this survey commences with a clear formulation of the task. This is followed by a detailed exploration of various benchmarks rich in side information that are pivotal for advancing research in this field. Afterwards, we delve into how different types of side information enhance the task, underscoring data characteristics and utility. Moreover, we discuss the usage of various side information, including data encoding, data injection, and involved techniques. A systematic review of research progress is then presented, with the taxonomy by the types of side information. Finally, we summarize the current limitations and present the future prospects of this vibrant topic.

Malevolence detection in dialogues aims to identify harmful or inappropriate utterances, significantly impacting dialogue quality and user satisfaction. Although existing studies have shown promising performance by modeling interaction patterns from dialogue history, various malevolence-invoking factors, such as fine-grained emotions, evolving topics and user profiles, are often overlooked. To comprehensively consider these factors, we propose a hypergraph fusion model by employing multi-view LLM-driven prompts for malevolence detection in dialogues. Our model integrates emotion context, topic context, user profile context and interaction context, utilizing hypergraphs to establish high-order contextual relationships from multi views for deducing malevolence-invoking semantics. Experimental results on two benchmark datasets demonstrate that our model achieves the state-of-the-art performance.

Session-based recommendation aims to predict intents of anonymous users based on their limited behaviors. Modeling user behaviors involves two distinct rationales: co-occurrence patterns reflected by item IDs, and fine-grained preferences represented by item modalities (e.g., text and images). However, existing methods typically entangle these causes, leading to their failure in achieving accurate and explainable recommendations. To this end, we propose a novel framework DIMO to disentangle the effects of ID and modality in the task. DIMO aims to disentangle these causes at both item and session levels. At the item level, we introduce a co-occurrence representation schema to explicitly incorporate co-occurrence patterns into ID representations. Simultaneously, DIMO aligns different modalities into a unified semantic space to represent them uniformly. At the session level, we present a multi-view self-supervised disentanglement, including proxy mechanism and counterfactual inference, to disentangle ID and modality effects without supervised signals. Leveraging these disentangled causes, DIMO provides recommendations via causal inference and further creates two templates for generating explanations. Extensive experiments on multiple real-world datasets demonstrate the consistent superiority of DIMO over existing methods. Further analysis also confirms DIMO's effectiveness in generating explanations.

Sequential recommendation is dedicated to offering items of interest for users based on their history behaviors. The attribute-opinion pairs, expressed by users in their reviews for items, provide the potentials to capture user preferences and item characteristics at a fine-grained level. To this end, we propose a novel framework FineRec that explores the attribute-opinion pairs of reviews to finely handle sequential recommendation. Specifically, we utilize a large language model to extract attribute-opinion pairs from reviews. For each attribute, a unique attribute-specific user-opinion-item graph is created, where corresponding opinions serve as the edges linking heterogeneous user and item nodes. Afterwards, we devise a diversity-aware convolution operation to aggregate information within the graphs, enabling attribute-specific user and item representation learning. Ultimately, we present an interaction-driven fusion mechanism to integrate attribute-specific user/item representations across all attributes for generating recommendations. Extensive experiments conducted on several real-world datasets demonstrate the superiority of our FineRec over existing state-ofthe-art methods. Further analysis also verifies the effectiveness of our fine-grained manner in handling the task.

Session-based recommendation is devoted to characterizing preferences of anonymous users based on short sessions. Existing methods mostly focus on mining limited item co-occurrence patterns exposed by item ID within sessions, while ignoring what attracts users to engage with certain items is rich multi-modal information displayed on pages. Generally, the multi-modal information can be classified into two categories: descriptive information (e.g., item images and description text) and numerical information (e.g., price). In this paper, we aim to improve session-based recommendation by modeling the above multi-modal information holistically. There are mainly three issues to reveal user intent from multi-modal information: (1) How to extract relevant semantics from heterogeneous descriptive information with different noise? (2) How to fuse these heterogeneous descriptive information to comprehensively infer user interests? (3) How to handle probabilistic influence of numerical information on user behaviors? To solve above issues, we propose a novel multi-modal session-based recommendation (MMSBR) that models both descriptive and numerical information under a unified framework. Specifically, a pseudo-modality contrastive learning is devised to enhance the representation learning of descriptive information. Afterwards, a hierarchical pivot transformer is presented to fuse heterogeneous descriptive information. Moreover, we represent numerical information with Gaussian distribution and design a Wasserstein self-attention to handle the probabilistic influence mode. Extensive experiments on three real-world datasets demonstrate the effectiveness of the proposed MMSBR. Further analysis also proves that our MMSBR can alleviate the cold-start problem in SBR effectively.

Session-based recommendation intends to predict next purchased items based on anonymous behavior sequences. Numerous economic studies have revealed that item price is a key factor influencing user purchase decisions. Unfortunately, existing methods for session-based recommendation only aim at capturing user interest preference, while ignoring user price preference. Actually, there are primarily two challenges preventing us from accessing price preference. First, the price preference is highly associated to various item features (i.e., category and brand), which asks us to mine price preference from heterogeneous information. Second, price preference and interest preference are interdependent and collectively determine user choice, necessitating that we jointly consider both price and interest preference for intent modeling. To handle above challenges, we propose a novel approach Bi-Preference Learning Heterogeneous Hypergraph Networks (BiPNet) for session-based recommendation. Specifically, the customized heterogeneous hypergraph networks with a triple-level convolution are devised to capture user price and interest preference from heterogeneous features of items. Besides, we develop a Bi-Preference Learning schema to explore mutual relations between price and interest preference and collectively learn these two preferences under the multi-task learning architecture. Extensive experiments on multiple public datasets confirm the superiority of BiPNet over competitive baselines. Additional research also supports the notion that the price is crucial for the task.

Memes have become the popular means of communication for Internet users worldwide. Understanding the Internet meme is one of the most tricky challenges in natural language processing (NLP) tasks due to its convenient non-standard writing and network vocabulary. Recently, many linguists suggested that memes contain rich metaphorical information. However, the existing researches ignore this key feature. Therefore, to incorporate informative metaphors into the meme analysis, we introduce a novel multimodal meme dataset called MET-Meme, which is rich in metaphorical features. It contains 10045 text-image pairs, with manual annotations of the metaphor occurrence, sentiment categories, intentions, and offensiveness degree. Moreover, we propose a range of strong baselines to demonstrate the importance of combining metaphorical features for meme sentiment analysis and semantic understanding tasks, respectively. MET-Meme, and its code are released publicly for research in \urlhttps://github.com/liaolianfoka/MET-Meme-A-Multi-modal-Meme-Dataset-Rich-in-Metaphors.

Session-based recommendation aims to predict items that an anonymous user would like to purchase based on her short behavior sequence. The current approaches towards session-based recommendation only focus on modeling users' interest preferences, while they all ignore a key attribute of an item, i.e., the price. Many marketing studies have shown that the price factor significantly influences users' behaviors and the purchase decisions of users are determined by both price and interest preferences simultaneously. However, it is nontrivial to incorporate price preferences for session-based recommendation. Firstly, it is hard to handle heterogeneous information from various features of items to capture users' price preferences. Secondly, it is difficult to model the complex relations between price and interest preferences in determining user choices.

Recommendation systems can help users process large amounts of information, and generative adversarial networks (GANs) show great potential in recommendation systems. In this paper, we propose a new GAN model to enhance the information flow within the generator based on the information flow between the original generator and discriminator. Our experimental results indicate that our model reduces the discrepancy between the generator and the discriminator. Both the generator and discriminator yield considerable performance improvements compared to other strong baselines. The improvements by [email protected] and MRR are significant, which can reach 30.98% and 30.17%, respectively.

Deep cross-modal retrieval methods have shown their competitiveness among different cross-modal retrieval algorithms. Generally, these methods require a large amount of training data. However, aggregating large amounts of data will incur huge privacy risks and high maintenance costs. Inspired by the recent success of federated learning, we propose the federated cross-modal retrieval (FedCMR), which learns the model with decentralized multi-modal data. Specifically, we first train the cross-modal retrieval model and learn the common space across multiple modalities in each client using its local data. Then, we jointly learn the common subspace of multiple clients on the trusted central server. Finally, each client updates the common subspace of the local model based on the aggregated common subspace on the server, so that all clients participated in the training can benefit from federated learning. Experiment results on four benchmark datasets demonstrate the effectiveness proposed method.

Deep multimodal clustering have shown their competitiveness among different multimodal clustering algorithms. Existing algorithms usually boost the multimodal clustering by exploring the common knowledge among multiple modalities, which underutilizes the uniqueness of multiple modalities. In this paper, we enhance the mining of modality-common knowledge by extracting the modality-unique knowledge of each modality simultaneously. Specifically, we first utilize autoencoders to extract the modality-common and modality-unique features of each modality respectively. Meanwhile, the cross reconstruction is used to build latent connections among different modalities, i.e., maintain the consistency of modality-common features of each modality as well as heightening the diversity of modality-unique features of each modality. After that, modality-common features are fused to cluster the multimodal data. Experimental results on several benchmark datasets demonstrate that the proposed method outperforms state-of-art works obviously.

Learning to rank utilizes machine learning methods to solve ranking problems by constructing ranking models in a supervised way, which needs fixed-length feature vectors of documents as inputs, and outputs the ranking models learned by iteratively reducing the pre-defined ranking loss. The document features are always extracted based on classic textual statistics, and different features contribute differently to ranking performance. Given that well-defined features would contribute more to the retrieval performance, we investigate the usage of autoencoders to enrich the feature representations of documents. Autoencoders, as basic building blocks of deep neural networks, have been successfully used in many text mining tasks for generating effective features. To enrich the feature space for learning to rank, we introduce supervision into the loss functions of autoencoders. Specifically, we first train a linear ranking model on the training data, and then incorporate the learned weights into the reconstruction costs of an autoencoder. Meanwhile, we accumulate the costs of documents for a given query with query-level constraints for producing more useful features. We evaluate the effectiveness of our model on three LETOR datasets, and show that our model can generate effective document features to improve the retrieval performance.

Pseudo relevance feedback, as an effective query expansion method, can significantly improve information retrieval performance. However, the method may negatively impact the retrieval performance when some irrelevant terms are used in the expanded query. Therefore, it is necessary to refine the expansion terms. Learning to rank methods have proven effective in information retrieval to solve ranking problems by ranking the most relevant documents at the top of the returned list, but few attempts have been made to employ learning to rank methods for term refinement in pseudo relevance feedback. This article proposes a novel framework to explore the feasibility of using learning to rank to optimize pseudo relevance feedback by means of reranking the candidate expansion terms. We investigate some learning approaches to choose the candidate terms and introduce some state‐of‐the‐art learning to rank methods to refine the expansion terms. In addition, we propose two term labeling strategies and examine the usefulness of various term features to optimize the framework. Experimental results with three TREC collections show that our framework can effectively improve retrieval performance.

Ranking plays an important role in information retrieval system. In recent years, a kind of research named 'learning to rank' becomes more and more popular, which applies machine learning technology to solve ranking problems. Lots of ranking models belonged to learning to rank have been proposed, such as Regression, RankNet, and ListNet. Inspired by this, we proposed a novel learning to rank algorithm named GPQ in this paper, in which genetic programming was employed to directly optimize Q-measure evaluation metric. Experimental results on OHSUMED benchmark dataset indicated that our method GPQ could be competitive with Ranking SVM, SVMMAP and ListNet, and improve the ranking accuracies.