Retrieval-Augmented Generation (RAG) has emerged as an effective paradigm for enhancing large language models (LLMs) with external knowledge, delivering substantial performance gains without costly parameter updates. However, for beginners, RAG remains difficult to approach due to its conceptual complexity, computational requirements, and non-trivial system design choices. Motivated by this observation, we propose EasyRAG, a beginner-friendly and interactive framework designed to lower the entry barrier to understanding and experimenting with RAG systems. EasyRAG provides multi-level demonstrations of RAG, progressively supporting different depths of conceptual and practical understanding. At the first level, EasyRAG adopts a Search-API-based RAG setup, emphasizing ease of deployment. This level avoids large-scale dataset preprocessing and complex retriever configurations, enabling users to quickly grasp the fundamental differences between LLM-only generation and naive RAG. At the second level, EasyRAG implements three representative RAG methods (i.e., FiD, FiD-Light, and Stochastic RAG), which are strongly related, with each method extending the previous one. Through these implementations, users can gain deeper insights into key RAG properties, including computational efficiency trade-offs, the impact of end-to-end optimization, and source attribution. At the third level, EasyRAG supports systematic comparison of different RAG approaches on standard benchmark datasets. In addition, we provide an interactive interface that exposes fine-grained details of both training and inference, allowing users to closely examine and understand each step of the RAG pipeline. Overall, EasyRAG provides a unified platform for learning, experimenting with, and analyzing RAG systems, making it particularly suitable for beginners in RAG. The source code and a video demonstration are available at https://github.com/ii-research/EasyRAG.

The recent studies show that Large Language Models (LLMs) often fall short in tasks demanding creative, lateral thinking due to lacking a clear awareness of their own reasoning processes. To cope with this issue, we propose a novel metacognitive prompting method (titled as MP) by mimicking human metacognition. Through integrating metacognitive principles, MP endows LLMs with lateral thinking ability, thereby enhancing their abilities to strategize, monitor, and reflect on their responses when dealing with creative tasks. The experimental results with five base LLMs across three lateral thinking datasets demonstrate that: All LLMs armed with MP consistently outperform the representative baseline methods. For example, MP demonstrates superior performance over CoT prompting across Sentence Puzzle (+5.00%), Word Puzzle (+10.07%), BiRdQA (+6.48%), and RiddleSense (+2.65%) with GPT-3.5-turbo model. In particular, the deployment of MP with GPT-4 achieves significant performance improvements that even surpass human performance on BRAINTEASER benchmark, demonstrating the transformative potential of MP in enhancing the creative problem-solving abilities of LLMs.

Abstract Commonsense explanation generation refers to reasoning and explaining why a commonsense statement contradicts commonsense knowledge, such as why the statement “My dad grew volleyballs in his garden” is nonsensical. While such reasoning is trivial for humans, it remains a challenge for AI systems. Despite their notable performance in tasks like text generation and reasoning, large language models (LLMs) often fall short of consistently generating coherent and accurate commonsense explanations. To bridge this gap, we propose a novel Two-stage Identification and Prompting (TIP) framework for enhancing LLMs’ ability to handle the task of commonsense explanation generation. Specifically, in the first stage, TIP identifies the nonsensical concept in the given statement, pinpointing the specific element that contradicts commonsense knowledge. In the second stage, TIP generates implicit knowledge based on the identified nonsensical concept and then leverages this implicit knowledge to guide the adopted LLMs in generating explanations. In order to demonstrate the effectiveness of the proposed TIP framework for commonsense explanation generation, we conducted extensive experiments based on the ComVE dataset and a newly constructed CSE dataset, where a variety of LLMs are evaluated. The experimental results show that TIP consistently outperforms all baseline methods across multiple metrics, demonstrating its effectiveness in improving LLMs’ commonsense reasoning and explanation generation capabilities.

Graph is the universal language for modeling data across various domains. In recent years, graph representation learning has achieved outstanding performance in a series of computational tasks on graphs, such as graph node classification and community detection, etc. However, a commonly hidden assumption underlying these tasks is that label information corresponds to nodes in the training set on a one-to-one basis. In real-world scenarios, node label information may be uncertain and concealed within higher-order graph structural labels. In this paper, we define a set of arbitrary nodes on the graph as a bag and assume that label information corresponds to bags rather than individual nodes. Furthermore, bag labels are generated based on node-level labels through some hidden mechanism, thus inexactly encompassing node label information1. Therefore, we propose for the first time a novel and widely applicable task: learning the latent representation of bags on the graph and predicting their labels. For this task, we propose a hierarchical model that is highly interpretable and scalable, incorporating information propagation among nodes, information aggregation from nodes to bags and inter-bag relationship prediction. Experiments on diverse standard datasets demonstrate that our proposed model shows higher classification accuracy compared to strong baselines. Our research introduces a new perspective to the field of graph representation learning.

Facial Expression Recognition (FER) poses significant challenges due to various imaging conditions, including diverse head poses, lighting conditions, resolutions, and occlusions. Additionally, different personal attributes such as age, gender, and racial background further contribute to the complexity of FER. To accurately extract meaningful expression features amidst these interfering factors to enhance recognition accuracy and the model’s generalization, we propose a Self Decoupling-Reconstruction Network (SDRNet). Specifically, our approach involves two learning processes. In the first phase, the network is trained to decouple facial images with expressions into expression and neutral components. This process involves reconstructing neutral facial images and the original input, ensuring the preservation of meaningful expression components devoid of interference in the decoupling process. In the second learning phase, we employ simple convolutional neural networks (CNNs) to recognize the extracted expression components. Our method has achieved state-of-the-art results across multiple widely used datasets, providing substantial evidence of its effectiveness. Additionally, we demonstrate the robust generalization performance of our approach through cross-database evaluations.

Action-related KnowledGe (AKG) is important for facilitating deeper understanding of people’s life patterns, objectives and motivations. In this study, we present a novel framework for automatically predicting missing human biography records in Wikipedia by generating such knowledge. The generation method, which is based on a neural network matrix factorization model, is capable of encoding action semantics from diverse perspectives and discovering latent inter-action relations. By correctly predicting missing information and correcting errors, our work can effectively improve the quality of data about the behavioral records of historical figures in the knowledge base (e.g., biographies in Wikipedia), thus contributing to the understanding and study of human actions by the general public on the one hand, and can be considered as a new paradigm for managing action-related knowledge in digital libraries on the other. Extensive experiments demonstrate that the AKG we generate can capture well missing or “forgotten” human biography related information in Wikipedia.

Most of the existing methods for search result diversification (SRD) appeal to the greedy strategy for generating diversified results, which is formulated as a sequential process of selecting documents one-by-one, and the locally optimal choice is made at each round. Unfortunately, this strategy suffers from the following shortcomings: (1) Such a one-by-one selection process is rather time-consuming for both training and inference. (2) It works well on the premise that the preceding choices are optimal or close to the optimal solution. (3) The mismatch between the objective function used in training and the final evaluation measure used in testing has not been taken into account. We propose a novel framework through direct metric optimization for SRD (referred to as MO4SRD) based on the score-and-sort strategy. Specifically, we represent the diversity score of each document that determines its rank position based on a probability distribution. These distributions over scores naturally give rise to expectations over rank positions. Armed with this advantage, we can get the differentiable variants of the widely used diversity metrics. Thanks to this, we are able to directly optimize the evaluation measure used in testing. Moreover, we have devised a novel probabilistic neural scoring function. It jointly scores candidate documents by taking into account both cross-document interaction and permutation equivariance, which makes it possible to generate a diversified ranking via a simple sorting. The experimental results on benchmark collections show that the proposed method achieves significantly improved performance over the state-of-the-art results.

Background articles are crucial for readers to grasp the context of news stories fully. However, existing approaches of background article search tend to apply a single ranking method to all types of search topics. In this paper, we focus on exploring search topics on news articles by classifying them into two types:time-sensitive andnon-time-sensitive. To verify whether or not these two types of search topics can benefit from different retrieving methods, we examined a suite of strategies such as document expansion, query rewriting, and semantic re-ranking. Moreover, the relationship between background articles and topics is verified by the two strategies of document expansion (specificity and diversity). The experimental results demonstrate that the optimal usage of the aforementioned strategies is indeed different between the two types of search topics. Furthermore, our in-depth analysis of topics and search results verified that: time-sensitive topics benefit from background articles that can provide more specific knowledge, while non-time-sensitive topics benefit from diversified retrieved documents.

In graph node classification tasks, traditional graph neural network (GNN) models assume that different types of misclassification have equal loss and thus seek to maximize the posterior probability of sample nodes under labeled classes. However, the graph data in realistic scenarios tend to follow unbalanced long-tail class distributions, making it difficult for GNN to accurately represent the minority class samples because of the overfitting tendency to the majority class features. To address this problem, in this paper we propose a novel GNN model, named Dual Cost-sensitive Graph Convolutional Network (DCSGCN). The DCSGCN is a two-tower model containing two sub-networks that compute the posterior probability and the misclassification cost separately. It uses the cost as complementary information in classification to correct the posterior probability under the minimal risk perspective. Furthermore, we propose a series of new methods to compute node cost labels based on graph topological information and node class distribution. Extensive experiments lead to the observation that DCSGCN outperforms the state-of-the-art model on diverse real-world imbalanced graphs.

Deep metric learning has shown significantly increasing values in a wide range of domains, such as image retrieval, face recognition, zero-shot learning, to name a few. When evaluating the methods for deep metric learning, top-k precision is commonly used as a key metric, since few users bother to scroll down to lower-ranked items. Despite being widely studied, how to directly optimize top-k precision is still an open problem. In this paper, we propose a new method on how to optimize top-k precision in a rank-sensitive manner. Given the cutoff value k, our key idea is to impose different weights to further differentiate misplaced images sampled according to the top-k precision. To validate the effectiveness of the proposed method, we conduct a series of experiments on three widely used benchmark datasets. The experimental results demonstrate that: (1) Our proposed method outperforms the baseline methods on two datasets, which shows the potential value of rank-sensitive optimization of top-k precision for deep metric learning. (2) The factors, such as batch size and cutoff value k, significantly affect the performance of approaches that rely on optimising top-k precision for deep metric learning. Careful examinations of these factors are highly recommended.

Query logs contain rich feedback information from a large number of users interacting with search engines. Various click models have been developed to decode users' search behavior and to extract useful knowledge from query logs. Although the state-of-the-art neural click models have been shown to be very effective in click modeling, the input representations of queries and documents rely on either manually crafted features or on automatic methods suffering from the high-dimensionality issue. Moreover, these neural click models are still rather restrictive when coping with commonly biased user clicks. In this paper, we investigate how to effectively deploy a neural network model for decoding users' click behavior. First, we present two novel rank-biased neural network models ($RBNN$ and $RBNN^* $) for click modeling. The key idea is to deploy different weight matrices across different rank positions. Second, we introduce a new method ($QD\mymathhyphen DCCA$) for automatically learning the vector representations for both queries and documents within the same low-dimensional space, which provides high-quality inputs for $RBNN$ and $RBNN^* $. Finally, a series of experiments are conducted on two different real query logs to validate the effectiveness and efficiency of the proposed neural click models. The experiments demonstrate that: (1) The proposed models can achieve substantially improved performance over the state-of-the-art baseline on two datasets across multiple metrics. By incorporating rank-specific weight matrices, $RBNN$ and $RBNN^* $ are more capable of dealing with the position-bias problem. (2) The input representations of queries, documents and context information significantly affect the performance of neural click models. Thanks to the application of $QD\mymathhyphen DCCA$, not only $RBNN$ and $RBNN^* $ but also the baseline method exhibit enhanced performance. Furthermore, the training cost under the proposed models is greatly reduced.

Learning to rank has been intensively studied and has shown great value in many fields, such as web search, question answering and recommender systems. This paper focuses on listwise document ranking, where all documents associated with the same query in the training data are used as the input. We propose a novel ranking method, referred to as WassRank, under which the problem of listwise document ranking boils down to the task of learning the optimal ranking function that achieves the minimum Wasserstein distance. Specifically, given the query level predictions and the ground truth labels, we first map them into two probability vectors. Analogous to the optimal transport problem, we view each probability vector as a pile of relevance mass with peaks indicating higher relevance. The listwise ranking loss is formulated as the minimum cost (the Wasserstein distance) of transporting (or reshaping) the pile of predicted relevance mass so that it matches the pile of ground-truth relevance mass. The smaller the Wasserstein distance is, the closer the prediction gets to the ground-truth. To better capture the inherent relevance-based order information among documents with different relevance labels and lower the variance of predictions for documents with the same relevance label, ranking-specific cost matrix is imposed. To validate the effectiveness of WassRank, we conduct a series of experiments on two benchmark collections. The experimental results demonstrate that: compared with four non-trivial listwise ranking methods (i.e., LambdaRank, ListNet, ListMLE and ApxNDCG), WassRank can achieve substantially improved performance in terms of nDCG and ERR across different rank positions. Specifically, the maximum improvements of WassRank over LambdaRank, ListNet, ListMLE and ApxNDCG in terms of [email protected] are 15%, 5%, 7%, 5%, respectively.

Several previous approaches attempted to predict bursty topics on Twitter. Such approaches have usually reported that the time information (e.g. the topic popularity over time) of hashtag topics contribute the most to the prediction of bursty topics. In this paper, we propose a novel approach to use time features to predict bursty topics on Twitter. We model the popularity of topics as density curves described by the density function of a beta distribution with different parameters. We then propose various approaches to predict/classify the bursty topics by estimating the parameters of topics, using estimators such as Gradient Decent or Likelihood Maximization. In our experiments, we show that the estimated parameters of topics have a positive effect on classifying bursty topics. In particular, our estimators when combined together improve the bursty topic classification by 6.9 in terms of micro F1 compared to a baseline classifier using hashtag content features.

Given the proliferation of social media and the abundance of news feeds, a substantial amount of real-time content is distributed through disparate sources, which makes it increasingly difficult to glean and distill useful information. Although combining heterogeneous sources for topic detection has gained attention from several research communities, most of them fail to consider the interaction among different sources and their intertwined temporal dynamics. To address this concern, we studied the dynamics of topics from heterogeneous sources by exploiting both their individual properties (including temporal features) and their inter-relationships. We first implemented a heterogeneous topic model that enables topic–topic correspondence between the sources by iteratively updating its topic–word distribution. To capture temporal dynamics, the topics are then correlated with a time-dependent function that can characterise its social response and popularity over time. We extensively evaluate the proposed approach and compare to the state-of-the-art techniques on heterogeneous collection. Experimental results demonstrate that our approach can significantly outperform the existing ones.

Feature-based matrix factorization techniques such as Factorization Machines (FM) have been proven to achieve impressive accuracy for the rating prediction task. However, most common recommendation scenarios are formulated as a top-N item ranking problem with implicit feedback (e.g., clicks, purchases)rather than explicit ratings. To address this problem, with both implicit feedback and feature information, we propose a feature-based collaborative boosting recommender called BoostFM, which integrates boosting into factorization models during the process of item ranking. Specifically, BoostFM is an adaptive boosting framework that linearly combines multiple homogeneous component recommenders, which are repeatedly constructed on the basis of the individual FM model by a re-weighting scheme. Two ways are proposed to efficiently train the component recommenders from the perspectives of both pairwise and listwise Learning-to-Rank (L2R). The properties of our proposed method are empirically studied on three real-world datasets. The experimental results show that BoostFM outperforms a number of state-of-the-art approaches for top-N recommendation.

To cope with ambiguous and/or underspecified queries, search result diversification (SRD) is a key technique that has attracted a lot of attention. This paper focuses on implicit SRD, where the possible subtopics underlying a query are unknown beforehand. We formulate implicit SRD as a process of selecting and ranking k exemplar documents that utilizes integer linear programming (ILP). Unlike the common practice of relying on approximate methods, this formulation enables us to obtain the optimal solution of the objective function. Based on four benchmark collections, our extensive empirical experiments reveal that: (1) The factors, such as different initial runs, the number of input documents, query types and the ways of computing document similarity significantly affect the performance of diversification models. Careful examinations of these factors are highly recommended in the development of implicit SRD methods. (2) The proposed method can achieve substantially improved performance over the state-of-the-art unsupervised methods for implicit SRD.

In the age of Web 2.0, a substantial amount of unstructured content are distributed through multiple text streams in an asynchronous fashion, which makes it increasingly difficult to glean and distill useful information. An effective way to explore the information in text streams is topic modelling, which can further facilitate other applications such as search, information browsing, and pattern mining. In this paper, we propose a semantic graph based topic modelling approach for structuring asynchronous text streams. Our model integrates topic mining and time synchronization, two core modules for addressing the problem, into a unified model. Specifically, for handling the lexical gap issues, we use global semantic graphs of each timestamp for capturing the hidden interaction among entities from all the text streams. For dealing with the sources asynchronism problem, local semantic graphs are employed to discover similar topics of different entities that can be potentially separated by time gaps. Our experiment on two real-world datasets shows that the proposed model significantly outperforms the existing ones.

Community Question Answering (CQA) services, such as Yahoo! Answers and WikiAnswers, have become popular with users as one of the central paradigms for satisfying users’ information needs. The task of question retrieval aims to answer one's query directly by finding the most relevant questions (together with their answers) from an archive of past questions. However, questions are always short text that there is a lexical gap between the queried question and the past questions. Furthermore, the underlying intents of two questions could be very different even if they bear a close lexical resemblance. To alleviate these problems, we present a hybrid approach that blends several language modelling techniques for question retrieval, namely, the Classic (query-likelihood) Language Model, the state-of-the-art Translation-based Language Model, and our proposed Semantic-based Language Model and Intent-based Language Model. The semantics of each candidate question is derived by a Probabilistic Topic Model, which makes use of local and global semantic graphs for capturing the hidden interactions among entities (e.g. people, places and concepts) in question–answer pairs. Experiments on two real-world data sets show that our approach can significantly outperform existing ones.

State-of-the-art item recommendation algorithms, which apply Factorization Machines (FM) as a scoring function and pairwise ranking loss as a trainer (PRFM for short), have been recently investigated for the implicit feedback based context-aware recommendation problem (IFCAR). However, good recommenders particularly emphasize on the accuracy near the top of the ranked list, and typical pairwise loss functions might not match well with such a requirement. In this paper, we demonstrate, both theoretically and empirically, PRFM models usually lead to non-optimal item recommendation results due to such a mismatch. Inspired by the success of LambdaRank, we introduce Lambda Factorization Machines (LambdaFM), which is particularly intended for optimizing ranking performance for IFCAR. We also point out that the original lambda function suffers from the issue of expensive computational complexity in such settings due to a large amount of unobserved feedback. Hence, instead of directly adopting the original lambda strategy, we create three effective lambda surrogates by conducting a theoretical analysis for lambda from the top-N optimization perspective. Further, we prove that the proposed lambda surrogates are generic and applicable to a large set of pairwise ranking loss functions. Experimental results demonstrate LambdaFM significantly outperforms state-of-the-art algorithms on three real-world datasets in terms of four standard ranking measures.

Recommending users with preferred point-of-interests (POIs) has become an important task for location-based social networks, which facilitates users' urban exploration by helping them filter out unattractive locations. Although the influence of geographical neighborhood has been studied in the rating prediction task (i.e. regression), few work have exploited it to develop a ranking-oriented objective function to improve top-N item recommendations. To solve this task, we conduct a manual inspection on real-world datasets, and find that each individual's traits are likely to cluster around multiple centers. Hence, we propose a co-pairwise ranking model based on the assumption that users prefer to assign higher ranks to the POIs near previously rated ones. The proposed method can learn preference ordering from non-observed rating pairs, and thus can alleviate the sparsity problem of matrix factorization. Evaluation on two publicly available datasets shows that our method performs significantly better than state-of-the-art techniques for the top-N item recommendation task.

Research on temporal aspects of information retrieval has recently gained considerable interest within the Information Retrieval (IR) community. This paper describes our efforts for building test collections for the purpose of fostering temporal IR research. In particular, we overview the test collections created at the two recent editions of Temporal Information Access (Temporalia) task organized at NTCIR-11 and NTCIR-12, report on selected results and discuss several observations we made during the task design and implementation. Finally, we outline further directions for constructing test collections suitable for temporal IR.

Community Question Answering (CQA) services, such as Yahoo! Answers and WikiAnswers, have become popular with users as one of the central paradigms for satisfying users' information needs. The task of question retrieval aims to resolve one's query directly by finding the most relevant questions (together with their answers) from an archive of past questions. However, as the text of each question is short, there is usually a lexical gap between the queried question and the past questions. To alleviate this problem, we present a hybrid approach that blends several language modelling techniques for question retrieval, namely, the classic (query-likelihood) language model, the state-of-the-art translation-based language model, and our proposed semantics-based language model. The semantics of each candidate question is given by a probabilistic topic model which makes use of local and global semantic graphs for capturing the hidden interactions among entities (e.g., people, places, and concepts) in question-answer pairs. Experiments on two real-world datasets show that our approach can significantly outperform existing ones.

Result diversification is a topic of great value for enhancing user experience in many fields, such as web search and recommender systems. Many existing methods generate a diversified result in a sequential manner, but they work well only if the preceding choices are optimal or close to the optimal solution. Moreover, a manually tuned parameter (say,λ) is often required to trade off relevance and diversity. This makes it difficult to know whether the failures are caused by the optimization criterion or the setting of λ. In context of web search, we formulate the result diversification task as a 0-1 multiple subtopic knapsack problem (MSKP), where a subset of documents are optimally chosen like filling up multiple subtopic knapsacks. This formulation yields no trade-off parameters to be specified beforehand. Solving the 0-1 MSKP is NP-hard, we treat the optimization of 0-1 MSKP using a graphical model over latent binary variables as a maximum posterior inference problem, and tackle it with the max-sum belief propagation algorithm. To validate the effectiveness and efficiency of the proposed 0-1 MSKP model, we conduct a series of experiments on two TREC diversity collections. The experimental results show that the proposed model outperforms several state-of-the-art methods significantly, not only in terms of standard diversity metrics (α-nDCG, nERRIA and subtopic recall), but also in terms of efficiency.

Understanding the information need or intent encoded within a query has long been regarded as an essential factor of effective information retrieval. For better query representation and understanding, two intent roles (kernel-object and modifier) are introduced to structurally parse a class of role-explicit queries, which constitute a majority of common user queries. Furthermore, we focus on two research problems: RP-1: Given a role-explicit query, how to identify the kernel-object and modifier, namely intent role annotation; RP-2: How to determine whether an arbitrary query is role-explicit or not. To solve RP-1, we propose a simplified word n-gram role model (SWNR), which quantifies the generating probability of a role-explicit query and performs intent role annotation effectively. Using a set of discriminative features, we build classifiers to address RP-2 in a supervised manner. The experimental results show that: (1) SWNR can achieve a satisfactory performance, more than 73% in terms of different metrics; (2) The classifiers can achieve more than 90% precision in identifying role-explicit queries; (3) Compared with traditional techniques for query representation and understanding, e.g., name entity recognition in query and class-level query intent inference, intent role annotation provides a more flexible framework and a number of applications can benefit from annotating role-explicit queries, such as intent mining and diversified document ranking.