As search engines increasingly adopt answer-centric interfaces, AI-generated summaries are often consumed as final answers. These summaries are typically well-structured and citation-rich, creating a strong appearance of reliability that can encourage user trust, even when evidential grounding is uncertain. Motivated by this gap between appearance and evidence, we analyze 14,175 real-world queries from MS MARCO by examining Google Search AI summaries and their cited sources. We find that reliable-looking summaries mask substantial evidential failures. Despite their credible appearance, 32.31% of summaries are incorrect, and 56.16% of these errors arise even when supporting evidence exists in the cited sources. Moreover, 31.08% of summaries exhibit citation inconsistencies, with conflict risk increasing as more sources are cited and as citations appear in more prominent positions. Our findings reveal systematic, user-facing risks in answer-centric search, highlighting the need for evaluation and design practices beyond surface-level reliability signals. https://github.com/icip-cas/AISummary.

The widespread adoption of Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) systems is reshaping the landscape of information retrieval. However, the long-term effects of LLM-generated texts on retrieval systems remain underexplored, creating challenges for mitigating their impact. The effects are examined in this study, with a particular focus on the “Spiral of Silence” phenomenon, which refers to the marginalization of diverse information as certain types of content dominate, leading to a homogenized information ecosystem. To investigate this, a simulation pipeline is constructed to model the iterative introduction of LLM-generated texts into retrieval systems. Experimental results across multiple iterations reveal that as the presence of LLM-generated texts within the system grows, retrieval systems exhibit a stronger tendency to retrieve these texts. This trend, in turn, reduces the visibility of human-generated content, diminishes diversity, propagates errors, and results in a notable decline in retrieval performance. To address these challenges, we propose a Utility-Driven Multi-Objective Optimization (UMO) framework to effectively mitigate the “Spiral of Silence.” This framework employs a two-phase approach: an optimization phase, leveraging the NSGA-II algorithm to derive optimal preference weights for multiple objectives, and a memorization phase, which directly integrates these weights into the retrieval vector space without requiring additional model retraining. Experimental results demonstrate that this framework maintains stable retrieval effectiveness, improves the retrieval proportion of human-generated content, reduces the excessive influence of LLM-generated texts, and preserves information diversity, effectively mitigating the “Spiral of Silence.”

The proliferation of complex non-factoid questions in modern information seeking (IS) systems exposes critical limitations in conventional Retrieval-Augmented Generation (RAG) approaches, particularly their static search strategies and the lack of systematic multi-source information integration capabilities. Facing these limitations, we present PASS (Proactive Agent-driven Search System), a novel multi-agent framework that operationalizes human-like proactive search strategies through five specialized agents: Revealer for intent analysis, Navigator for search planning, Seeker/Reader for adaptive retrieval, and Writer for response synthesis, systematically expanding the search space through iterative query refinement and multi-perspective knowledge integration. Crucially, our framework demonstrates remarkable adaptability to mid-sized LLMs, demonstrating its scalability in resource-constrained environments. To comprehensively assess the effectiveness of the proposed framework, we carry out extensive experiments on both mid-sized and proprietary large-scale LLMs, evaluating response quality for complex non-factoid questions using a newly introduced nugget-based assessment. Experimental results from offline nugget-based evaluation and online A/B Tests confirm substantial improvements in answer quality, advancing proactive information seeking methodologies and offering practical pathways for democratizing complex reasoning capabilities to resource-constrained environments.

Knowledge graph completion (KGC) aims to infer missing knowledge triples based on known facts in a knowledge graph. Current KGC research mostly follows an entity ranking protocol, wherein the effectiveness is measured by the predicted rank of a masked entity in a test triple. The overall performance is then given by a micro(-average) metric over all individual answer entities. Due to the incomplete nature of the large-scale knowledge bases, such an entity ranking setting is likely affected by unlabelled top-ranked positive examples, raising questions on whether the current evaluation protocol is sufficient to guarantee a fair comparison of KGC systems. To this end, this paper presents a systematic study on whether and how the label sparsity affects the current KGC evaluation with the popular micro metrics. Specifically, inspired by the TREC paradigm for large-scale information retrieval (IR) experimentation, we create a relatively "complete" judgment set based on a sample from the popular FB15k-237 dataset following the TREC pooling method. According to our analysis, it comes as a surprise that switching from the original labels to our "complete" labels results in a drastic change of system ranking of a variety of 13 popular KGC models in terms of micro metrics. Further investigation indicates that the IR-like macro(-average) metrics are more stable and discriminative under different settings, meanwhile, less affected by label sparsity. Thus, for KGC evaluation, we recommend conducting TREC-style pooling to balance between human efforts and label completeness, and reporting also the IR-like macro metrics to reflect the ranking nature of the KGC task.

Online search latency is a major bottleneck in deploying large-scale pre-trained language models, e.g. BERT, in retrieval applications. Inspired by the recent advances in transformer-based document expansion technique, we propose to trade offline relevance weighting for online retrieval efficiency by utilizing the powerful BERT ranker to weight the neighbour documents collected by generated pseudo-queries for each document. In the online retrieval stage, the traditional query-document matching is reduced to the much less expensive query to pseudo-query matching, and a document rank list is quickly recalled according to the pre-computed neighbour documents. Extensive experiments on the standard MS MARCO dataset with both passage and document ranking tasks demonstrate promising results of our method in terms of both online efficiency and effectiveness.

Entity Resolution (ER) identifies records from different data sources that refer to the same real-world entity. Conventional ER approaches usually employ a structure matching mechanism, where attributes are aligned, compared and aggregated for ER decision. The structure matching approaches, unfortunately, often suffer from heterogeneous and dirty ER problems. That is, entities from different data sources are described using different schemas, and attribute values may be misplaced, missing, or noisy. In this paper, we propose a deep sequence-to-sequence entity matching model, denoted Seq2SeqMatcher, which can effectively solve the heterogeneous and dirty problems by modeling ER as a token-level sequence-to-sequence matching task. Specifically, we propose an align-compare-aggregate neural network for Seq2Seq entity matching, which can learn the representations of tokens, capture the semantic relevance between tokens, and aggregate matching evidence for accurate ER decisions in an end-to-end manner. Experimental results show that, by comparing entity records in token level and learning all components in an end-to-end manner, our Seq2Seq entity matching model can achieve remarkable performance improvements on 9 standard entity resolution benchmarks.

To accomplish a search task and satisfy a single information need, users usually submit a series of queries to web search engines. It is useful for web search engines to detect the task boundaries in a series of successive queries. Traditional task boundary detection methods are based on time gap and lexical comparisons, which often suffer from the vocabulary gap problem, that is, the topically related queries may not share any common words. In this paper we learn hidden topics from query log and leverage them to resolve the vocabulary gap problem. Unlike other external knowledge resources, such as WordNet and Wikipedia, the hidden topics discovered from query log cover long tail queries, which is useful to detect task boundaries. Experimental results on dataset from real world query log demonstrate that the proposed method achieves significant quality enhancement.

Entity Linking (EL) is the task of linking name mentions in Web text with their referent entities in a knowledge base. Traditional EL methods usually link name mentions in a document by assuming them to be independent. However, there is often additional interdependence between different EL decisions, i.e., the entities in the same document should be semantically related to each other. In these cases, Collective Entity Linking, in which the name mentions in the same document are linked jointly by exploiting the interdependence between them, can improve the entity linking accuracy.

Pseudo relevance feedback method is an effective method for query model refinement. Most existing pseudo relevance feedback methods only take into consideration the term distribution of the feedback documents, but omit the term's context information. This paper presents a graph-based method to improve query models, in which a word graph is constructed to encode terms and their co-occurrence dependencies within the feedback documents. Using a random walk, the weight of each term in the graph can be determined in a context-dependent manner, i.e. the weight of a term is strongly dependent on the weights of the connected context terms. Our experimental results on four TREC collections show that our proposed approach is more effective than the existing state-of-the-art approaches.

The text classification methods heavily depend on machine learning algorithms with abstract mathematic metrics, which obstruct the direct observation and intuitive understanding of the text-specific classification. In this paper, we model a document as a Document-Classes-Topics top-down hierarchical structure. Furthermore, by running the document generation procedure, we can obtain each class's content share, which not only can be used to make the classification decision but also can provide a natural visualization approach for text classification. We implement this idea by a new tool named TC-DCA, which provides the visualization of text classification result, where the target document is expressed graphically as its content's allocation on every class. TC-DCA can also perform the drilling down operation to reveal the classification effect of each word of the document.

Smoothing document model with word graph is a new and effective method in information retrieval. Word graph can naturally incorporate the dependency between the words; random walk algorithm based on the graph can be used to estimate the weight of each vertex. In this paper, we present a new way to construct a local word graph for smoothing document model, which exploits the document's k nearest neighbors: the vertices represent the words in the document and its k nearest neighbors, and the weights of the edges are estimated through word co-occurrence in the local document set. We argue that word graph is a key factor to the performance in graph-based smoothing method. By using the local document set, we can obtain a document specific word graph, and achieve better retrieval performance. Experimental results on three TREC collections show that our proposed approach is effective.