As a core module of web search, query understanding aims to bridge the semantic gap between user queries and web page documents, thereby enhancing the ability to deliver more relevant results.Recently, Large Language Models (LLMs) have achieved significant breakthroughs that have fundamentally altered the workflow of existing search ranking tasks.However, few researchers have explored the integration of LLMs into the field of query understanding.In this paper, we investigate the potential of LLMs in query understanding by conducting a comprehensive evaluation across three dimensions: term, structure, and topic.This evaluation includes several representative tasks such as segmentation, term weighting, error correction, query expansion, and intent recognition.The experimental results reveal that LLMs are particularly effective in query expansion and intent recognition but show limited improvement in other areas.This limitation may be attributed to LLMs' primary focus on modeling the semantic knowledge of entire queries, while lacking the capability to capture token-level information with finer granularity.Additionally, we explore potential practical applications of LLMs in query understanding, such as integrating the evaluation and training capabilities of smaller models with LLMs and constructing unsupervised samples.Based on comprehensive empirical results, collaborative training emerges as a promising approach to leverage LLMs for query understanding.We hope this research will advance the practical application of LLMs in query understanding and contribute to the development of this field.

Relevance plays a crucial role in commercial search engines by identifying documents related to user queries and fulfilling their search needs.Traditional approaches employ encoder-only models like BERT, which process concatenated query-document pairs to predict relevance scores.While autoregressive large language models (LLMs) have revolutionized numerous NLP domains, their direct application to web-scale search systems presents significant challenges.On one hand, the relevance modeling capabilities of LLMs have not been fully explored.On the other, the high computational costs and inference times make deploying LLMs in online search systems, which demand extremely low latency, nearly impossible.In this work, we address these challenges through two key contributions.First, we develop a comprehensive evaluation framework to systematically assess the effectiveness of LLMs in query-document relevance ranking.By conducting assessment experiments to LLMs in four perspectives: ranking objectives, model size, domain-specific continuous pre-training, and the integration of prior knowledge, we identify the best resource allocation strategy given a restricted budget and develop practical LLMs in a more efficient way.Second, we propose a novel framework to transfer the capabilities of LLMs in the ranking aspect to existing BERT models to avoid directly deploying LLMs.Finally, to fully leverage the improvements in relevance ranking brought by LLMs, we successfully nearline deploy LLMs in Tencent QQ Browser search engine using query-based ondemand computing and quantization.Experiments on real-world datasets and online A/B tests demonstrate that our approach significantly enhances search engine performance while maintaining practical operational efficiency.Our findings provide actionable insights for integrating LLMs into production search engines.

This paper addresses the challenge of the semantic gap between user queries and web content, commonly referred to as asymmetric text matching, within the domain of web search.By leveraging BERT for reading comprehension, current algorithms enable significant advancements in query understanding, but still encounter limitations in effectively resolving the asymmetrical ranking problem due to model comprehension and summarization constraints.To tackle this issue, we propose the QAGR (Question-Answer Generation and Ranking) method, comprising an offline module called QAGeneration and an online module called QARanking.The QAGeneration module utilizes large language models (LLMs) to generate high-quality question-answering pairs for each web page.This process involves two steps: generating question-answer pairs and performing verification to eliminate irrelevant questions, resulting in high-quality questions associated with their respective documents.The QARanking module combines and ranks the generated questions and web page content.To ensure efficient online inference, we design the QARanking model as a homogeneous dual-tower model, incorporating query intent to drive score fusion while balancing keyword matching and asymmetric matching.Additionally, we conduct a preliminary screening of questions for each document, selecting only the top-N relevant questions for further relevance calculation.Empirical results demonstrate the substantial performance improvement of our proposed method in web search.We achieve over

Query correction is a task that automatically detects and corrects errors in what users type into a search engine. Misspelled queries can lead to user dissatisfaction and churn. However, correcting a user query accurately is not an easy task. One major challenge is that a correction model must be capable of high-level language comprehension. Recently, pre-trained language models (PLMs) have been successfully applied to text correction tasks, but few works have been done on query correction. However, it is nontrivial to directly apply these PLMs to query correction in large-scale search systems due to the following challenging issues: 1) Expensive deployment. Deploying such a model requires expensive computations. 2) Lacking domain knowledge. A neural correction model needs massive training data to activate its power.

Passage ranking involves two stages: passage retrieval and passage re-ranking, which are important and challenging topics for both academics and industries in the area of Information Retrieval (IR). However, the commonly-used datasets for passage ranking usually focus on the English language. For non-English scenarios, such as Chinese, the existing datasets are limited in terms of data scale, fine-grained relevance annotation and false negative issues. To address this problem, we introduce T2Ranking, a large-scale Chinese benchmark for passage ranking. T2Ranking comprises more than 300K queries and over 2M unique passages from real-world search engines. Expert annotators are recruited to provide 4-level graded relevance scores (fine-grained) for query-passage pairs instead of binary relevance judgments (coarse-grained). To ease the false negative issues, more passages with higher diversities are considered when performing relevance annotations, especially in the test set, to ensure a more accurate evaluation. Apart from the textual query and passage data, other auxiliary resources are also provided, such as query types and XML files of documents which passages are generated from, to facilitate further studies. To evaluate the dataset, commonly used ranking models are implemented and tested on T2Ranking as baselines. The experimental results show that T2Ranking is challenging and there is still scope for improvement. The full data and all codes are available at https://github.com/THUIR/T2Ranking/.

Visual question answering is an important task in both natural language and vision understanding. However, in most of the public visual question answering datasets such as VQA, CLEVR, the questions are human generated that specific to the given image, such as 'What color are her eyes?'. The human generated crowdsourcing questions are relatively simple and sometimes have the bias toward certain entities or attributes [1, 55].