To satisfy users' diverse information needs, the aggregated search systems need to integrate heterogeneous results, with rich but different structural information, from a variety of verticals, such as news search, video search, and product search. A key challenge in aggregated search is to effectively and efficiently retrieve the most relevant results among a large number of heterogeneous information from different verticals. With the development of deep learning and pre-trained language models (PLMs), many researchers resort to Dense Retrieval (DR) models for a unified, efficient embedding-based retrieval and a better retrieval performance. However, existing dense retrieval models have limitations in: 1) capturing the structural information of search results ; and 2) generalizing across different vertical domains where the search results have different or even unseen structures. In this paper, we aim to tackle these limitations, and propose an effective and efficient dense retrieval model for aggregated search. Specifically, we utilize a deep prompt-tuning technique to make the pre-training model easily applied to downstream vertical search tasks. To capture the structural knowledge, we design a Graph Neural Network (GNN)-based structure prompt, to prompt how text segments are organized in the vanilla semi-structured data. We further incorporate a distributional prompt to model the theme of each domain, and enhance cross-domain generalization. Extensive experiments on the real-world data collected from the WeChat Search demonstrate that for aggregated search tasks, our models can achieve better performance over existing retrieval models, and have the superior ability to generalize to the various or even unseen vertical search tasks.

Unbiased learning to rank (ULTR), which aims to learn unbiased ranking models from biased user behavior logs, plays an important role in Web search. Previous research on ULTR has studied a variety of biases in users' clicks, such as position bias, presentation bias, and outlier bias. However, existing work often assumes that the behavior logs are collected from an ''average'' user, neglecting the differences between different users in their search and browsing behaviors. In this paper, we introduce personalized factors into the ULTR framework, which we term the user-aware ULTR problem. Through a formal causal analysis of this problem, we demonstrate that existing user-oblivious methods are biased when different users have different preferences over queries and personalized propensities of examining documents. To address such a personalized bias, we propose a novel user-aware inverse-propensity-score estimator for learning-to-rank objectives. Specifically, our approach models the distribution of user browsing behaviors for each query and aggregates user-weighted examination probabilities to determine propensities. We theoretically prove that the user-aware estimator is unbiased under some mild assumptions and shows lower variance compared to the straightforward way of calculating a user-dependent propensity for each impression. Finally, we empirically verify the effectiveness of our user-aware estimator by conducting extensive experiments on two semi-synthetic datasets and a real-world dataset.

Unbiased learning to rank (ULTR), which utilizes historical click logs to train ranking models, has attracted much attention in the IR community. Previous studies on ULTR have focused on mitigating a variety of biases in click logs, such as position bias, trust bias, and presentation bias, to recover the true relevance of the query-document pairs. However, they overlooked the intrinsic distribution shifts between the training data and test data. In this paper, we first validate and analyze the distribution shift problem with a real-world ULTR dataset. To solve this problem, we propose distributionally robust unbiased learning to rank (DRO-ULTR) methods. Specifically, we design two kinds of group distributionally robust optimization (group-DRO) frameworks for the existing ULTR methods, one using the pointwise click prediction loss and the other using the listwise counterfactual ranking loss. Finally, we empirically verify the effectiveness of our DRO-ULTR methods by conducting extensive experiments on the real-world dataset.

In the information retrieval (IR) area, dense retrieval (DR) models use deep learning techniques to encode queries and passages into embedding space to compute their semantic relations. It is important for DR models to balance both efficiency and effectiveness. Pre-trained language models (PLMs), especially Transformer-based PLMs, have been proven to be effective encoders of DR models. However, the self-attention component in Transformer-based PLM results in a computational complexity that grows quadratically with sequence length, and thus exhibits a slow inference speed for long-text retrieval. Some recently proposed non-Transformer PLMs, especially the Mamba architecture PLMs, have demonstrated not only comparable effectiveness to Transformer-based PLMs on generative language tasks but also better efficiency due to linear time scaling in sequence length. This paper implements the Mamba Retriever to explore whether Mamba can serve as an effective and efficient encoder of DR model for IR tasks. We fine-tune the Mamba Retriever on the classic short-text MS MARCO passage ranking dataset and the long-text LoCoV0 dataset. Experimental results show that (1) on the MS MARCO passage ranking dataset and BEIR, the Mamba Retriever achieves comparable or better effectiveness compared to Transformer-based retrieval models, and the effectiveness grows with the size of the Mamba model; (2) on the long-text LoCoV0 dataset, the Mamba Retriever can extend to longer text length than its pre-trained length after fine-tuning on retrieval task, and it has comparable or better effectiveness compared to other long-text retrieval models; (3) the Mamba Retriever has superior inference speed for long-text retrieval. In conclusion, Mamba Retriever is both effective and efficient, making it a practical model, especially for long-text retrieval.

Recently, geometric-inspired embedding methods draw research interests for their superior ability in representing transitive and asymmetric relations. A typical example, box embeddings, in which objects are parameterized as axis-aligned hyper-rectangles (i.e. boxes), can effectively model the partial orders and similarities between objects with the inclusion and overlapping relations of the boxes. However, the hard edges of the boxes present difficulties for gradient-based optimization. In this paper, we first identify two problems that may hinder the optimization of box embeddings, namely the zero-gradient problem and the vanishing gradient problem in high dimensional settings. Then, we propose a simple yet effective framework (Box++) to tackle these problems. For the first problem, Box++ combines the overlapping volumes and distance measures in optimizing box embeddings. The distance measures can naturally measure the “degree of disjointedness” for disjoint boxes and provide reasonable gradients for optimization. For the second problem, we theoretically prove that under certain conditions, the gradient would vanish exponentially, and therefore, make the optimization converges to suboptimal solutions. We further design a gradient normalization strategy for the Box++ models to alleviate this problem by rescaling the vanishing gradient. Extensive experiments on real-world datasets demonstrate the effectiveness of the proposed framework.

Point-of-interest (POI) search is important for location-based services, such as navigation and online ride-hailing service. The goal of POI search is to find the most relevant destinations from a large-scale POI database given a text query. To improve the effectiveness and efficiency of POI search, most existing approaches are based on a multi-stage pipeline that consists of an efficiency-oriented retrieval stage and one or more effectiveness-oriented re-rank stages. In this article, we focus on the first efficiency-oriented retrieval stage of the POI search. We first identify the limitations of existing first-stage POI retrieval models in capturing the semantic-geography relationship and modeling the fine-grained geographical context information. Then, we propose a Geo-Enhanced Dense Retrieval framework for POI search to alleviate the above problems. Specifically, the proposed framework leverages the capacity of pre-trained language models (e.g., BERT) and designs a pre-training approach to better model the semantic match between the query prefix and POIs. With the POI collection, we first perform a token-level pre-training task based on a geographical-sensitive masked language prediction and design two retrieval-oriented pre-training tasks that link the address of each POI to its name and geo-location. With the user behavior logs collected from an online POI search system, we design two additional pre-training tasks based on users’ query reformulation behavior and the transitions between POIs. We also utilize a late-interaction network structure to model the fine-grained interactions between the text and geographical context information within an acceptable query latency. Extensive experiments on the real-world datasets collected from the Didichuxing application demonstrate that the proposed framework can achieve superior retrieval performance over existing first-stage POI retrieval methods.

Ranking has been one of the most important tasks in information retrieval. With the development of deep representation learning, many researchers propose to encode both the query and items into embedding vectors and rank the items according to the inner product or distance measures in the embedding space. However, the ranking models based on vector embeddings may have shortages in effectiveness and efficiency. For effectiveness, they lack the intrinsic ability to model the diversity and uncertainty of queries and items in ranking. For efficiency, nearest neighbor search in a large collection of item vectors can be costly. In this work, we propose to use the recently proposed probabilistic box embeddings for effective and efficient ranking, in which queries and items are parameterized as high-dimensional axis-aligned hyper-rectangles. For effectiveness, we utilize probabilistic box embeddings to model the diversity and uncertainty with the overlapping relations of the hyper-rectangles, and prove that such overlapping measure is a kernel function which can be adopted in other kernel-based methods. For efficiency, we propose a box embedding-based indexing method, which can safely filter irrelevant items and reduce the retrieval latency. We further design a training strategy to increase the proportion of irrelevant items that can be filtered by the index. Experiments on public datasets show that the box embeddings and the box embedding-based indexing approaches are effective and efficient in two ranking tasks: ad hoc retrieval and product recommendation.