Zhanyu Liu, Shiyao Wang, Xingmei Wang, Rongzhou Zhang, Jiaxin Deng, Honghui Bao, Jinghao Zhang, Wuchao Li, PengFei Zheng, Xiangyu Wu, Yifei Hu, Qigen Hu, Xinchen Luo, Lejian Ren, Zhang Zixing, Qianqian Wang, Kuo Cai, Yunfan Wu, Hongtao Cheng, Zexuan Cheng, Lu Ren, Huanjie Wang, Yi Su, Ruiming Tang, Kun Gai, Guorui Zhou. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Modeling user long-term behavior sequences is critical for enhancing Click-Through Rate (CTR) prediction. Existing methods typically employ two cascaded search units-General Search Unit (GSU) for rapid retrieval and Exact Search Unit (ESU) for precise modeling-to balance efficiency and effectiveness. However, they are constrained to recent behaviors due to computational limitations. Clustering user behaviors offers a potential solution, enabling GSU to access lifelong behaviors while maintaining inference efficiency, but current clustering approaches often lack generalizability, or fail to remain effective in high-dimensional data due to non-end-to-end clustering and recommendation. Given that centroids in clustering group similar data points based on proximity, similar to how queries function in transformers, we can integrate the learning of queries with CTR tasks in an end-to-end manner, shifting clustering from meaningless Euclidean distances to meaningful semantic distances. Therefore, we propose C-Former, a transformer-based clustering model specifically designed for modeling lifelong behavior sequences. The C-Former encoder leverages a group of learnable clustering anchor points that access the lifelong user behaviors to extract personalized interests. Then, the C-Former decoder reconstructs lifelong user behaviors based on the compact output of the encoder. The reconstruction and orthogonal loss ensure that centroids are informative and diverse in capturing user preferences. Clustering is further guided by supervisory signals from CTR, establishing an end-to-end framework. The proposed C-Former achieves linear time complexity in training with respect to sequence length and significantly reduces inference latency by directly utilizing cached centroids. Experiments on four benchmark datasets demonstrate the effectiveness of C-Former for lifelong user behavior sequence modeling. The code is available at https://github.com/pepsi2222/C-Former.

Generative recommendation has emerged as a transformative paradigm in recommender systems, enabling modeling user behavior autoregressively without explicit target conditioning. While this approach eliminates the need for target signals, it necessitates compressing extensive historical interactions-potentially spanning lifelong sequences-into coherent interest representations. Conventional methods for handling long sequences typically rely on target-guided search mechanisms (e.g., SIM) to efficiently filter and compress behaviors. However, this strategy is incompatible with generative frameworks due to their target-agnostic nature. To address these challenges, we propose a novel encoder-decoder model named HiCoGen (Hierarchical Compression-based Session-wise Generative Model), which efficiently models long-term interests in generative models. In the encoder, HiCoGen compresses behavior sequences using hierarchical content similarity clustering and employs a hierarchical attention architecture to reduce sequence length while preserving information integrity. In the decoder, HiCoGen uses session-wise generation instead of point-wise generation to better align with industrial short-video applications. To enhance the stability of session-wise generation, we introduce an auxiliary Hierarchical Multi-Token Prediction module. Extensive experiments on public and industrial datasets show significant performance gains over state-of-the-art methods (21.2% in ML-1M and 35.6% in industrial datasets on NDCG@3). We also conducted visualization and performance analysis to explore the advantages of long sequence modeling.

Retrieval aims to find the top-k items most relevant to a query/user from a large dataset. Traditional retrieval models represent queries/users and items as embedding vectors and use Approximate Nearest Neighbor (ANN) search for retrieval. Recently, researchers have proposed a generative-based retrieval method that represents items as token sequences and uses a decoder model for autoregressive training. Compared to traditional methods, this approach uses more complex models and integrates index structure during training, leading to better performance. However, these methods remain two-stage processes, where index construction is separate from the retrieval model, limiting the model's overall capacity. Additionally, existing methods construct indices by clustering pre-trained item representations in Euclidean space. However, real-world scenarios are more complex, making this approach less accurate. To address these issues, we propose a \underline{U}nified framework for \underline{R}etrieval and \underline{I}ndexing, termed \textbf{URI}. URI ensures strong consistency between index construction and the retrieval model, typically a Transformer decoder. URI simultaneously builds the index and trains the decoder, constructing the index through the decoder itself. It no longer relies on one-sided item representations in Euclidean space but constructs the index within the interactive space between queries and items. Experimental comparisons on three real-world datasets show that URI significantly outperforms existing methods.

Industrial recommendation systems (RS) rely on the multi-stage pipeline to balance effectiveness and efficiency when delivering items from a vast corpus to users. Existing RS benchmark datasets primarily focus on the exposure space, where novel RS algorithms are trained and evaluated. However, when these algorithms transition to real-world industrial RS, they face two critical challenges: (1) handling unexposed items—a significantly larger space than the exposed one, profoundly impacting their practical performance; and (2) overlooking the intricate interplay between multiple stages of the recommendation pipeline, resulting in suboptimal system performance. To bridge the gap between offline RS benchmarks and real-world online environments, we introduce RecFlow—an industrial full-flow recommendation dataset. Unlike existing datasets, RecFlow includes samples not only from the exposure space but also from unexposed items filtered at each stage of the RS funnel. RecFlow comprises 38 million interactions from 42,000 users across nearly 9 million items with additional 1.9 billion stage samples collected from 9.3 million online requests over 37 days and spanning 6 stages. Leveraging RecFlow, we conduct extensive experiments to demonstrate its potential in designing novel algorithms that enhance effectiveness by incorporating stage-specific samples. Some of these algorithms have already been deployed online at KuaiShou, consistently yielding significant gains. We propose RecFlow as the first comprehensive whole-pipeline benchmark dataset for the RS community, enabling research on algorithm design across the entire recommendation pipeline, including selection bias study, debiased algorithms, multi-stage consistency and optimality, multi-task recommendation, and user behavior modeling.

Sequential Recommendation (SR) plays a pivotal role in recommender systems by tailoring recommendations to user preferences based on their non-stationary historical interactions. Achieving high-quality performance in SR requires attention to both item representation and diversity. However, designing an SR method that simultaneously optimizes these merits remains a long-standing challenge. In this study, we address this issue by integrating recent generative Diffusion Models (DM) into SR. DM has demonstrated utility in representation learning and diverse image generation. Nevertheless, a straightforward combination of SR and DM leads to sub-optimal performance due to discrepancies in learning objectives (recommendation vs. noise reconstruction) and the respective learning spaces (non-stationary vs. stationary). To overcome this, we propose a novel framework called DimeRec (Di ffusion with multi-interest enhanced Rec ommender). DimeRec synergistically combines a guidance extraction module (GEM) and a generative diffusion aggregation module (DAM). The GEM extracts crucial stationary guidance signals from the user's non-stationary interaction history, while the DAM employs a generative diffusion process conditioned on GEM's outputs to reconstruct and generate consistent recommendations. Our numerical experiments demonstrate that DimeRec significantly outperforms established baseline methods across three publicly available datasets. Furthermore, we have successfully deployed DimeRec on a large-scale short video recommendation platform, serving hundreds of millions of users. Live A/B testing confirms that our method improves both users' time spent and result diversification.

Vector retrieval focuses on finding the k-nearest neighbors from a bunch of data points, and is widely used in a diverse set of areas such as information retrieval and recommender system. The current state-of-the-art methods represented by HNSW usually generate indexes with a big memory footprint, restricting the scale of data they can handle, except resorting to a hybrid index with external storage. The space-partitioning learned indexes, which only occupy a small memory, have made great breakthroughs in recent years. However, these methods rely on a large amount of labeled data for supervised learning, so model complexity affects the generalization.

Recommender systems (RS) have to select the top-N items from a massive item set. For the sake of efficient recommendation, RS usually represents user and item as latent embeddings, and relies on approximate nearest neighbour search (ANNs) to retrieve the recommendation result. Despite the reduction of running time, the representation learning is independent of ANNs index construction; thus, the two operations can be incompatible, which results in potential loss of recommendation accuracy. To overcome the above problem, we propose the Recommender Forest (a.k.a., RecForest), which jointly learns latent embedding and index for efficient and high-fidelity recommendation. RecForest consists of multiple k-ary trees, each of which is a partition of the item set via hierarchical balanced clustering such that each item is uniquely represented by a path from the root to a leaf. Given such a data structure, an encoder-decoder based routing network is developed: it first encodes the context, i.e., user information, into hidden states; then, leveraging a transformer-based decoder, it identifies the top-N items via beam search. Compared with the existing methods, RecForest brings in the following advantages: 1) the false partition of the boundary items can be effectively alleviated by the use of multiple trees; 2) the routing operation becomes much more accurate thanks to the powerful transformer decoder; 3) the tree parameters are shared across different tree levels, making the index to be extremely memory-efficient. The experimental studies are performed on five popular recommendation datasets: with a significantly simplified training cost, RecForest outperforms competitive baseline approaches in terms of both recommendation accuracy and efficiency.