Personalized review summarization is a challenging task in recommender systems, which aims to generate condensed and readable summaries for product reviews. Recently, some methods propose to adopt the sentiment signals of reviews to enhance the review summarization. However, most previous works only share the semantic features of reviews via preliminary multi-task learning, while ignoring the rich personalized information of users and products, which is crucial to both sentiment identification and comprehensive review summarization. In this paper, we propose a sentiment-aware review summarization method with an elaborately designed multi-task fine-tuning framework to make full use of personalized information of users and products effectively based on Pretrained Language Models (PLMs). We first denote two types of personalized information including IDs and historical summaries to indicate their identification and semantics information respectively. Subsequently, we propose to incorporate the IDs of the user/product into the PLMs-based encoder to learn the personalized representations of input reviews and their historical summaries in a fine-tuning way. Based on this, an auxiliary context-aware review sentiment classification task and a further sentiment-guided personalized review summarization task are jointly learned. Specifically, the sentiment representation of input review is used to identify relevant historical summaries, which are then treated as additional semantic context features to enhance the summary generation process. Extensive experimental results show our approach could generate sentiment-consistent summaries and outperforms many competitive baselines on both review summarization and sentiment classification tasks.

Recently, with the popularity of ChatGPT, large-scale language models have experienced rapid development. However, there is a scarcity of open-sourced chat models specifically designed for the Chinese language, especially in the field of Chinese finance, at the scale of hundreds of billions. To address this gap, we introduce XuanYuan 2.0, the largest Chinese chat model to date, built upon the BLOOM-176B architecture. Additionally, we propose a novel training method called hybrid-tuning to mitigate catastrophic forgetting. By integrating general and domain-specific knowledge, as well as combining the stages of pre-training and fine-tuning, XuanYuan 2.0 is capable of providing accurate and contextually appropriate responses in the Chinese financial domain.

Expert finding aims at seeking potential users to answer new questions in Community Question Answering (CQA) websites. Most existing methods focus on designing matching frameworks between questions and experts, and rely on negative sampling technology for model training. However, sampling would lose lots of useful information about experts and questions, and make these sampling-based methods suffer the bias and non-robust issues, which may lead to an insufficient matching performance for expert findings. In this paper, we propose a novel Efficient Non-sampling Expert Finding model, named ENEF, which could learn accurate representations of questions and experts from whole training data. In our approach, we adopt a rather basic question encoder and a simple matching framework, then an efficient whole-data optimization method is elaborately designed to learn the model parameters without negative sampling with rather a low space and time complexity. Extensive experimental results on four real-world CQA datasets demonstrate that our model ENEF could achieve better performance and faster training efficiency than existing state-of-the-art expert finding methods.

Expert Finding is an important task in Community Question Answering (CQA) platforms, which could help route questions to potential expertise users to answer. The key is to model the question content and experts based on their historical answered questions accurately. Recently Pretrained Language Models (PLMs, e.g., Bert) have shown superior text modeling ability and have been used in expert finding preliminary. However, most PLMs-based models focus on the corpus or document granularity during pretraining, which is inconsistent with the downstream expert modeling and finding task. In this paper, we propose an expert-level pretraining language model named ExpertBert, aiming to model questions, experts as well as question-expert matching effectively in a pretraining manner. In our approach, we aggregate the historical answered questions of an expert as the expert-specific input.Besides, we integrate the target question into the input and design a label-augmented Masked Language Model (MLM) task to further capture the matching pattern between question and experts, which makes the pretraining objectives that more closely resemble the downstream expert finding task. Experimental results and detailed analysis on real-world CQA datasets demonstrate the effectiveness of our ExpertBert.

Personalized news recommendation aims to provide people with customized content, which can effectively improve the reading experience. Because user interests in news are diverse and changeable, how to learn accurate user representations is the core challenge in news recommendation. However, most of the previous works only apply news-level representation for user modeling directly, the views of news, such as title, abstract, and category, are only implied and compressed into a single vector of news, which makes it impossible for different views in different news to interact with each other. In this paper, we first focus on the view-level information for user modeling and propose Deep View-Temporal Interaction Network (DeepVT) for news recommendation. It mainly contains two components, i.e., 2D semi-causal convolutional neural network (SC-CNN) and multi-operator attention (MoA). SC-CNN can synthesize interaction information at the view-level and temporal information at the news-level simultaneously and efficiently. And MoA integrates different similarity operators in self-attention functions to avoid attention bias and enhance robustness. By collaboration with SC-CNN, the global interaction at the view-level becomes more sufficient. Experiments on a large-scale real-world dataset, Microsoft News Dataset (MIND), show that our model outperforms previous models in terms of all metrics significantly.

Recently, pre-trained language models have been successfully applied to the task of text retrieval and ranking. However, in real scenes, users' click behavior is usually affected by selection, position, or exposure bias, which may lead to insufficient positive annotations and introduce additional noise. And for different candidate documents of the same query, the previous optimization objectives usually use a single granularity and static loss weights. It makes the performance of ranking models more susceptible to the bias issue mentioned above. Thus, in this paper, we focus on BERT-based document reranking and propose Dynamic Multi-Granularity Learning (DML). By introducing Gaussian distribution into traditional loss functions, the weights of different documents can change dynamically according to the prediction probability to avoid the impact of unlabeled positive documents. Besides, both document-granularity and instance-granularity are considered to balance relative relations and absolute scores of candidate documents. Extensive experiments show that DML significantly outperforms previous state-of-the-art models on the MS MARCO document ranking dataset.