Finding experts is essential in Community Question Answering (CQA) platforms as it enables the effective routing of questions to potential users who can provide relevant answers. The key is to personalized learning expert representations based on their historical answered questions, and accurately matching them with target questions. Recently, the applications of Pre-Trained Language Models (PLMs) have gained significant attraction due to their impressive capability to comprehend textual data, and are widespread used across various domains. There have been some preliminary works exploring the usability of PLMs in expert finding, such as pre-training expert or question representations. However, these models usually learn pure text representations of experts from histories, disregarding personalized and fine-grained expert modeling. For alleviating this, we present a personalized pre-training and fine-tuning paradigm, which could effectively learn expert interest and expertise simultaneously. Specifically, in our pre-training framework, we integrate historical answered questions of one expert with one target question, and regard it as a candidate-aware expert-level input unit. Then, we fuse expert IDs into the pre-training for guiding the model to model personalized expert representations, which can help capture the unique characteristics and expertise of each individual expert. Additionally, in our pre-training task, we design (1) a question-level masked language model task to learn the relatedness between histories, enabling the modeling of question-level expert interest; (2) a vote-oriented task to capture question-level expert expertise by predicting the vote score the expert would receive. Through our pre-training framework and tasks, our approach could holistically learn expert representations including interests and expertise. Our method has been extensively evaluated on six real-world CQA datasets, and the experimental results consistently demonstrate the superiority of our approach over competitive baseline methods.

Personalized news recommendation aims to help users find news content they prefer, which has attracted increasing attention recently. There are two core issues in news recommendation: learning news representation and matching candidate news with user interests. In this context, “candidate” indicates potential for interest. Due to the superior ability to understand natural language demonstrated by Pretrained Language Models (PLMs), recent works utilize PLMs (e.g., BERT) to strengthen news modeling, obtaining more accurate user interest matching and achieving notable improvement in news recommendation. However, the existing PLM-based methods are usually incapable of fully exploring the fine-grained (i.e., word-level) relatedness between user behaviors and candidate news due to the heavy computational cost brought by PLMs. In this article, we propose a group-based personalized news recommendation method with long- and short-term matching mechanisms between users and candidate news based on PLMs to learn fine-grained matching efficiently and effectively. In our approach, we design to group user historical clicked news into chunks with quite shorter news sequences according to their clicked timestamps, which could alleviate the computation issues of PLMs. PLMs are applied in each group jointly with the candidate news to capture their word-level interaction, and global group-level matching is learned across different groups. In addition, the group-based mechanism could be naturally adapted for long- and short-term user representation learning, in which we build users’ long preferences from the representations of all groups and treat the last group as short interests, respectively. Finally, we employ a gate network to dynamically unify the group-level, long- and short-term representations, yielding comprehensive user-news matching effectively. Extensive experiments are conducted on two real-world datasets. The results show that our proposed method achieves superior performance in news recommendations.

Expert finding aims at seeking potential users to answer new questions in Community Question Answering (CQA) websites. Most existing methods focus on designing matching frameworks between questions and experts, and rely on negative sampling technology for model training. However, sampling would lose lots of useful information about experts and questions, and make these sampling-based methods suffer the bias and non-robust issues, which may lead to an insufficient matching performance for expert findings. In this paper, we propose a novel Efficient Non-sampling Expert Finding model, named ENEF, which could learn accurate representations of questions and experts from whole training data. In our approach, we adopt a rather basic question encoder and a simple matching framework, then an efficient whole-data optimization method is elaborately designed to learn the model parameters without negative sampling with rather a low space and time complexity. Extensive experimental results on four real-world CQA datasets demonstrate that our model ENEF could achieve better performance and faster training efficiency than existing state-of-the-art expert finding methods.

Expert Finding is an important task in Community Question Answering (CQA) platforms, which could help route questions to potential expertise users to answer. The key is to model the question content and experts based on their historical answered questions accurately. Recently Pretrained Language Models (PLMs, e.g., Bert) have shown superior text modeling ability and have been used in expert finding preliminary. However, most PLMs-based models focus on the corpus or document granularity during pretraining, which is inconsistent with the downstream expert modeling and finding task. In this paper, we propose an expert-level pretraining language model named ExpertBert, aiming to model questions, experts as well as question-expert matching effectively in a pretraining manner. In our approach, we aggregate the historical answered questions of an expert as the expert-specific input.Besides, we integrate the target question into the input and design a label-augmented Masked Language Model (MLM) task to further capture the matching pattern between question and experts, which makes the pretraining objectives that more closely resemble the downstream expert finding task. Experimental results and detailed analysis on real-world CQA datasets demonstrate the effectiveness of our ExpertBert.

In Community Question Answering (CQA) websites, expert finding aims at seeking suitable experts to answer questions. The key is to explore the inherent relevance based on the representations of questions and experts. Existing methods usually learn these features from single view information (e.g., question title), which would be not insufficient to fully learn their representations. In this paper, we propose a personalized expert finding method with a multi-view attentive matching mechanism. We design three modules under the multi-view paradigm, including a question encoder, an intra-view encoder, and an inter-view encoder, which aims to comprehend the comprehensive relationships between experts and questions. In the question encoder, we learn the multi-view question features from its title, body and tag views respectively. In the intra-view encoder, we design an interactive attention network to capture the view-specific relevance between the target question and the historical answered questions of experts for all different views. Furthermore, in the inter-view encoder we employ a personalized attention network to aggregate different view information to learn expert/question representations. In this way, the match of the expert and question could be fully captured from the multi-view information via the intra- and inter-view mechanisms. Experimental results on six datasets demonstrate that the proposed method could achieve better performance than existing state-of-the-art methods.

Product reviews can provide rich information about the opinions users have of products. However, it is nontrivial to effectively infer user preference and item characteristics from reviews due to the complicated semantic understanding. Existing methods usually learn features for users and items from reviews in single static fashions and cannot fully capture user preference and item features. In this article, we propose a neural review-based recommendation approach that aims to learn comprehensive representations of users/items under a three-tier attention framework. We design a review encoder to learn review features from words via a word-level attention, an aspect encoder to learn aspect features via a review-level attention, and a user/item encoder to learn the final representations of users/items via an aspect-level attention. In word- and review-level attentions, we adopt the context-aware mechanism to indicate importance of words and reviews dynamically instead of static attention weights. In addition, the attentions in the word and review levels are of multiple paradigms to learn multiple features effectively, which could indicate the diversity of user/item features. Furthermore, we propose a personalized aspect-level attention module in user/item encoder to learn the final comprehensive features. Extensive experiments are conducted and the results in rating prediction validate the effectiveness of our method.

There are two main paradigms to exploit review information for recommendation. One is to concatenate all reviews of a user/item into a long document, which may neglect the different usefulness of reviews. The other paradigm is review-level i.e., analyzing each review separately to learn user/item features. In fact, the two paradigms are complementary, and fusing them together has the potential to learn more comprehensive features of users/items. Hence, we propose a unified framework to jointly learn document- and review-level representations of users/items. We design a document encoder to learn document-level features of users/items. Then, we use a review encoder to learn representations of reviews from words, and a user/item encoder to learn review-level features of users/items. Besides, different reviews from the same user may have different importance for different target items due to different item characteristics. We propose a cross attention model for user representation learning whose query vector is the embedding of target item ID, and apply it to the above three encoders to select different informative words and reviews for different target items. Extensive experiments validate the effectiveness of our method.