Chatbots such as Xiaoice have gained huge popularity in recent years. Users frequently mention their favorite works such as songs and movies in conversations with chatbots. Detecting these entities can help design better chat strategies and improve user experience. Existing named entity recognition methods are mainly designed for formal texts, and their performance on the informal chatbot conversation texts may not be optimal. In addition, these methods rely on massive manually annotated data for model training. In this article, we propose a neural approach to detect entities of works for Chinese chatbot. Our approach is based on a language model (LM) long-short term memory (LSTM) convolutional neural network (CNN) conditional random value (CRF), or LM-LSTM-CNN-CRF, framework, which contains a language model to generate context-aware character embeddings, a Bi-LSTM network to learn contextual character representations from global contexts, a CNN to learn character representations from local contexts, and a CRF layer to jointly decode the character label sequence. In addition, we propose an automatic text annotation method via quote marks to reduce the effort of manual annotation. Besides, we propose an iterative data purification method to improve the quality of the automatically constructed labeled data. Massive experiments on a real-world dataset validate that our approach can achieve good performance on entity detection for Chinese chatbots.

Sentiment classification is an important task in the sentiment analysis field. Many deep learning based sentiment classification methods have been proposed in recent years. However, these methods usually rely on massive labeled texts to train sentiment classifiers, which are expensive and time-consuming to annotate. Luckily, many high-quality sentiment lexicons have been constructed and can cover a large number of sentiment words. Since sentiment words are the basic units to convey sentiments in texts, these sentiment lexicons have the potential to improve the performance of neural sentiment classification. In this paper, we propose two approaches to exploit sentiment lexicons to enhance neural sentiment classification. In our first approach we use sentiment lexicons to learn sentiment-aware attentions. We propose a word sentiment classification task to classify the sentiments of words in a sentence based on their hidden representations in the attention network of neural sentiment classification models. We jointly train this task with neural sentiment classifier to facilitate the attention network to recognize and highlight sentiment-bearing words. In our second approach we use sentiment lexicons to learn sentiment-aware word embeddings. We design an auxiliary task to classify the sentiments of words in sentiment lexicons based on their word embeddings, and jointly train this task with neural sentiment classifier to encode sentiment information in sentiment lexicons to word embeddings. Extensive experiments on three benchmark datasets validate the effectiveness of our approach.

Review rating prediction is an important task in data mining and natural language processing fields, and has wide applications. Users usually express opinions towards many aspects in their reviews, and the overall review rating is a synthesis of these opinions. However, most existing review rating prediction methods ignore users' opinions on aspects, which is insufficient. In this paper, we propose a neural aspect-aware rating prediction approach for Chinese reviews. In our approach we propose a collaborative learning framework to jointly train review-level rating predictor and multiple aspect-level rating predictors. In our framework different rating predictors share the same review encoder model to exploit the inherent relatedness between them, but have different attention networks to focus on different informative texts for each task. The final review representation for rating prediction is a concatenation of the review representations from all predictors. Since word segmentation of Chinese reviews is usually inaccurate, we propose a multi-view learning model to learn review representations from both words and characters. Extensive experiments on real-world dataset validate the effectiveness of our approach.

Demographics of social media users such as gender are very important for personalized online services. However, the gender information of many users is usually not available. Luckily, the messages posted by social media users can provide rich clues for inferring their genders, since male and female users usually have differences in their message content. In addition, users with different genders often have different patterns in expressing emotions. In this paper, we propose a neural approach for gender prediction in social media based on both content and emotion of messages posted by users. The core of our approach is an emotion-aware hierarchical user representation model. Our model first learns message representations from words using message encoder and then learns user representations from messages using user encoder with hierarchical attention networks selecting important words and messages to learn informative user representations. In addition, we propose two methods to incorporate emotion information in messages into user representation learning. The first one is to incorporate emotion-aware message representations generated by a pre-trained emotion classifier into message representations. The second one is to train emotion-aware message encoders via jointly training our model with an auxiliary emotion classification task. Extensive experiments on two real-world datasets validate the effectiveness of our approach.

Neural network methods have achieved great success in sentiment classification. Recent studies have found that incorporating user and product information can effectively improve the performance of review sentiment classification. However, most of these studies only concentrate on the influence of users and products, ignoring the inherent correlation between users or products. This information is important for users or products since they can obtain more information from similar users or products. In this paper, we propose a novel framework for review rating prediction with user and product memory. First, besides the original user or product representations, we construct inferred representations from representative users or products which are stored in memory slots. These memory units can be viewed as refined knowledge representations of users or products learned from the data. Then, we employ two hierarchical networks with user attention and product attention using both the original and inferred representations. Experiments on benchmark datasets show that our method can achieve state-of-the-art performance. Besides, our approach performs much more better in cold-start scenarios where the training data is scarce.

Demographics of online users such as age and gender play an important role in personalized web applications. However, it is difficult to directly obtain the demographic information of online users. Luckily, search queries can cover many online users and the search queries from users with different demographics usually have some difference in contents and writing styles. Thus, search queries can provide useful clues for demographic prediction. In this paper, we study predicting users' demographics based on their search queries, and propose a neural approach for this task. Since search queries can be very noisy and many of them are not useful, instead of combining all queries together for user representation, in our approach we propose a hierarchical user representation with attention (HURA) model to learn informative user representations from their search queries. Our HURA model first learns representations for search queries from words using a word encoder, which consists of a CNN network and a word-level attention network to select important words. Then we learn representations of users based on the representations of their search queries using a query encoder, which contains a CNN network to capture the local contexts of search queries and a query-level attention network to select informative search queries for demographic prediction. Experiments on two real-world datasets validate that our approach can effectively improve the performance of search query based age and gender prediction and consistently outperform many baseline methods.

Twitter is a popular social media platform for information sharing and dissemination. Many Twitter users post tweets to share their experiences about drugs and adverse drug reactions. Automatic detection of tweets mentioning drug names and adverse drug reactions at a large scale has important applications such as pharmacovigilance. However, detecting drug name and adverse drug reaction mentioning tweets is very challenging, because tweets are usually very noisy and informal, and there are massive misspellings and user-created abbreviations for these mentions. In addition, these mentions are usually context dependent. In this paper, we propose a neural approach with hierarchical tweet representation and multi-head self-attention mechanism to jointly detect tweets mentioning drug names and adverse drug reactions. In order to alleviate the influence of massive misspellings and user-created abbreviations in tweets, we propose to use a hierarchical tweet representation model to first learn word representations from characters and then learn tweet representations from words. In addition, we propose to use multi-head self-attention mechanism to capture the interactions between words to fully model the contexts of tweets. Besides, we use additive attention mechanism to select the informative words to learn more informative tweet representations. Experimental results validate the effectiveness of our approach.

Chinese word segmentation (CWS) is very important for Chinese text processing. Existing methods for CWS usually rely on a large number of labeled sentences to train word segmentation models, which are expensive and time-consuming to annotate. Luckily, the unlabeled data is usually easy to collect and many high-quality Chinese lexicons are off-the-shelf, both of which can provide useful information for CWS. In this paper, we propose a neural approach for Chinese word segmentation which can exploit both lexicon and unlabeled data. Our approach is based on a variant of posterior regularization algorithm, and the unlabeled data and lexicon are incorporated into model training as indirect supervision by regularizing the prediction space of CWS models. Extensive experiments on multiple benchmark datasets in both in-domain and cross-domain scenarios validate the effectiveness of our approach.

Chinese named entity recognition (CNER) is an important task in Chinese natural language processing field. However, CNER is very challenging since Chinese entity names are highly context-dependent. In addition, Chinese texts lack delimiters to separate words, making it difficult to identify the boundary of entities. Besides, the training data for CNER in many domains is usually insufficient, and annotating enough training data for CNER is very expensive and time-consuming. In this paper, we propose a neural approach for CNER. First, we introduce a CNN-LSTM-CRF neural architecture to capture both local and long-distance contexts for CNER. Second, we propose a unified framework to jointly train CNER and word segmentation models in order to enhance the ability of CNER model in identifying entity boundaries. Third, we introduce an automatic method to generate pseudo labeled samples from existing labeled data which can enrich the training data. Experiments on two benchmark datasets show that our approach can effectively improve the performance of Chinese named entity recognition, especially when training data is insufficient.

Supervised learning methods are widely used in sentiment classification. However, when sentiment distribution is imbalanced, the performance of these methods declines. In this paper, we propose an effective approach for imbalanced sentiment classification. In our approach, multiple balanced subsets are sampled from the imbalanced training data and a multi-task learning based framework is proposed to learn robust sentiment classifier from these subsets collaboratively. In addition, we incorporate prior knowledge of sentiment expressions extracted from both existing sentiment lexicons and massive unlabeled data into our approach to enhance the learning of sentiment classifier in imbalanced scenario. Experimental results on benchmark datasets validate the effectiveness of our approach in improving imbalanced sentiment classification.

It is important to detect social spammers and spam messages in microblogging platforms. Existing methods usually handle the detection of social spammers and spam messages as two separate tasks using supervised learning techniques. However, labeled samples are usually scarce and manual annotation is expensive. In this paper, we propose a semi-supervised collaborative learning approach to jointly detect social spammers and spam messages in microblogging platforms. In our approach, the social spammer classifier and spam message classifier are collaboratively trained by exploiting the inherent relatedness between these tasks. In addition, unlabeled samples are incorporated into model training with the help of social contexts of users and messages. Experiments on real-world dataset show our approach can effectively improve the performance of both social spammer detection and spam message detection.

Sentence-level sentiment classification aims to mine fine-grained sentiment information from texts. Existing methods for this task are usually based on supervised learning and rely on massive labeled sentences for model training. However, annotating sufficient sentences is expensive and time-consuming. In this paper, we propose a neural sentence-level sentiment classification approach which can exploit heterogeneous sentiment supervision and reduce the dependence on labeled sentences. Besides the sentence-level supervision from labeled sentences, our approach can also incorporate the word-level supervision extracted from sentiment lexicons, document-level supervision extracted from labeled documents and sentiment relations between sentences extracted from unlabeled documents. A unified neural framework is proposed to fuse heterogeneous sentiment supervision to train sentence-level sentiment classification model. Experiments on benchmark datasets validate the effectiveness of our approach.