Query rewriting is a crucial task for improving retrieval, especially in professional domains such as law and medicine, where user queries are often underspecified and ambiguous. While large language models (LLMs) offer strong understanding and generation capabilities, existing LLM-based approaches reduce the task to text transformation or expansion, neglecting reasoning to disambiguate queries, which fails to bridge the cognitive gap between user queries and specialized documents. In this paper, we propose Think-Then-Rewrite (TTR), a reinforcement learning based framework that unleashes LLMs' reasoning ability for domain-specific query rewriting. TTR introduces a contrastive mutual information reward to encourage the LLM to generate reasoning processes that effectively distinguish confusing distractors. To boost early-stage training, TTR also constructs golden query rewrites as off‑policy data, providing strong guidance for RL learning. A mixed-policy optimization then combines on-policy and off-policy signals, ensuring both effectiveness and stability. Extensive experiments on legal and medical retrieval benchmarks demonstrate that TTR achieves state-of-the-art performance.

While reinforcement learning with verifiable rewards (RLVR) has advanced LLM reasoning in structured domains like mathematics and programming, its application to general-domain reasoning tasks remains challenging due to the absence of verifiable reward signals. To this end, methods like Reinforcement Learning with Reference Probability Reward (RLPR) have emerged, leveraging the probability of generating the final answer as a reward signal. However, these outcome-focused approaches neglect crucial step-by-step supervision of the reasoning process itself. To address this gap, we introduce Probabilistic Process Supervision (P2S), a novel self-supervision framework that provides fine-grained process rewards without requiring a separate reward model or human-annotated reasoning steps. During reinforcement learning, P2S synthesizes and filters a high-quality reference reasoning chain (gold-CoT). The core of our method is to calculate a Path Faithfulness Reward (PFR) for each reasoning step, which is derived from the conditional probability of generating the gold-CoT's suffix, given the model's current reasoning prefix. Crucially, this PFR can be flexibly integrated with any outcome-based reward, directly tackling the reward sparsity problem by providing dense guidance. Extensive experiments on reading comprehension and medical Question Answering benchmarks show that P2S significantly outperforms strong baselines.

The emergence of large language models (LLMs) has made legal consultation resources more accessible. However, due to the inherent ability of LLMs to response to any input, in practical use, if the query itself is incomplete or complex, the model’s response may generate hallucinations, misleading the user. In fact, LLMs are best suited to answer complete and simple legal questions, while complex issues should be handled by legal experts. Therefore, it is essential to route queries as simple or complex before the LLM provides an answer. Yet, current approaches rely solely on internal confidence metrics, overlooking the inherent complexity of legal queries. To address this limitation, we propose a novel method that incorporates first-order logic (FOL) rules as additional evidence to assess the complexity of queries. Our approach consists of two key stages: FOL-based Matching and Inference-driven Routing. In the first stage, LLMs extract key information from user inputs and map it to a pool of FOL rules to match relevant legal evidence. In the second stage, symbolic reasoning is applied to the matched evidence to derive logical inferences and make routing decisions. We fine-tune a 7B model (Qwen2-7B-Instruct) to combine FOL generation with query routing, enhancing the model’s reasoning capabilities and interpretability. This approach achieves a 24.77% improvement.

Ang Li, Yiquan Wu, Ming Cai, Adam Jatowt, Xiang Zhou, Weiming Lu, Changlong Sun, Fei Wu, Kun Kuang. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.

Information Extraction (IE) and Text Classification (CLS) serve as the fundamental pillars of NLU, with both disciplines relying on analyzing input sequences to categorize outputs into pre-established schemas. However, there is no existing encoder-based model that can unify IE and CLS tasks from this perspective. To fully explore the foundation shared within NLU tasks, we have proposed arecursive method with explicit schema instructor for universal NLU. Specifically, we firstly redefine the true universal information extraction (UIE) with a formal formulation that covers almost all extraction schemas, including quadruples and quintuples which remain unsolved for previous UIE models. Then, we expands the formulation to all CLS and multi-modal NLU tasks. Based on that, we introduce RexUniNLU, an universal NLU solution that employs explicit schema constraints for IE and CLS, which encompasses all IE and CLS tasks and prevent incorrect connections between schema and input sequence. To avoid interference between different schemas, we reset the position ids and attention mask matrices. Extensive experiments are conducted on IE, CLS in both English and Chinese, and multi-modality, revealing the effectiveness and superiority. Our codes are publicly released athttps://modelscope.cn/models/iic/nlp_deberta_rex-uninlu_chinese-base.

In text classification models, while the unsupervised attention mechanism can enhance performance, it often produces attention distributions that are puzzling to humans, such as assigning high weight to seemingly insignificant conjunctions. Recently, numerous studies have explored Attention Supervision (AS) to guide the model toward more interpretable attention distributions. However, such AS can impact classification performance, especially in specialized domains. In this paper, we address this issue from a causality perspective. Firstly, we leverage the causal graph to reveal two biases in the AS: 1) Bias caused by the label distribution of the dataset. 2) Bias caused by the words' different occurrence ranges that some words can occur across labels while others only occur in a particular label. We then propose a novel De-biased Attention Supervision (DAS) method to eliminate these biases with causal techniques. Specifically, we adopt backdoor adjustment on the label-caused bias and reduce the word-caused bias by subtracting the direct causal effect of the word. Through extensive experiments on two professional text classification datasets (e.g., medicine and law), we demonstrate that our method achieves improved classification accuracy along with more coherent attention distributions.

While self-supervised graph pretraining techniques have shown promising results in various domains, their application still experiences challenges of limited topology learning, human knowledge dependency, and incompetent multi-level interactions. To address these issues, we propose a novel solution, Dual-level Graph self-supervised Pretraining with Motif discovery (DGPM), which introduces a unique dual-level pretraining structure that orchestrates node-level and subgraph-level pretext tasks. Unlike prior approaches, DGPM autonomously uncovers significant graph motifs through an edge pooling module, aligning learned motif similarities with graph kernel-based similarities. A cross-matching task enables sophisticated node-motif interactions and novel representation learning. Extensive experiments on 15 datasets validate DGPM's effectiveness and generalizability, outperforming state-of-the-art methods in unsupervised representation learning and transfer learning settings. The autonomously discovered motifs demonstrate the potential of DGPM to enhance robustness and interpretability.

Recently, we have witnessed the breakthroughs of meta-learning for few-shot learning scenario. Data augmentation is essential for meta-learning, particularly in situations where data is extremely scarce. However, existing text data augmentation methods can not ensure the diversity and quality of the generated data, which leads to sub-optimal performance. Inspired by the recent success of large language models (LLMs) which demonstrate improved language comprehension abilities, we propose a Meta-learning framework with Progressive Data Augmentation (PDAMeta) for few-shot text classification, which contains a two-stage data augmentation strategy. First, the prompt-based data augmentation enriches the diversity of the training instances from a global perspective. Second, the attention-based data augmentation further improves the data quality from a local perspective. Last, we propose a dual-stream contrastive meta-learning strategy to learn discriminative text representations from both original and augmented instances. Extensive experiments conducted on four public few-shot text classification datasets show that PDAMeta significantly outperforms several state-of-the-art models and shows better robustness.

Large language models (LLMs) have demonstrated remarkable capabilities across various NLP tasks. However, their computational costs are prohibitively high. To address this issue, previous research has attempted to distill the knowledge of LLMs into smaller models by generating annotated data. Nonetheless, these works have mainly focused on the direct use of LLMs for text generation and labeling, without fully exploring their potential to comprehend the target task and acquire valuable knowledge. In this paper, we propose EvoKD: Evolving Knowledge Distillation, which leverages the concept of active learning to interactively enhance the process of data generation using large language models, simultaneously improving the task capabilities of small domain model (student model). Different from previous work, we actively analyze the student model’s weaknesses, and then synthesize labeled samples based on the analysis. In addition, we provide iterative feedback to the LLMs regarding the student model’s performance to continuously construct diversified and challenging samples. Experiments and analysis on different NLP tasks, namely, text classification and named entity recognition show the effectiveness of EvoKD.

The widespread use of pre-trained language models (PLMs) in natural language processing (NLP) has greatly improved performance outcomes. However, these models’ vulnerability to adversarial attacks (e.g., camouflaged hints from drug dealers), particularly in the Chinese language with its rich character diversity/variation and complex structures, hatches vital apprehension. In this study, we propose a novel method, CHinese vAriatioN Graph Enhancement (CHANGE), to increase the robustness of PLMs against character variation attacks in Chinese content. CHANGE presents a novel approach to incorporate a Chinese character variation graph into the PLMs. Through designing different supplementary tasks utilizing the graph structure, CHANGE essentially enhances PLMs’ interpretation of adversarially manipulated text. Experiments conducted in a multitude of NLP tasks show that CHANGE outperforms current language models in combating against adversarial attacks and serves as a valuable contribution to robust language model research. Moreover, these findings highlight the substantial potential of graph-guided pre-training strategies for real-world applications.

Chengyuan Liu, Yangyang Kang, Shihang Wang, Lizhi Qing, Fubang Zhao, Chao Wu, Changlong Sun, Kun Kuang, Fei Wu. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024.

While Large Language Models (LLMs) demonstrate impressive generation abilities, they frequently struggle when it comes to specialized domains due to their limited domain-specific knowledge.Studies on domain-specific LLMs resort to expanding the vocabulary before finetuning on domain-specific corpus, aiming to decrease the sequence length and enhance efficiency during decoding, without thoroughly investigating the results of vocabulary expansion to LLMs over different domains.Our pilot study reveals that expansion with only a subset of the entire vocabulary may lead to superior performance.Guided by the discovery, this paper explores how to identify a vocabulary subset to achieve the optimal results.We introduce VEGAD, an adaptive method that automatically identifies valuable words from a given domain vocabulary.Our method has been validated through experiments on three Chinese datasets, demonstrating its effectiveness.Additionally, we have undertaken comprehensive analyses of the method.The selection of a optimal subset for expansion has shown to enhance performance on both domain-specific tasks and general tasks, showcasing the potential of VE-GAD.

Accurately evaluating the impact of scientific papers is crucial. However, existing methodologies face certain challenges, including latent factors affecting citation behaviors and dynamic intrinsic of citation networks. To address these challenges, this study introduces a novel framework named CoDy (modeling scholarly Collaboration and temporal Dynamics in citation networks for impact prediction). CoDy strategically predicts author collaborations as an auxiliary task, forecasting not only the number of current collaborations between scholars but also the number of future collaborations among them. Besides, CoDy proposes a fine-grained temporal encoding module to model the multiple different temporal patterns for publication and citation. Extensive experimental validations demonstrate CoDy's effectiveness in predicting citation counts and classifying impact levels. In-depth analyses provide further validation of its reliability and robustness. CoDy can significantly enhance impact prediction by explicitly modeling collaboration and temporal patterns and offer valuable insights into paper impact formation.

In the training procedures of many real-world learning models, gathering and annotating decent amounts of labeled data can be cost-prohibitive. To mitigate this data-hungry problem, active learning (AL) and semi-supervised learning (SSL) are frequently adopted as two effective but often isolated means. Some recent studies explored the potential of combining AL and SSL to better probe the unlabeled data. However, almost all these contemporary SSL-AL works use a simple combination strategy, ignoring SSL and AL's inherent relation. Further, other methods suffer from high computational costs when dealing with large-scale, high-dimensional datasets. Motivated by the industry practice of labeling data, we first propose an innovativeInconsistency-based virtual aDvErsarialActiveLearning (IDEAL) algorithm to further investigate SSL-AL's potential superiority and achieve mutual enhancement of AL and SSL, i.e., SSL propagates label information to unlabeled samples and provides smoothed embeddings for AL, while AL excludes samples with inconsistent predictions and considerable uncertainty for SSL. We estimate unlabeled samples’ inconsistency by augmentation strategies of different granularities, including fine-grained continuous perturbation exploration and coarse-grained data transformations. Moreover, to solve the problem that the utilization efficiency of unlabeled samples is still insufficient in the process of semi-supervised training, we extend our IDEAL to a curriculum-guided version, namely SPL-IDEAL algorithm. The SPL-IDEAL algorithm can regularize the training process towards better regions in parameter space and denoise the pseudo labels with low confidence, achieving better performance. The extensive experiments, in both text and image benchmark datasets, validate the effectiveness of our proposed IDEAL and SPL-IDEAL algorithms, comparing them against state-of-the-art baselines. Two real-world case studies visualize the practical industrial value of applying and deploying the proposed data sampling algorithms.

User Satisfaction Estimation is an important task and increasingly being applied in goal-oriented dialogue systems to estimate whether the user is satisfied with the service. It is observed that whether the user’s needs are met often triggers various sentiments, which can be pertinent to the successful estimation of user satisfaction, and vice versa. Thus, User Satisfaction Estimation (USE) and Sentiment Analysis (SA) should be treated as a joint, collaborative effort, considering the strong connections between the sentiment states of speakers and the user satisfaction. Existing joint learning frameworks mainly unify the two highly pertinent tasks over cascade or shared-bottom implementations, however they fail to distinguish task-specific and common features, which will produce sub-optimal utterance representations for downstream tasks. In this paper, we propose a novel Speaker Turn-Aware Multi-Task Adversarial Network (STMAN) for dialogue-level USE and utterance-level SA. Specifically, we first introduce a multi-task adversarial strategy which trains a task discriminator to make utterance representation more task-specific, and then utilize a speaker-turn aware multi-task interaction strategy to extract the common features which are complementary to each task. Extensive experiments conducted on two real-world service dialogue datasets show that our model outperforms several state-of-the-art methods.

Verifying the facts alleged by the prosecutors before the trial requires the judges to retrieve evidence within the massive materials accompanied. Existing Legal AI applications often assume the facts are already determined and fail to notice the difficulty of reconstructing them. To build a practical Legal AI application and free the judges from the manually searching work, we introduce the task of Legal Evidence Retrieval, which aims at automatically retrieving the precise fact-related verbal evidence within a single case. We formulate the task in a dense retrieval paradigm, and jointly learn the constrastive representations and alignments between facts and evidence. To get rid of the tedious annotations, we construct an approximated positive vector for a given fact by aggregating a set of evidence from the same case. An entropy-based denoise technique is further applied to mitigate the impact of false positive samples. We train our models on tens of thousands of unlabeled cases and evaluate them on a labeled dataset containing 919 cases and 4,336 queries. Experimental results indicate that our approach is effective and outperforms other state-of-the-art representation and retrieval models. The dataset and code are available at https://github.com/yaof20/LER.

Positive-Unlabeled (PU) Learning is a challenge presented by binary classification problems where there is an abundance of unlabeled data along with a small number of positive data instances, which can be used to address chronic disease screening problem. State-of-the-art PU learning methods have resulted in the development of various risk estimators, yet they neglect the differences among distinct populations. To address this issue, we present a novel Positive-Unlabeled Learning Tree (PUtree) algorithm. PUtree is designed to take into account communities such as different age or income brackets, in tasks of chronic disease prediction. We propose a novel approach for binary decision-making, which hierarchically builds community-based PU models and then aggregates their deliverables. Our method can explicate each PU model on the tree for the optimized non-leaf PU node splitting. Furthermore, a mask-recovery data augmentation strategy enables sufficient training of the model in individual communities. Additionally, the proposed approach includes an adversarial PU risk estimator to capture hierarchical PU-relationships, and a model fusion network that integrates data from each tree path, resulting in robust binary classification results. We demonstrate the superior performance of PUtree as well as its variants on two benchmarks and a new diabetes-prediction dataset.

Xurui Li, Yue Qin, Rui Zhu, Tianqianjin Lin, Yongming Fan, Yangyang Kang, Kaisong Song, Fubang Zhao, Changlong Sun, Haixu Tang, Xiaozhong Liu. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023.

Yiquan Wu, Siying Zhou, Yifei Liu, Weiming Lu, Xiaozhong Liu, Yating Zhang, Changlong Sun, Fei Wu, Kun Kuang. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023.

Legal Question Answering (LQA) is a promising artificial intelligence application with high practical value. A professional and effective legal question answering (QA) agent can assist in reducing the workload of lawyers and judges, and help to achieve judicial accessibility. However, the NLP community lacks a large-scale LQA dataset with high quality, making it difficult to develop practical data-driven LQA agents. To tackle this bottleneck, this work presents EQUALS, a well-annotated real-world dataset for lEgal QUestion Answering via reading Chinese LawS. EQUALS contains 6,914 {question, article, answer} triplets as well as a pool of articles of laws that covers 10 different collections of Chinese Laws. Questions and the corresponding answers in EQUALS are collected from a professional law consultation forum. More importantly, the exact spans of law articles are annotated by senior law students as the answers. In this way, we could assure the quality and professionalism of EQUALS. Furthermore, this work proposes a QA framework that encompasses a law article retrieval module and a machine reading comprehension module for extracting accurate answers from the law article. We conduct thorough experiments with representative baselines on EQUALS, and the results indicate that EQUALS is a challenging question answering task. To the best of our knowledge, EQUALS is the largest real-world LQA dataset which shall significantly promote the research of LQA tasks. The work has been open-sourced and is available at: https://github.com/andongBlue/EQUALS.

Legal judgment prediction (LJP) is a significant task in legal intelligence, which aims to assist the judges and determine the judgment result based on the case's fact description. The judgment result consists of law articles, charge, and prison term. The law articles serve as the basis for the charge and the prison term, which can be divided into two types, named as charge-related law article and term-related law article, respectively. Recently, many methods have been proposed and made tremendous progress in LJP. However, the existing methods only focus on the prediction of the charge-related law articles, ignoring the term-related law articles (e.g., laws about lenient treatment), which limits the performance in the prison term prediction. In this paper, following the actual legal process, we expand the law article prediction as a multi-label classification task that includes both the charge-related law articles and term-related law articles and propose a novel multi-law aware LJP (ML-LJP) method to improve the performance of LJP. Given the case's fact description, firstly, the label (e.g., law article and charge) definitions in the Code of Law are used to transform the representation of the fact into several label-specific representations and make the prediction of the law articles and the charge. To distinguish the similar content of different label definitions, contrastive learning is conducted in the training. Then, a graph attention network (GAT) is applied to learn the interactions among the multiple law articles for the prediction of the prison term. Since numbers (e.g., amount of theft and weight of drugs) are important for LJP but often ignored by conventional encoders, we design a corresponding number representation method to locate and better represent these effective numbers. Extensive experiments on real-world dataset show that our method achieves the best results compared to the state-of-the-art models, especially in the task of prison term prediction where ML-LJP achieves a 10.07% relative improvement over the best baseline.

Enterprise risk detection from financial documents (ERD) is known to be a key decision-making tool for a company that relies on the mass media. It would help those companies to prepare for potential risks in advance or prevent further deterioration of risks. However, ERD is usually hindered by its natural complexity and ambiguity. For a typical ERD task, the risk factors are considered to be multi-labeled while the inputs are mostly redundant. To overcome these difficulties, we proposed a hierarchical multi-task learning module ERD-NET. A novel article encoder is introduced to combine company information with the document’s relative information well. A hierarchical multi-task framework is also involved so that the main risk detection task could utilize the information learned from two easier auxiliary tasks. Our evaluation on the collected dataset shows that our proposed method outperformed the current state-of-art models.

Neural text ranking models have witnessed significant advancement and are increasingly being deployed in practice. Unfortunately, they also inherit adversarial vulnerabilities of general neural models, which have been detected but remain underexplored by prior studies. Moreover, the inherit adversarial vulnerabilities might be leveraged by blackhat SEO to defeat better-protected search engines. In this study, we propose an imitation adversarial attack on black-box neural passage ranking models. We first show that the target passage ranking model can be transparentized and imitated by enumerating critical queries/candidates and then train a ranking imitation model. Leveraging the ranking imitation model, we can elaborately manipulate the ranking results and transfer the manipulation attack to the target ranking model. For this purpose, we propose an innovative gradient-based attack method, empowered by the pairwise objective function, to generate adversarial triggers, which causes premeditated disorderliness with very few tokens. To equip the trigger camouflages, we add the next sentence prediction loss and the language model fluency constraint to the objective function. Experimental results on passage ranking demonstrate the effectiveness of the ranking imitation attack model and adversarial triggers against various SOTA neural ranking models. Furthermore, various mitigation analyses and human evaluation show the effectiveness of camouflages when facing potential mitigation approaches. To motivate other scholars to further investigate this novel and important problem, we make the experiment data and code publicly available.

Legal judgment prediction (LJP) is a fundamental task in legal AI, which aims to assist the judge to hear the case and determine the judgment.The legal judgment usually consists of the law article, charge, and term of penalty.In the real trial scenario, the judge usually makes the decision step-by-step: first concludes the rationale according to the case's facts and then determines the judgment.Recently, many models have been proposed and made tremendous progress in LJP, but most of them adopt an endto-end manner that cannot be manually intervened by the judge for practical use.Moreover, existing models lack interpretability due to the neglect of rationale in the prediction process.Following the judge's real trial logic, in this paper, we propose a novel Rationale-based Legal Judgment Prediction (RLJP) framework.In the RLJP framework, the LJP process is split into two steps.In the first phase, the model generates the rationales according to the fact description.Then it predicts the judgment based on the fact and the generated rationales.Extensive experiments on a real-world dataset show RLJP achieves the best results compared to the stateof-the-art models.Meanwhile, the proposed framework provides good interactivity and interpretability which enables practical use. Fact DescriptionAfter the hearing, the court held the facts as follows: on December 3, 2014, the defendant A went to victim B's house and said he wanted to borrow B's money.But B refused, so A pushed B to the ground, bound B with a red soft cloth belt, and robbed B's cash of 1100.After the case, the parents of the defendant A returned 1100 yuan to B. On December 7, 2014, the defendant A surrendered to the court. Rationale Charge RationaleFor the purpose of illegal possession, the defendant A forcibly robbed B's property by means of violence. Penalty RationaleAfter the case, the defendant A voluntarily surrendered and actively returned the stolen goods, and may be given a lighter punishment as appropriate.

While annotating decent amounts of data to satisfy sophisticated learning models can be cost-prohibitive for many real-world applications. Active learning (AL) and semi-supervised learning (SSL) are two effective, but often isolated, means to alleviate the data-hungry problem. Some recent studies explored the potential of combining AL and SSL to better probe the unlabeled data. However, almost all these contemporary SSL-AL works use a simple combination strategy, ignoring SSL and AL's inherent relation. Further, other methods suffer from high computational costs when dealing with large-scale, high-dimensional datasets. Motivated by the industry practice of labeling data, we propose an innovative Inconsistency-based virtual aDvErsarial Active Learning (IDEAL) algorithm to further investigate SSL-AL's potential superiority and achieve mutual enhancement of AL and SSL, i.e., SSL propagates label information to unlabeled samples and provides smoothed embeddings for AL, while AL excludes samples with inconsistent predictions and considerable uncertainty for SSL. We estimate unlabeled samples' inconsistency by augmentation strategies of different granularities, including fine-grained continuous perturbation exploration and coarse-grained data transformations. Extensive experiments, in both text and image domains, validate the effectiveness of the proposed algorithm, comparing it against state-of-the-art baselines. Two real-world case studies visualize the practical industrial value of applying and deploying the proposed data sampling algorithm.

In constructing a smart court, to provide intelligent assistance for achieving more efficient, fair, and explainable trial proceedings, we propose a full-process intelligent trial system (FITS). In the proposed FITS, we introduce essential tasks for constructing a smart court, including information extraction, evidence classification, question generation, dialogue summarization, judgment prediction, and judgment document generation. Specifically, the preliminary work involves extracting elements from legal texts to assist the judge in identifying the gist of the case efficiently. With the extracted attributes, we can justify each piece of evidence’s validity by establishing its consistency across all evidence. During the trial process, we design an automatic questioning robot to assist the judge in presiding over the trial. It consists of a finite state machine representing procedural questioning and a deep learning model for generating factual questions by encoding the context of utterance in a court debate. Furthermore, FITS summarizes the controversy focuses that arise from a court debate in real time, constructed under a multi-task learning framework, and generates a summarized trial transcript in the dialogue inspectional summarization (DIS) module. To support the judge in making a decision, we adopt first-order logic to express legal knowledge and embed it in deep neural networks (DNNs) to predict judgments. Finally, we propose an attentional and counterfactual natural language generation (AC-NLG) to generate the court’s judgment.

Is chatbot able to completely replace the human agent? The short answer could be – ``it depends...''. For some challenging cases, e.g., dialogue's topical spectrum spreads beyond the training corpus coverage, the chatbot may malfunction and return unsatisfied utterances. This problem can be addressed by introducing the Machine-Human Chatting Handoff (MHCH) which enables human-algorithm collaboration. To detect the normal/transferable utterances, we propose a Difficulty-Assisted Matching Inference (DAMI) network, utilizing difficulty-assisted encoding to enhance the representations of utterances. Moreover, a matching inference mechanism is introduced to capture the contextual matching features. A new evaluation metric, Golden Transfer within Tolerance (GT-T), is proposed to assess the performance by considering the tolerance property of the MHCH. To provide insights into the task and validate the proposed model, we collect two new datasets. Extensive experimental results are presented and contrasted against a series of baseline models to demonstrate the efficacy of our model on MHCH.

Identifying pornographic text online is practically useful to protect children from access to such adult content. However, some authors may intentionally avoid using sensitive words in their pornographic texts to take advantage of the lack of human audits. Without prior knowledge guidance, real semantics of such pornographic text is difficult to understand by existing methods due to its high context-sensitivity and heavy usage of figurative language, which brings huge challenges to the porn detection systems used in social media platforms. In this paper, we approach to the problem as a document-level porn identification task by locating and integrating sentence-level evidence and propose a novel Evidence-Aware Neural Porn Classification (eNPC) model. Specifically, we first propose a basic model which locates porn indicative sentences in the document with a multiple instance learning model, and then aggregate the sentence-level evidence to induce document label with self-attention mechanism. Moreover, we consider label dependencies within local context. Finally, we further enhance the sentence representation with prior knowledge produced by an automatic porn lexicon construction strategy. Extensive experimental results show that our model exhibits consistent superiority over competitors on two real-world Chinese novel datasets and an English story dataset.

Automatic chat summarization can help people quickly grasp important information from numerous chat messages. Unlike conventional documents, chat logs usually have fragmented and evolving topics. In addition, these logs contain a quantity of elliptical and interrogative sentences, which make the chat summarization highly context dependent. In this work, we propose a novel unsupervised framework called RankAE to perform chat summarization without employing manually labeled data. RankAE consists of a topic-oriented ranking strategy that selects topic utterances according to centrality and diversity simultaneously, as well as a denoising auto-encoder that is carefully designed to generate succinct but context-informative summaries based on the selected utterances. To evaluate the proposed method, we collect a large-scale dataset of chat logs from a customer service environment and build an annotated set only for model evaluation. Experimental results show that RankAE significantly outperforms other unsupervised methods and is able to generate high-quality summaries in terms of relevance and topic coverage.

In a customer service system, dialogue summarization can boost service efficiency by automatically creating summaries for long spoken dialogues in which customers and agents try to address issues about specific topics. In this work, we focus on topic-oriented dialogue summarization, which generates highly abstractive summaries that preserve the main ideas from dialogues. In spoken dialogues, abundant dialogue noise and common semantics could obscure the underlying informative content, making the general topic modeling approaches difficult to apply. In addition, for customer service, role-specific information matters and is an indispensable part of a summary. To effectively perform topic modeling on dialogues and capture multi-role information, in this work we propose a novel topic-augmented two-stage dialogue summarizer (TDS) jointly with a saliency-aware neural topic model (SATM) for topic-oriented summarization of customer service dialogues. Comprehensive studies on a real-world Chinese customer service dataset demonstrated the superiority of our method against several strong baselines.

Xiyan Fu, Yating Zhang, Tianyi Wang, Xiaozhong Liu, Changlong Sun, Zhenglu Yang. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

Conversation generation as a challenging task in Natural Language Generation (NLG) has been increasingly attracting attention over the last years. A number of recent works adopted sequence-to-sequence structures along with external knowledge, which successfully enhanced the quality of generated conversations. Nevertheless, few works utilized the knowledge extracted from similar conversations for utterance generation. Taking conversations in customer service and court debate domains as examples, it is evident that essential entities/phrases, as well as their associated logic and inter-relationships, can be extracted and borrowed from similar conversation instances. Such information could provide useful signals for improving conversation generation. In this paper, we propose a novel reading and memory framework called Deep Reading Memory Network (DRMN) which is capable of remembering useful information of similar conversations for improving utterance generation. We apply our model to two large-scale conversation datasets of justice and e-commerce fields. Experiments prove that the proposed model outperforms the state-of-the-art approaches.

Chatbot is increasingly thriving in different domains, however, because of unexpected discourse complexity and training data sparseness, its potential distrust hatches vital apprehension.Recently, Machine-Human Chatting Handoff (MHCH), predicting chatbot failure and enabling human-algorithm collaboration to enhance chatbot quality, has attracted increasing attention from industry and academia.In this study, we propose a novel model, Role-Selected Sharing Network (RSSN), which integrates both dialogue satisfaction estimation and handoff prediction in one multi-task learning framework.Unlike prior efforts in dialog mining, by utilizing local user satisfaction as a bridge, global satisfaction detector and handoff predictor can effectively exchange critical information.Specifically, we decouple the relation and interaction between the two tasks by the role information after the shared encoder.Extensive experiments on two public datasets demonstrate the effectiveness of our model.

The performance of computer vision models significantly improves with more labeled data. However, the acquisition of labeled data is limited by the high cost. To mitigate the reliance on large labeled datasets, active learning (AL) and semi-supervised learning (SSL) are frequently adopted. Although current mainstream methods begin to combine SSL and AL (SSL-AL) to excavate the diverse expressions of unlabeled samples, these methods’ fully supervised task models are still trained only with labeled data. Besides, these method’s SSL-AL frameworks suffer from mismatch problems. Here, we propose a graph-based SSL-AL framework to unleash the SSL task models’ power and make an effective SSL-AL interaction. In the framework, SSL leverages graph-based label propagation to deliver virtual labels to unlabeled samples, rendering AL samples’ structural distribution and boosting AL. AL finds samples near the clusters’ boundary to help SSL perform better label propagation by exploiting adversarial examples. The information exchange in the closed-loop realizes mutual enhancement of SSL and AL. Experimental results show that our method outperforms the state-of-the-art methods against classification and segmentation benchmarks.

Millions of trademarks were registered last year in China, and thousands of applications are submitted daily. A trademark must be unique in the category it belongs to. Therefore, each new trademark application needs to be checked against all the existing ones in its category. A trademark can be a text string (characters, words or phrases), a figure (symbol or design), or both. In this study, we focus on the textual trademark in Chinese, and propose a model for finding similar trademarks for a given one. This neural network model exploits the semantic, phonetic and visual similarities between two textual trademarks. We evaluated our model based on a dataset that were built from the real trademark application data. Our evaluation shows that the proposed model outperforms other approaches.

Legal judgment prediction(LJP) is an essential task for legal AI. While prior methods studied on this topic in a pseudo setting by employing the judge-summarized case narrative as the input to predict the judgment, neglecting critical case life-cycle information in real court setting could threaten the case logic representation quality and prediction correctness. In this paper, we introduce a novel challenging dataset from real courtrooms to predict the legal judgment in a reasonably encyclopedic manner by leveraging the genuine input of the case - plaintiff's claims and court debate data, from which the case's facts are automatically recognized by comprehensively understanding the multi-role dialogues of the court debate, and then learnt to discriminate the claims so as to reach the final judgment through multi-task learning. An extensive set of experiments with a large civil trial data set shows that the proposed model can more accurately characterize the interactions among claims, fact and debate for legal judgment prediction, achieving significant improvements over strong state-of-the-art baselines. Moreover, the user study conducted with real judges and law school students shows the neural predictions can also be interpretable and easily observed, and thus enhancing the trial efficiency and judgment quality.

Sentiment analysis in dialogues plays a critical role in dialogue data analysis. However, previous studies on sentiment classification in dialogues largely ignore topic information, which is important for capturing overall information in some types of dialogues. In this study, we focus on the sentiment classification task in an important type of dialogue, namely customer service dialogue, and propose a novel approach which captures overall information to enhance the classification performance. Specifically, we propose a topic-aware multi-task learning (TML) approach which learns topic-enriched utterance representations in customer service dialogue by capturing various kinds of topic information. In the experiment, we propose a large-scale and high-quality annotated corpus for the sentiment classification task in customer service dialogue and empirical studies on the proposed corpus show that our approach significantly outperforms several strong baselines.

Multi-role dialogue understanding comprises a wide range of diverse tasks such as question answering, act classification, dialogue summarization etc. While dialogue corpora are abundantly available, labeled data, for specific learning tasks, can be highly scarce and expensive. In this work, we investigate dialogue context representation learning with various types unsupervised pretraining tasks where the training objectives are given naturally according to the nature of the utterance and the structure of the multi-role conversation. Meanwhile, in order to locate essential information for dialogue summarization/extraction, the pretraining process enables external knowledge integration. The proposed fine-tuned pretraining mechanism is comprehensively evaluated via three different dialogue datasets along with a number of downstream dialogue-mining tasks. Result shows that the proposed pretraining mechanism significantly contributes to all the downstream tasks without discrimination to different encoders.

In the literature, existing studies always consider Aspect Sentiment Classification (ASC) as an independent sentence-level classification problem aspect by aspect, which largely ignore the document-level sentiment preference information, though obviously such information is crucial for alleviating the information deficiency problem in ASC. In this paper, we explore two kinds of sentiment preference information inside a document, i.e., contextual sentiment consistency w.r.t. the same aspect (namely intra-aspect sentiment consistency) and contextual sentiment tendency w.r.t. all the related aspects (namely inter-aspect sentiment tendency). On the basis, we propose a Cooperative Graph Attention Networks (CoGAN) approach for cooperatively learning the aspect-related sentence representation. Specifically, two graph attention networks are leveraged to model above two kinds of document-level sentiment preference information respectively, followed by an interactive mechanism to integrate the two-fold preference. Detailed evaluation demonstrates the great advantage of the proposed approach to ASC over the state-of-the-art baselines. This justifies the importance of the document-level sentiment preference information to ASC and the effectiveness of our approach capturing such information.

We propose a Semi-supervIsed GeNerative Active Learning (SIGNAL) model to address the imbalance, efficiency, and text camouflage problems of Chinese text spam detection task.A "self-diversity" criterion is proposed for measuring the "worthiness" of a candidate for annotation.A semi-supervised variational autoencoder with masked attention learning approach and a character variation graph-enhanced augmentation procedure are proposed for data augmentation.The preliminary experiment demonstrates the proposed SIGNAL model is not only sensitive to spam sample selection, but also can improve the performance of a series of conventional active learning models for Chinese spam detection task.To the best of our knowledge, this is the first work to integrate active learning and semisupervised generative learning for text spam detection.

Dynamic summarization on product aspects, as a newly proposed topic, is an important task in E-commerce for tracking and understanding the nature of products. This can benefit both customers and sellers in different downstream tasks, such as explainable recommendations. However, most existing research works focus on analyzing product static reviews but miss dynamic sentiment changes. In this paper, we propose an innovative multi-task model to sample neighbour products whose information is simultaneously utilized to generate product summarization. In detail, a reinforcement learning approach selects neighbour products from a group of seed products by considering their pairwise similarities calculated from user behaviors. Meanwhile, a generative model helps to summarize product aspects via product descriptive phrases and selected neighbour products' sentimental phrases. To the best of our knowledge, this is the first work that studies dynamic product summarization leveraging user behaviors instead of self-reviews. It means that the proposed approach can naturally address the cold-start scenario where few recent product reviews are available. Extensive experiments are conducted with real-world reviews plus behavior data to validate the proposed method against several strong alternatives.

07 2 8 22 0 2 02 !7" 2 #$% &' () (* + ' + ,+ -./0-12(.3 .456#$%&' (672' ($ 83 ' &$&$9% .,:6#$(4;2.,672' ($ ) (<' $($=(' >-% * ' + 5?3 ..@' (4+ .(6=A8BCDEFEGHIJGKIILMBINOPQREBMCSORTURTUMCVGWHTKEXTXTYEUBMBIN KEJZ[\HEU]ETUTMQ]JO BFTU^KIU^M_BKHGTIXTIMBINOPBIU^FJEOGDCFTIWHFEGMQ]JMBU `a3 8 1b8 ) (+ 2-:$* +/ -c5-$% * 6$,<' -(1-*/ % .@<'/ / -% -(+ d-3 <*c' + (-* *+ 2-$12' ->-@-(+ *./* -e,-(1-f + .f* -e,-(1-@.<-3*g -h 4h 6iA0jk$+ + -(+ ' .(6l.' (+ -%9-(-% $+ .%m-+c.% n*$(<0% $(* / .%@-% o + .-(2$(1-<'$3 .4,-1.(+ -(+4-(-% $+ ' .(hp2' 3 -1.(+-(+q,-(15$(<$11,% $15./ + -(* -% >-$*+ 2-@$r .%'(<' 1$+ .%*/ .%@.<-3+ % $' (' (46<' $3 .4,-3.4'1* 61$% % 5' (41% ' + ' 1$3' (/ .%@$+ ' .(/.%*.@-:$%+ ' 1,3 $%<.@$' (* 6$% -./ + -(' 4(.% - ' 1-$(<1.,%+<-&$+ -<' $3 .4,-$*-s$@:3 -* 61.@:$+ ' &3 -3 .4'1*1$(&-.&*-% >-< $1% .**<' / / -% -(+<' $3 .4,-'(* + $(1-* 6$(<+ 2' *' (f / .%@$+ ' .(1$(:%.>'<->' + $3->' <-(1-/ .%,++ -% f $(1-4-(-% $+ ' .(h)(+ 2' *:$:-% 6c-:% .:.* -$ (.>-3(-+ c.% n$% 12' + -1+ ,% -f7% .**7.:5m-+ f c.% n*g 006o+ .-s:3.%-+ 2-1,% % -(+<' $3 .41.(+ -s+$(<* ' @' 3 $%<' $3 .4,-'(* + $(1-* t3 .4'1$3 * + % ,1+ ,% -* ' @,3 + $(-.,* 3 5h us:-% ' @-(+ *c' + 2 + c.+ $* n* 61.,% +<-&$+ -$(<1,* + .@-%*-% >' 1-1.(+ -(+4-(-% $+ ' .(6:%.>-<+2$++ 2-:% .:.* -< $3 4.% ' + 2@ ' ** ,:-% ' .%+.-s'* + ' (4* + $+ -f ./f $% + 1.(+ -(+4-(-% $+ ' .(@.<-3 * h v w 8 12 b8 2 8*$(' @:.% + $(++ $* n' (m$+ ,% $3i$(4,$4-9-(-% $f + ' .(x6y 6<' $3 .4,-4-(-%$+ ' .(-@:.c-% *$c' <-* :-1+ % ,@./$::3 ' 1$+ ' .(*6* ,12$*12$+ &.+$(<1,* f + .@-%*-% >' 1-$,+ .@$+' .(h)(+ 2-:$* +/ -c5-$% * 6 &% -$n+ 2% .,42*'(<' $3 .4,-4-(-%$+ ' .(+-12(.3 .45/ .1,*-<.($* -% ' -*./* -e,-(1-f + .f* -e,-(1-@. -%-+$3 h 6z{|}o hj.% -% -1-(+ 3 56-sf + -% ($3n(.c3-<4-' *-@:3 .5-<+.-(2$(1-@.<-3:-% / .%@$(1-hp,-+$3 hg z{|~o i' ,-+$3 hg z{|o 1$($* * ' * +<' $3 .4,-4-(-%$+ ' .(&5,*' (4n(.c3-<4-+ % ' :3 -* hA' @' 3 $% 3 56i'-+$3 hg z{|~o $r :,% n$%-+$3 h g z{|o #,$(4-+$3 hg z{|o -<<5-+$3 hg z{|~o -s:3 .%-<.1,@-(+$*n(.c3-<4-<' * 1.>-% 5/ .%<'f $3 .4,-4-(-%$+ ' .(6$(<'$-+$3 hg z{|o --+$3 h g z{z{o 92$;>' (' (-r $<-+$3 hg z{|o l$% + 2$* $% $+ 2' $( -% 6,($/ / .%<$&3 -n(.c3 -<4-1.(*+ % ,1f + ' .($(<<-/-1+ ' >-<.@$' ($<$:+ $+ ' .(%-* + % ' 1++ 2-' % ,+ ' 3 ' ;$+ ' .(h7.:5f &$* -<4-(-% $+ ' .(@.<-3 *g ' (5$3 *-+$3 h 6 z{|9,-+$3 h 6z{|o2$>-&--(c' <-3 5$<.:+ -< ' (1.(+ -(+4-(-% $+ ' .(+$* n*$(<* 2.c&-+ + -%% -* ,3 + * 1.@:$% -<+ .*-e,-(1-f + .f* -e,-(1-@.<-3*c2-( / $1- .1$&,3 $% 5:% .&3-@h02$(n*+ .+ 2-' %($+ ,% -./3 ->-% $4' (4>.1$&,3 $% 5$(<1.(+-s+ <' * + % ' &,+ ' .(*/.%1.(+ -(+1.:56'+-($&3 -*+ .1.:5e,' % ' -*/ % .@+2-1,* + .@-%*c' 3 34-+* ' @' 3 $%% -* :.(* -*/ % .@+2-* + $/ / h ) +@.+ ' >$+ -*,*+ .&,' 3 <$@.<-3+2$+1$((.+.(3 5 1.:5+ 2-1.(+ -(+c' + 2' (+ 2-,::-%1.(+-s+./+ 2-+ $% 4-+<' $3 .4,-'(* + $(1-6&,+$3 * .3-$% (+ 2-* ' @' 3 $% :$+ + -% (*$1% .**<' / / -% -(+* ' @' 3 $%1$* -*./+ 2-+ $% 4-+ ' (*+ $(1-hA,12-s+ -% ($31.:51$(&-1%' + ' 1$3' (*.@-* 1-($% ' .*h 8** 2.c(' (' 4,% -h |6c-:% .:.* -+ c.<' / / -% f -(+n' (<*./1.:5@-12$(' * @*' (+ 2' ** + ,<571 8 bb2451.(+-s+ f <-:-(<-(+' (/ .%@$+ ' .(c'+ 2' ( + 2-+ $% 4-+<' $3 .4,-'(* + $(1-6$(<21 28 b245 3 .4'1f <-:-(<-(+1.(+-(+$1% .**<' / / -% -(+t A' @' 3 $% 7$* -* tx 0y h02' */ % $@-c.%n' *3 $&-3 -<$*7% .** f 7.:5m-+ c.% n*x 006y h8*-s-@:3 $%<' $3 .4,-<-:'1+ -<6r ,<4-*@$5% -:-$+g 2.% ' ;.(+ $31.:' - $3' <$+ -+ 2-:% .:.* -<@.<-3 6c--@f :3 .5+c.<' / / -% -(+<' $3 .4,-<$+$* -+ */ % .@+c..% f + 2.4.($3<.@$'(*f $(<

Yiquan Wu, Kun Kuang, Yating Zhang, Xiaozhong Liu, Changlong Sun, Jun Xiao, Yueting Zhuang, Luo Si, Fei Wu. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2020.

Implied semantics is a complex language act that can appear everywhere on the Cyberspace. The prevalence of implied spam texts, such as implied pornography, sarcasm, and abuse hidden within the novel, tweet, microblog, or review, can be extremely harmful to the physical and mental health of teenagers. The non-literal interpretation of the implied text is hard to be understood by machine models due to its high context-sensitivity and heavy usage of figurative language. In this study, inspired by human reading comprehension, we propose a novel, simple, and effective deep neural framework, called Skim and Intensive Reading Model (SIRM), for figuring out implied textual meaning. The proposed SIRM consists of three main components, namely the skim reading component, intensive reading component, and adversarial training component. N-gram features are quickly extracted from the skim reading component, which is a combination of several convolutional neural networks, as skim (entire) information. An intensive reading component enables a hierarchical investigation for both sentence-level and paragraph-level representation, which encapsulates the current (local) embedding and the contextual information (context) with a dense connection. More specifically, the contextual information includes the near-neighbor information and the skim information mentioned above. Finally, besides the common training loss function, we employ an adversarial loss function as a penalty over the skim reading component to eliminate noisy information (noise) arisen from special figurative words in the training data. To verify the effectiveness, robustness, and efficiency of the proposed architecture, we conduct extensive comparative experiments on an industrial novel dataset involving implied pornography and three sarcasm benchmarks. Experimental results indicate that (1) the proposed model, which benefits from context and local modeling and consideration of figurative language (noise), outperforms existing state-of-the-art solutions, with comparable parameter scale and running speed; (2) the SIRM yields superior robustness in terms of parameter size sensitivity; (3) compared with ablation and addition variants of the SIRM, the final framework is efficient enough.

Technological advancements have led to increasing availability of erotic literature and pornography novels online, which can be alluring to adolescence and children. Unfortunately, because of the inherent complexity of these indecent contents and training data sparseness, it is a challenging task to detect these readings in the Cyberspace while children can easily access them. In this study, we propose a novel framework, Joint Learning of Content and Human Attention (GoodMan), to identify indecent readings by augmenting natural language understanding models with large scale human reading behaviors (dwell time per page) on portable devices. From the text modeling viewpoint, the innovative joint attention trained by joint learning is employed to orchestrate the content attention and human behavior attention via the BiGRU. From the data augmentation perspective, various users' reading behaviors on the same text can generate considerable training instances with joint attention, which can be effective to address the cold start problem. We conduct an extensive set of experiments on an online ebook dataset (with human reading behaviors on portable devices). The experimental results show insights into the task and demonstrate the superiority of the proposed model against alternative solutions.

In the digital era, information retrieval, text/knowledge mining, and NLP techniques are playing increasingly vital roles in legal domain. While the open datasets and innovative deep learning methodologies provide critical potentials, in the legal-domain, efforts need to be made to transfer the theoretical/algorithmic models into the real applications to assist users, lawyers, judges and the legal professions to solve the real problems. The objective of this workshop is to aggregate studies/applications of text mining/retrieval and NLP automation in the context of classical/novel legal tasks, which address algorithmic, data and social challenges of legal intelligence. Keynote and invited presentations from industry and academic will be able to fill the gap between ambition and execution in the legal domain.

In the literature, existing studies on aspect sentiment classification (ASC) focus on individual non-interactive reviews.This paper extends the research to interactive reviews and proposes a new research task, namely Aspect Sentiment Classification towards Question-Answering (ASC-QA), for real-world applications.This new task aims to predict sentiment polarities for specific aspects from interactive QA style reviews.In particular, a high-quality annotated corpus is constructed for ASC-QA to facilitate corresponding research.On this basis, a Reinforced Bidirectional Attention Network (RBAN) approach is proposed to address two inherent challenges in ASC-QA, i.e., semantic matching between question and answer, and data noise.Experimental results demonstrate the great advantage of the proposed approach to ASC-QA against several state-of-the-art baselines.

Multi-role court debate is a critical component in a civil trial where parties from different camps (plaintiff, defendant, witness, judge, etc.) actively involved. Unlike other types of dialogue, court debate can be lengthy, and important information, with respect to the controversy focus(es), often hides within the redundant and colloquial dialogue data. Summarizing court debate can be a novel but significant task to assist judge to effectively make the legal decision for the target trial. In this work, we propose an innovative end-to-end model to address this problem. Unlike prior summarization efforts, the proposed model projects the multi-role debate into the controversy focus space, which enables high-quality essential utterance(s) extraction in terms of legal knowledge and judicial factors. An extensive set of experiments with a large civil trial dataset shows that the proposed model can provide more accurate and readable summarization against several alternatives in the multi-role court debate scene.

Aspect category detection is an essential task for sentiment analysis and opinion mining. However, the cost of categorical data labeling, e.g., label the review aspect information for a large number of product domains, can be inevitable but unaffordable. In this study, we propose a novel problem, cross-domain aspect category transfer and detection, which faces three challenges: various feature spaces, different data distributions, and diverse output spaces. To address these problems, we propose an innovative solution, Traceable Heterogeneous Graph Representation Learning (THGRL). Unlike prior text-based aspect detection works, THGRL explores latent domain aspect category connections via massive user behavior information on a heterogeneous graph. Moreover, an innovative latent variable "Walker Tracer" is introduced to characterize the global semantic/aspect dependencies and capture the informative vertexes on the random walk paths. By using THGRL, we project different domains' feature spaces into a common one, while allowing data distributions and output spaces stay differently. Experiment results show that the proposed method outperforms a series of state-of-the-art baseline models.

Zhuoren Jiang, Zhe Gao, Guoxiu He, Yangyang Kang, Changlong Sun, Qiong Zhang, Luo Si, Xiaozhong Liu. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

Kaisong Song, Lidong Bing, Wei Gao, Jun Lin, Lujun Zhao, Jiancheng Wang, Changlong Sun, Xiaozhong Liu, Qiong Zhang. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

Jingjing Wang, Changlong Sun, Shoushan Li, Jiancheng Wang, Luo Si, Min Zhang, Xiaozhong Liu, Guodong Zhou. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

Various types of target information have been considered in aspect-based sentiment analysis, such as entities and aspects. Existing research has realized the importance of targets and developed methods with the goal of precisely modeling their contexts via generating target-specific representations. However, all these methods ignore that these representations cannot be learned well due to the lack of sufficient human-annotated target-related reviews, which leads to the data sparsity challenge, a.k.a. cold-start problem here. In this paper, we focus on a more general multiple entity aspect-based sentiment analysis (ME-ABSA) task which aims at identifying the sentiment polarity of different aspects of multiple entities in their context. Faced with severe cold-start scenario, we develop a novel and extensible deep memory network framework with cold-start aware computational layers which use frequency-guided attention mechanism to accentuate on the most related targets, and then compose their representations into a complementary vector for enhancing the representations of cold-start entities and aspects. To verify the effectiveness of the framework, we instantiate it with a concrete context encoding method and then apply the model to the ME-ABSA task. Experimental results conducted on two public datasets demonstrate that the proposed approach outperforms state-of-the-art baselines on ME-ABSA task.

Recently, emotion detection in conversations becomes a hot research topic in the Natural Language Processing community. In this paper, we focus on emotion detection in multi-speaker conversations instead of traditional two-speaker conversations in existing studies. Different from non-conversation text, emotion detection in conversation text has one specific challenge in modeling the context-sensitive dependence. Besides, emotion detection in multi-speaker conversations endorses another specific challenge in modeling the speaker-sensitive dependence. To address above two challenges, we propose a conversational graph-based convolutional neural network. On the one hand, our approach represents each utterance and each speaker as a node. On the other hand, the context-sensitive dependence is represented by an undirected edge between two utterances nodes from the same conversation and the speaker-sensitive dependence is represented by an undirected edge between an utterance node and its speaker node. In this way, the entire conversational corpus can be symbolized as a large heterogeneous graph and the emotion detection task can be recast as a classification problem of the utterance nodes in the graph. The experimental results on a multi-modal and multi-speaker conversation corpus demonstrate the great effectiveness of the proposed approach.

It is an important and urgent research problem for decentralized eCommerce services, e.g., eBay, eBid, and Taobao, to detect illegal products, e.g., unclassified pornographic products. However, it is a challenging task as some sellers may utilize and change camouflaged text to deceive the current detection algorithms. In this study, we propose a novel task to dynamically locate the pornographic products from very large product collections. Unlike prior product classification efforts focusing on textual information, the proposed model, BerryPIcking TRee MoDel (BIRD), utilizes both product textual content and buyers' seeking behavior information as berrypicking trees. In particular, the BIRD encodes both semantic information with respect to all branches sequence and the overall latent buyer intent during the whole seeking process. An extensive set of experiments have been conducted to demonstrate the advantage of the proposed model against alternative solutions. To facilitate further research of this practical and important problem, the codes and buyers' seeking behavior data have been made publicly available1.

Various e-commerce platforms produce millions of transactions per day with many transaction disputes. This generates the demand for effective and efficient dispute resolutions for e-commerce transactions. This paper proposes a novel research task of Legal Dispute Judgment (LDJ) prediction for e-commerce transactions, which connects two yet isolated domains, e-commerce data mining and legal intelligence. Different from traditional legal intelligence with the focus on textual evidence of the dispute itself, the new research utilizes multiview information such as past behavior information of seller and buyer as well as textual evidence of the current transaction. The multiview dispute representation is integrated into an innovative multi-task learning framework for predicting the legal result. An extensive set of experiments with a large dispute case dataset collected from a world leading e-commerce platform shows that the proposed model can more accurately characterize a dispute case through buyer, seller, and transaction viewpoints for legal judgment prediction against several alternatives.

''User reviews” are becoming an essential component of e-commerce. When buyers write a negative or doubting review, ideally, the sellers need to quickly give a response to minimize the potential impact. When the number of reviews is growing at a frightening speed, there is an urgent need to build a response writing assistant for customer service providers. In order to generate high-quality responses, the algorithm needs to consume and understand the information from both the original review and the target product. The classical sequence-to-sequence (Seq2Seq) methods can hardly satisfy this requirement. In this study, we propose a novel deep neural network model based on the Seq2Seq framework for the review response generation task in e-commerce platforms, which can incorporate product information by a gated multi-source attention mechanism and a copy mechanism. Moreover, we employ a reinforcement learning technique to reduce the exposure bias problem. To evaluate the proposed model, we constructed a large-scale dataset from a popular e-commerce website, which contains product information. Empirical studies on both automatic evaluation metrics and human annotations show that the proposed model can generate informative and diverse responses, significantly outperforming state-of-the-art text generation models.

Question-Answer (QA) matching is a fundamental task in the Natural Language Processing community. In this paper, we first build a novel QA matching corpus with informal text which is collected from a product reviewing website. Then, we propose a novel QA matching approach, namely One vs. Many Matching, which aims to address the novel scenario where one question sentence often has an answer with multiple sentences. Furthermore, we improve our matching approach by employing both word-level and sentence-level attentions for solving the noisy problem in the informal text. Empirical studies demonstrate the effectiveness of the proposed approach to question-answer matching.

Chenlin Shen, Changlong Sun, Jingjing Wang, Yangyang Kang, Shoushan Li, Xiaozhong Liu, Luo Si, Min Zhang, Guodong Zhou. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 2018.