Event Argument Extraction (EAE) aims to identify arguments and assign them to predefined roles within a document. Existing methods face challenges in modeling intra-class variance and inter-class ambiguity, hindering accurate role assignment. Inspired by how humans dynamically adjust classification criteria while maintaining category consistency (e.g., distinguishing ''Victim'' and ''Attacker'' roles based on contextual relationships), we propose a HDMAR (Hyperspherical Dynamic Multi-Prototype with Arguments Dependencies and Role Consistency) method, where three innovations tackle these challenges: (1) Hyperspherical dynamic multi-prototype learning is used to capture intra-role diversity and enforce inter-role separation via hyperspherical optimization and optimal transport, (2) cross-event role consistency is used to align role representations across events, and (3) an arguments dependencies-guided encoding module enhances contextual understanding of intra-event and inter-event dependencies. Experiments on RAMS and WikiEvents demonstrate gains in accuracy, with further analysis validating the contributions of each module.

Role-playing systems powered by large language models (LLMs) have become increasingly influential in emotional communication applications. However, these systems are susceptible to character hallucinations, where the model deviates from predefined character roles and generates responses that are inconsistent with the intended persona. This paper presents the first systematic analysis of character hallucination from an attack perspective, introducing the RoleBreak framework. Our framework identifies two core mechanisms—query sparsity and role-query conflict—as key factors driving character hallucination. Leveraging these insights, we construct a novel dataset, RoleBreakEval, to evaluate existing hallucination mitigation techniques. Our experiments reveal that even enhanced models trained to minimize hallucination remain vulnerable to attacks. To address these vulnerabilities, we propose a novel defence strategy, the Narrator Mode, which generates supplemental context through narration to mitigate role-query conflicts and improve query generalization. Experimental results demonstrate that Narrator Mode significantly outperforms traditional refusal-based strategies by reducing hallucinations, enhancing fidelity to character roles and queries, and improving overall narrative coherence.

Empathy improves human-machine dialogue systems by enhancing the user’s experience. While traditional models have aimed to detect and express users’ emotions from dialogue history, they neglect the crucial and complex interactions among emotion, emotion causes, and commonsense. To address this, we introduce the ECC (Emotion, Cause, and Commonsense) framework, which leverages specialized encoders to capture the key features of emotion, cause, and commonsense and collaboratively models these through a Conditional Variational Auto-Encoder. ECC further employs novel loss functions to refine the interplay of three factors and generates empathetic responses using an energy-based model supported by ODE sampling. Empirical results on the EmpatheticDialogues dataset demonstrate that ECC outperforms existing baselines, offering a robust solution for empathetic dialogue generation.

Document-level event argument extraction (EAE) aims to recognize arguments involved in an event based on trigger throughout the document. According to event types, previous prompt-based researches focus on designing templates and promote the interaction between argument roles and the input through Pre-trained Language Models (PLMs). However, these methods hardly explore semantic structures of input documents, such as the trigger-argument information. In this paper, we inject Abstract Meaning Representation (AMR) graph from sentence and document perspectives into the prompt-based EAE model to leverage semantic information and prior knowledge simultaneously. We design the event inspired prompt template and construct the AMR graphs for the different-grained input, which contain relations between arguments and the trigger and help to bring semantic structure into the prompt-based EAE model. What’s more, we introduce a dual-level encoder for AMR graphs construction. It consists of a document-level encoder to capture cross-sentence arguments and a sentence-level encoder to reduce the probability of extracting incorrect role arguments. Comprehensive experimental results on two benchmarks show the effectiveness of our proposed approach.

Implicit discourse relation recognition (IDRR) is faced with a domain dilemma. Recent studies have achieved breakthroughs in standard datasets, while they are not appropriate in domains with insufficient data, such as bio-medicine. In this paper, we treat this problem as a cross-domain IDRR task, which transfers knowledge from the source domain to improve the understanding of the target domain. However, cross-domain IDRR is supervised and suffers from striking domain gaps, and general domain adaptation methods are not applicable. Therefore, we propose a Conditional-Balanced Adversarial Delta Tuning (CBADT) framework, which 1) leverages delta tuning to mine the dense domain-specific knowledge in low-resource scenarios; 2) builds a conditional adversarial verbalizer to inject domain-invariant knowledge into soft prompts; 3) proposes a domain label fusion method to balance domain-specific label words and fit the hybrid features from two domains. Experiments on both Chinese and English corpora demonstrate the transferability of our model.

Current research shows that providing instructions to guide Large Language Models (LLMs) improves reasoning tasks, but existing methods struggle to balance accuracy and generalization. Manually crafted instructions tailored to specific LLMs and tasks improve performance but reduce generalizability, while more general instructions lack detail and lower performance. To address this, we propose a dynamic instruction-generation method using an Instruction-Generation Prompt (IGP). IGP categorizes problems into domains and integrates the model's capabilities to generate detailed task-specific instructions, resulting in a comprehensive plan. This approach achieves high precision with general prompts without requiring in-depth knowledge of LLMs or tasks. We validated our method across five LLMs and ten datasets in three task categories. Our dynamically generated instructions outperformed traditionally handcrafted, LLM-specific instructions across various LLMs and tasks.

Word Sense Disambiguation (WSD) aims to determine the meaning of the target word according to the given context. Currently, a single representation enhanced by glosses from different dictionaries or languages is used to characterize each word sense. By analyzing the similarity between glosses of the same word sense, we find semantic biases among them, revealing that the glosses have their own descriptive perspectives. Therefore, the traditional approach of integrating all glosses by a single representation results in failing to present the unique semantics revealed by the individual glosses. In this paper, a quantum superposition state is employed to formalize the representations of multiple glosses of the same word sense to reveal their distributions. Furthermore, the quantum interference model is leveraged to calculate the probability that the target word belongs to this superposition state. The advantage is that the interference term can be regarded as a confidence level to guide word sense recognition. Finally, experiments are performed under standard WSD evaluation framework and the latest cross-lingual datasets, and the results verify the effectiveness of our model.

Social summarization aims to provide summaries for a large number of social texts (called posts) about a single topic. To extract a summary, both the representation of post and summary selection method are crucial. Previous methods introduce social relation to enhance post embedding to mitigate the sparse representation due to its brief and informal expression. However, they ignore that there are multiple relations between posts. Besides, existing graph-based centrality calculation approaches tend to select posts from one aspect. This leads to facet bias especially when there are multiple viewpoints. In this paper, we propose a model named MultiSum to improve social summarization. Specifically, 1) We use graph convolutional networks to fuse text content with social and semantic relations to improve post representation; 2) The similarity between the summary and all aspects is incorporated into the centrality score during the selection phase, encouraging the model to pay attention to different facets. Experimental results on English and Chinese corpora support the effectiveness of this model. Furthermore, external evaluations by human experts and large language models demonstrate the validity of MultiSum in facet coverage and redundancy reduction.

Emotion recognition in conversation (ERC) is a field that aims to classify the emotion of each utterance within conversational contexts. This presents significant challenges, particularly in handling emotional ambiguity across various speakers and contextual factors. Existing ERC approaches have primarily focused on modeling conversational contexts while incorporating only superficial speaker attributes such as names, memories, and interactions. Recent works introduce personality as an essential deep speaker factor for emotion recognition, but relies on static personality, overlooking dynamic variability during conversations. Advances in personality psychology conceptualize personality as dynamic, proposing that personality states can change across situations. In this paper, we introduce ERC-DP, a novel model considering the dynamic personality of speakers during conversations. ERC-DP accounts for past utterances from the same speaker as situation impacting dynamic personality. It combines personality modeling with prompt design and fine-grained classification modules. Through a series of comprehensive experiments, ERC-DP demonstrates superior performance on three benchmark conversational datasets.

Topic model is a statistical model that leverages unsupervised learning to mine hidden topics in document collections. The data sparsity and colloquialism of social texts make it difficult to accurately mine the topics. Traditional methods assume that there are only 0/1-state relationships between the two parties in the social networks, but the relationship status in real life is more complicated, such as continuously changing relationships with different degrees of intimacy. This paper proposes a continuous relational diffusion driven topic model (CRTM) with multi-grained text for microblog to realize the continuous representation of the relationship state and make up for the context and structural information lost by previous representation methods. Multi-grained text representation learning distinguishes the impact of formal and informal expression on the topics further and alleviates colloquialism problems. Specifically, based on the original social network, the reconstructed social network with continuous relationship status is obtained by using information diffusion technology. The graph convolution model is utilized to learn node embeddings through the new social network. Finally, the neural variational inference is applied to generate topics according to continuous relationships. We validate CRTM on three real datasets, and the experimental results show the effectiveness of the scheme.

Multi-level implicit discourse relation recognition (MIDRR) is a challenging task to recognize the hierarchical discourse relations between the arguments with the absence of connectives. Recent methods tend to incorporate the static hierarchical structure containing all senses (defined as global hierarchy) into prompt tuning through a path prompt template or hierarchical label refining. Howerver, hierarchical modeling is independent of the verbalizer, resulting in a failure to effectively utilize the output probability distribution information of verbalizer. Besides, they ignore the utilization of the dynamic hierarchical label sequence for each instance (defined as local hierarchy) in prompt tuning. In this paper, we propose a global and local hierarchical prompt tuning (GLHPT) framework, which utilize prior knowledge of PLMs while better incorporating hierarchical information from two aspects. We leverage bottom-up propagated probability as the global hierarchy to inject it into multi-level verbalizer (MLV). Furthermore, we design a local hierarchy-driven contrastive learning (LHCL) to improve the probability distribution of MLV. Finally, our model achieves competitive results on two benchmacks.

Prompt-based fine-tuning (PF), by aligning with the training objective of pre-trained language models (PLMs), has shown improved performance on many few-shot natural language understanding (NLU) benchmarks. However, the word embedding space of PLMs exhibits anisotropy, which is called the representation degeneration problem. In this paper, we explore the self-similarity within the same context and identify the anisotropy of the feature embedding space in PF model. Given that the performance of PF models is dependent on feature embeddings, we inevitably pose the hypothesis: this anisotropy limits the performance of the PF models. Based on our experimental findings, we propose CLMA, a Contrastive Learning framework based on the [MASK] token and Answers, to alleviate the anisotropy in the embedding space. By combining our proposed counter-intuitive SSD, a Supervised Signal based on embedding Distance, our approach outperforms mainstream methods on the many NLU benchmarks in the few-shot experimental settings. In subsequent experiments, we analyze the capability of our method to capture deep semantic cues and the impact of the anisotropy in the feature embedding space on the performance of the PF model.

While extensive work has examined the explicit and implicit biases in large language models (LLMs), little research explores the relation between these two types of biases. This paper presents a comparative study of the explicit and implicit biases in LLMs grounded in social psychology. Social psychology distinguishes between explicit and implicit biases by whether the bias can be self-recognized by individuals. Aligning with this conceptualization, we propose a self-evaluation-based two-stage measurement of explicit and implicit biases within LLMs. First, the LLM is prompted to automatically fill templates with social targets to measure implicit bias toward these targets, where the bias is less likely to be self-recognized by the LLM. Then, the LLM is prompted to self-evaluate the templates filled by itself to measure explicit bias toward the same targets, where the bias is more likely to be self-recognized by the LLM. Experiments conducted on state-of-the-art LLMs reveal human-like inconsistency between explicit and implicit occupational gender biases. This work bridges a critical gap where prior studies concentrate solely on either explicit or implicit bias. We advocate that future work highlight the relation between explicit and implicit biases in LLMs.

Personalized Dialogue Generation (PDG) aims to create coherent responses according to roles or personas.Traditional PDG relies on external role data, which can be scarce and raise privacy concerns.Approaches address these issues by extracting role information from dialogue history, which often fail to generically model roles in continuous space.To overcome these limitations, we introduce a novel framework MOdels Roles from Personalized Dialogue History by Exploring and Utilizing Latent Space (MOR-PHEUS) through a three-stage training process.Specifically, we create a persona codebook to represent roles in latent space compactly, and this codebook is used to construct a posterior distribution of role information.This method enables the model to generalize across roles, allowing the generation of personalized dialogues even for unseen roles.Experiments on both Chinese and English datasets demonstrate that MORPHEUS enhances the extraction of role information, and improves response generation without external role data.Additionally, MORPHEUS can be considered an efficient fine-tuning for large language models.* Morpheus excels at imitating others' forms, effortlessly embodying their physicality, posture, facial expressions, and vocal tone.He flawlessly embodies every intricate detail, including attire and unique mannerisms.

Data imbalance, also known as the long-tail distribution of data, is an important challenge for data-driven models. In the Word Sense Disambiguation (WSD) task, the long-tail phenomenon of word sense distribution is more common, making it difficult to effectively represent and identify Long-Tail Senses (LTSs). Therefore exploring representation methods that do not rely heavily on the training sample size is an important way to combat LTSs. Considering that many new states, namely superposition states, can be constructed from several known states in quantum mechanics, superposition states provide the possibility to obtain more accurate representations from inferior representations learned from a small sample size. Inspired by quantum superposition states, a representation method in Hilbert space is proposed to reduce the dependence on large sample sizes and thus combat LTSs. We theoretically prove the correctness of the method, and verify its effectiveness under the standard WSD evaluation framework and obtain state-of-the-art performance. Furthermore, we also test on the constructed LTS and the latest cross-lingual datasets, and achieve promising results.

Yihong Tang, Bo Wang, Miao Fang, Dongming Zhao, Kun Huang, Ruifang He, Yuexian Hou. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.

Multi-level implicit discourse relation recognition (MIDRR) aims at identifying hierarchical discourse relations among arguments.Previous methods achieve the promotion through finetuning PLMs.However, due to the data scarcity and the task gap, the pre-trained feature space cannot be accurately tuned to the task-specific space, which even aggravates the collapse of the vanilla space.Besides, the comprehension of hierarchical semantics for MIDRR makes the conversion much harder.In this paper, we propose a prompt-based Parameter-Efficient Multilevel IDRR (PEMI) framework to solve the above problems.First, we leverage parameterefficient prompt tuning to drive the inputted arguments to match the pre-trained space and realize the approximation with few parameters.Furthermore, we propose a hierarchical label refining (HLR) method for the prompt verbalizer to deeply integrate hierarchical guidance into the prompt tuning.Finally, our model achieves comparable results on PDTB 2.0 and 3.0 using about 0.1% trainable parameters compared with baselines and the visualization demonstrates the effectiveness of our HLR method.

For human-like dialogue systems, it is significant to inject the empathetic ability or elicit the opposite’s positive emotions, while existing studies mostly only focus on either of the above two research lines. In this work, we propose a novel and grafted task named Empathetic Emotion Elicitation Dialog to make a dialog system able to possess both aspects of ability simultaneously. We do not train an empathetic dialog system and an emotion elicitation dialog system separately and then simply concatenate the responses generated by these two systems, which will cause illogical and repetitive responses. Instead, we propose a unified solution: (1) To generate empathetic responses and emotion elicitation responses within the same semantic space, we design a unified framework. (2) The unified framework has three stages which first retrieve the empathetic and emotion elicitation exemplars as external knowledge, then fine-tune the emotion/action prediction on a pre-trained language model to enhance the empathetic ability, and finally model the user feedback by reinforcement learning to enhance the emotion elicitation ability. Experiments show that our method outperforms the baselines in the response generation quality and simultaneously empathizes with the user and elicits their positive emotions.

Implicit discourse relation recognition aims to understand and infer the relations between discourse arguments, which is a classification task. Existing models mostly model the interaction of discourse arguments from the perspective of a single relation, which may fail to learn the complicated interaction patterns between two arguments under different discourse relations. In this paper, we propose a novel neural approach based on capsule networks with dynamic interactive routing mechanism, named Interactive Capsule Networks (ICN) model to address the issues. It dynamically maps argument feature capsules to discourse relation capsules in an iterative refinement manner, which can mine the discourse relation-indicative semantic clues of argument features. Then, the designed dynamic interactive routing mechanism captures the more comprehensive argument interaction from the perspective of multiple relations. The experimental results on the Penn Discourse TreeBank (PDTB) demonstrate the effectiveness of our proposed ICN model.

The long-tailed distribution, also called the heavy-tailed distribution, is common in nature. Since both words and their senses in natural language have long-tailed phenomenon in usage frequency, the Word Sense Disambiguation (WSD) task faces serious data imbalance. The existing learning strategies or data augmentation methods are difficult to deal with the lack of training samples caused by the single application scenario of long-tail senses, and the word sense representations caused by unique word sense definitions. Considering that the features extracted from the Disentangled Representation (DR) independently describe the essential properties of things, and DR does not require deep feature extraction and fusion processes, it alleviates the dependence of the representation learning on the training samples. We propose a novel DR by constraining the covariance matrix of a multivariate Gaussian distribution, which can enhance the strength of independence among features compared to β-VAE. The WSD model implemented by the reinforced DR outperforms the baselines on the English all-words WSD evaluation framework, the constructed long-tail word sense datasets, and the latest cross-lingual datasets.

Target-oriented dialog aims to reach a global target through multi-turn conversation. The key to the task is the global planning towards the target, which flexibly guides the dialog concerning the context. However, existing target-oriented dialog works take a local and greedy strategy for response generation, where global planning is absent. In this work, we propose global planning for target-oriented dialog on a commonsense knowledge graph (KG). We design a global reinforcement learning with the planned paths to flexibly adjust the local response generation model towards the global target. We also propose a KG-based method to collect target-oriented samples automatically from the chit-chat corpus for model training. Experiments show that our method can reach the target with a higher success rate, fewer turns, and more coherent responses.

Conversational recommendation systems (CRS) aim to determine a goal item by sequentially tracking users’ interests through multi-turn conversation. In CRS, implicit patterns of user interest sequence guide the smooth transition of dialog utterances to the goal item. However, with the convenient explicit knowledge of knowledge graph (KG), existing KG-based CRS methods over-rely on the explicit separate KG links to model the user interests but ignore the rich goal-aware implicit interest sequence patterns in a dialog. In addition, interest sequence is also not fully used to generate smooth transited utterances. We propose CR-GIS with a parallel star framework. First, an interest-level star graph is designed to model the goal-aware implicit user interest sequence. Second, a hierarchical Star Transformer is designed to guide the multi-turn utterances generation with the interest-level star graph. Extensive experiments verify the effectiveness of CR-GIS in achieving more accurate recommended items with more fluent and coherent dialog utterances.

Human conversations of recommendation naturally involve the shift of interests which can align the recommendation actions and conversation process to make accurate recommendations with rich explanations.However, existing conversational recommendation systems (CRS) ignore the advantage of user interest shift in connecting recommendation and conversation, which leads to an ineffective loose coupling structure of CRS.To address this issue, by modeling the recommendation actions as recommendation paths in a knowledge graph (KG), we propose DICR (Dual Imitation for Conversational Recommendation), which designs a dual imitation to explicitly align the recommendation paths and user interest shift paths in a recommendation module and a conversation module, respectively.By exchanging alignment signals, DICR achieves bidirectional promotion between recommendation and conversation modules and generates high-quality responses with accurate recommendations and coherent explanations.Experiments demonstrate that DICR outperforms the state-of-theart models on recommendation and conversation performance with automatic, human, and novel explainability metrics.

Topic detection in social media is a challenging task due to the large-scale short and informal nature of messages. Existing methods choose to integrate structure or contexts among posts to alleviate the data sparsity of short text, however they ignore the spread characteristics of topics. A topic can spread in depth-wise and width-wise way on social networks. Specifically, wide dispersion gives priority to the paths that will spread to the low-order neighborhood, while deep propagation may spread to the more distant users along the propagation chains. Therefore, we propose a Dual-streams Graph convolution networks based Topic Model (DGTM) for microblog conversations. It aims to learn the contexts with full spread mode feature to alleviate data sparsity of microblog posts. Firstly, a user-level social network with posts information is built by the user interactions. Then, we design a dual-streams graph convolution networks module to learn the context-richer node embeddings by modeling spread characteristics. Lastly, the variational auto-encoder (VAE) is used to fuse the features of two spread modes and generate the more coherent topics. The experiments on three real-world datasets validate the effectiveness of our proposed model.

Social summarization aims to produce a concise summary that describes the core content of a collection of posts on a specific topic. Existing methods tend to produce sparse or ambiguous representations of posts due to only using short and informal text content. Latest researches use social relations to improve diversity of summaries, yet they model social relations as a regularization item, which has poor flexibility and generalization. Those methods could not embody the deep semantic and social interactions among posts, making summaries still suffer from redundancy. We propose to use Social Context and Multi-Granularity Relations (SCMGR) to improve unsupervised social summarization. It learns more informative representations of posts considering both text semantics and social structure information without any annotated data. First, we design two sociologically motivated meta-paths to construct a social context graph among posts, and adopt a graph convolutional network to aggregate social context information from neighbors. Second, we design a multi-granularity relation decoder to capture the deeper semantic and social interactions from post-word and post-post aspects respectively, which can provide guidance for summary selection from semantic and social structure perspectives. Finally, a sparse reconstruction-based extractor is used to select posts that can best reconstruct original content and social network structure as summaries. Our approach improves the coverage and diversity of summaries. Experimental results on both English and Chinese corpora prove the effectiveness of our model.

Although paths of user interests shift in knowledge graphs (KGs) can benefit conversational recommender systems (CRS), explicit reasoning on KGs has not been well considered in CRS, due to the complex of high-order and incomplete paths.We propose CRFR, which effectively does explicit multi-hop reasoning on KGs with a conversational context-based reinforcement learning model.Considering the incompleteness of KGs, instead of learning single complete reasoning path, CRFR flexibly learns multiple reasoning fragments which are likely contained in the complete paths of interests shift.A fragments-aware unified model is then designed to fuse the fragments information from item-oriented and concept-oriented KGs to enhance the CRS response with entities and words from the fragments.Extensive experiments demonstrate CRFR's SOTA performance on recommendation, conversation and conversation interpretability.

In machine learning, classification algorithms often use statistical methods to build the correspondence between features (or attributes) and categories (or labels), that is, the statistical correlation between features and categories. In quantum theory, a large number of experimental results show that quantum correlation is far stronger than what can be explained by local hidden theory (i.e., classical or non-quantum theory), that is, quantum mechanics theory reveals a statistical correlation stronger than that described by classical theory. Based on this, this paper will use the strong statistical correlation revealed by Bell state to build a classification algorithm to verify the validity and superiority of the formal framework of quantum mechanics in specific classification tasks. Specifically, we use quantum joint probabilities derived from the measurement process of Bell state to model the quantum statistical correlation between features and categories. The paper first theoretically proves that the formal framework used has the ability to violate Bell inequality; moreover, a classification algorithm is implemented and verified on classic machine learning datasets. Experimental results show that the algorithm is significantly better than most mainstream machine learning algorithms.

Quantum theory has attracted people's attention since it was proposed. Due to its unique advantages in information storage and processing, quantum information processing has become the most popular research field. Quantum theory also provides us with new methods or concepts for information manipulation and processing. The basic problems of classical physics are basically trying to be solved in a situation of isolation from the surrounding environment to reduce the complexity of the analysis problem, but the quantum system inevitably produces decoherence and establishes a close relationship with the environment such as entanglement, so the formal framework of quantum mechanics is inherently capable of depicting complex relationships. Based on the principle of quantum open system, a classifier under the formal framework of quantum mechanics is established to simulate the evolution process of open systems, that is, the interaction process between the target system and the environment. Specifically, we regard the features (or attributes) of the sample as environmental factors that affect the decision-making of the target system, and the target system can obtain the categories (or labels) of the sample through measurement. Based on this, we use the formal framework of quantum mechanics to establish a more natural and tighter correspondence between attributes and labels. Limited by the limitations of simulating quantum operations on classical computers, we conducted experiments on two lightweight machine learning datasets and compared them with mainstream classification algorithms. Experimental results show that the classification algorithm is better than the comparison models, and it also reveals the potential of the algorithm.

Social influence is essential to social recommendation. Current influence-based social recommendation focuses on the explicit influence on observed social links. However, in real cases, implicit social influence can also impact users' preference in an unobserved way. In this work, we concern two kinds of implicit influence: Local Implicit Influence of persons on unobserved interpersonal relations, and Global Implicit Influence of items broadcasted to users. We improve the state-of-the-art GNN-based social recommendation methods by modeling two kinds of implicit influences separately. Local implicit influence is involved by predicting unobserved social relationships. Global implicit influence is involved by defining global popularity of each item and personalize the impact of the popularity on each user. In a GCN network, explicit and implicit influence are integrated to learn the social embedding of users and items in social recommendation. Experimental results on Yelp initially demonstrate the effectiveness of proposed model.

Recognizing implicit discourse relation is a challenging task in discourse analysis, which aims to understand and infer the latent relations between two discourse arguments, such as temporal, comparison. Most of the present models largely focus on learning-based methods that utilize only intra-sentence textual information to identify discourse relations, ignoring the wider contexts beyond the discourse. Moreover, people comprehend the meanings and the relations of discourses, heavily relying on their interconnected working memories (e.g., instant memory, long-term memory). Inspired by this, we propose a Knowledge-Enhanced Attentive Neural Network (KANN) framework to address these issues. Specifically, it establishes a mutual attention matrix to capture the reciprocal information between two arguments, as instant memory. While implicitly stated knowledge in the arguments is retrieved from external knowledge source and encoded as inter-words semantic connection embeddings to further construct knowledge matrix, as long-term memory. We devise a novel paradigm with two ways by the collaboration of the memories to enrich the argument representation: 1) integrating the knowledge matrix into the mutual attention matrix, which implicitly maps knowledge into the process of capturing asymmetric interactions between two discourse arguments; 2) directly concatenating the argument representations and the semantic connection embeddings, which explicitly supplements knowledge to help discourse understanding. The experimental results on the PDTB also show that our KANN model is effective.

Implicit discourse relation recognition is a challenging task due to the lack of connectives as strong linguistic clues.Previous methods primarily encode two arguments separately or extract the specific interaction patterns for the task, which have not fully exploited the annotated relation signal.Therefore, we propose a novel TransS-driven joint learning architecture to address the issues.Specifically, based on the multi-level encoder, we 1) translate discourse relations in low-dimensional embedding space (called TransS), which could mine the latent geometric structure information of argumentrelation instances; 2) further exploit the semantic features of arguments to assist discourse understanding; 3) jointly learn 1) and 2) to mutually reinforce each other to obtain the better argument representations, so as to improve the performance of the task.Extensive experimental results on the Penn Discourse TreeBank (PDTB) show that our model achieves competitive results against several state-of-the-art systems.

Topic detection in social media is a challenging task due to large-scale short, noisy and informal nature of messages. Most existing methods only consider textual content or simultaneously model the posts and the first-order structural characteristics of social networks. They ignore the impact of larger neighborhoods in microblog conversations on topics. Moreover, the simple combination of separated content and structure representations fails to capture their nonlinear correlation and different importance in topic inference. To this end, we propose a novel random walk based Parallel Social Contexts Fusion Topic Model (PCFTM) for weibo conversations. Firstly, a user-level conversation network with content information is built by the reposting and commenting relationships among users. Through random walks of different lengths on network, we obtain the user sequences containing the parallel content and structure contexts, which are used to acquire the flexible-order proximity of users. Then we propose a self-fusion network embedding to capture the nonlinear correlation between parallel social contexts. It is achieved by taking the content embedding sequence processed by CNN as the initial value of structure embedding sequence fed to Bi-LSTM. Meanwhile, a user-level self-attention is further used to mine the different importance of users to topics. Lastly, the user sequence embedding is incorporated into neural variational inference for detecting topics, which adaptively balances the intrinsic complementarity between content and structure, and fully uses both local and global social contexts in topic inference. Extensive experiments on three real-world weibo datasets demonstrate the effectiveness of our proposed model.

With social media becoming popular, a vast of short and noisy messages are produced by millions of users when a hot event happens.Developing social summarization systems becomes more and more critical for people to quickly grasp core and essential information.However, the publicly available and high-quality large scale dataset under social media situation is rare.Constructing such corpus is not easy and very expensive since short texts have very complex social characteristics.Though there exist some datasets, they only consider the text on social media and ignore the potential user relations relevant signals on social network.In this paper, we construct TWEETSUM, a new event-oriented dataset for social summarization.The original data is collected from twitter and contains 12 real world hot events with a total of 44,034 tweets and 11,240 users.We create expert summaries for each event, and we also have the annotation quality evaluation.In addition, we collect additional social signals (i.e.user relations, hashtags and user profiles) and further establish user relation network for each event.To our knowledge, it is the first event-oriented social summarization dataset that contains social relationships.Besides the detailed dataset description, we show the performance of several typical extractive summarization methods on TWEETSUM to establish baselines.For further researches, we will release this dataset to the public.

Knowledge is essential for intelligent conversation systems to generate informative responses. This knowledge comprises a wide range of diverse modalities such as knowledge graphs (KGs), grounding documents and conversation topics. However, limited abilities in understanding language and utilizing different types of knowledge still challenge existing approaches. Some researchers try to enhance models’ language comprehension ability by employing the pre-trained language models, but they neglect the importance of external knowledge in specific tasks. In this paper, we propose a novel universal transformer-based architecture for dialogue system, the Multiple Knowledge Syncretic Transformer (MKST), which fuses multi-knowledge in open-domain conversation. Firstly, the model is pre-trained on a large-scale corpus to learn commonsense knowledge. Then during fine-tuning, we divide the type of knowledge into two specific categories that are handled in different ways by our model. While the encoder is responsible for encoding dialogue contexts with multifarious knowledge together, the decoder with a knowledge-aware mechanism attentively reads the fusion of multi-knowledge to promote better generation. This is the first attempt that fuses multi-knowledge in one conversation model. The experimental results have been demonstrated that our model achieves significant improvement on knowledge-driven dialogue generation tasks than state-of-the-art baselines. Meanwhile, our new benchmark could facilitate the further study in this research area.

Jinxin Chang, Ruifang He, Longbiao Wang, Xiangyu Zhao, Ting Yang, Ruifang Wang. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

Open-domain multi-turn dialogue systems are booming in human-machine interactions, which encourage to chat actively and freely in an intelligent natural way. Previous generative conversational models usually employ a single and deterministic encoder-decoder framework to model the semantic consistency between the context and corresponding response. However, they neglect the various dialog patterns (we denote the regularity of topic shifting as dialog pattern) in the conversations, leading to uninformative, non-initiative yet plausible responses. Although the existing variational methods have improved the response diversity to some extent by introducing a global variability into the generative process, they fail to simulate the transfer between topics with directional information due to the weak interpretability of the Gaussian-distributed latent variables. In this paper, we propose a novel Neural Variational Scaling Reasoning Network (NVSRN) for initiative response generation. To this end, our approach has two core ingredients: neural dialog pattern reasoner (reasoner) and topic scaling mechanism. Specifically, inspired by the advantage of von Mises-Fisher (vMF) distribution modeling the directional data (e.g., the topic transfer state), we employ it as the latent space of the reasoner to explore the regularity of topic shifting, which is then used to reason the topic of response. Based on this, a topic scaling mechanism is designed to control the transfer degree of topic in the response generator. The experimental results on two large dialog datasets demonstrate that the proposed model outperforms state-of-the-art baselines. The human evaluation shows the proposed model can produce more informative and initiative responses actively.

With the rapid growth of microblogging services, such as Twitter, a vast of short and noisy messages are produced by millions of users, which makes people difficult to quickly grasp essential information of their interested topics. In this paper, we study extractive topic-oriented Twitter summarization as a solution to address this problem. Traditional summarization methods only consider text information, which is insufficient in social media situation. Existing Twitter summarization techniques rarely explore relations between tweets explicitly, ignoring that information can spread along the social network. Inspired by social theories that expression consistence and expression contagion are observed in social network, we propose a novel approach for Twitter summarization in short and noisy situation by integrating Social Network and Sparse Reconstruction (SNSR). We explore whether social relations can help Twitter summarization, modeling relations between tweets described as the social regularization and integrating it into the group sparse optimization framework. It conducts a sparse reconstruction process by selecting tweets that can best reconstruct the original tweets in a specific topic, with considering coverage and sparsity. We simultaneously design the diversity regularization to remove redundancy. In particular, we present a mathematical optimization formulation and develop an efficient algorithm to solve it. Due to the lack of public corpus, we construct the gold standard twitter summary datasets for 12 different topics. Experimental results on this datasets show the effectiveness of our framework for handling the large scale short and noisy messages in social media.

Community detection has been extensively studied for various applications, focusing primarily on network topologies. Recent research has started to explore node contents to identify semantically meaningful communities and interpret their structures using selected words. However, links in real networks typically have semantic descriptions, e.g., comments and emails in social media, supporting the notion of communities of links. Indeed, communities of links can better describe multiple roles that nodes may play and provide a richer characterization of community behaviors than communities of nodes. The second issue in community finding is that most existing methods assume network topologies and descriptive contents to be consistent and to carry the compatible information of node group membership, which is generally violated in real networks. These methods are also restricted to interpret one community with one topic. The third problem is that the existing methods have used top ranked words or phrases to label topics when interpreting communities. However, it is often difficult to comprehend the derived topics using words or phrases, which may be irrelevant. To address these issues altogether, we propose a new unified probabilistic model that can be learned by a dual nested expectation-maximization algorithm. Our new method explores the intrinsic correlation between communities and topics to discover link communities robustly and extract adequate community summaries in sentences instead of words for topic labeling at the same time. It is able to derive more than one topical summary per community to provide rich explanations. We present experimental results to show the effectiveness of our new approach, and evaluate the quality of the results by a case study.

Implicit discourse relation recognition aims to understand and annotate the latent relations between two discourse arguments, such as temporal, comparison, etc. Most previous methods encode two discourse arguments separately, the ones considering pair specific clues ignore the bidirectional interactions between two arguments and the sparsity of pair patterns. In this paper, we propose a novel neural Tensor network framework with Interactive Attention and Sparse Learning (TIASL) for implicit discourse relation recognition. (1) We mine the most correlated word pairs from two discourse arguments to model pair specific clues, and integrate them as interactive attention into argument representations produced by the bidirectional long short-term memory network. Meanwhile, (2) the neural tensor network with sparse constraint is proposed to explore the deeper and the more important pair patterns so as to fully recognize discourse relations. The experimental results on PDTB show that our proposed TIASL framework is effective.

Traditional topic models are insufficient for topic extraction in social media. The existing methods only consider text information or simultaneously model the posts and the static characteristics of social media. They ignore that one discusses diverse topics when dynamically interacting with different people. Moreover, people who talk about the same topic have different effects on the topic. In this paper, we propose an Interaction-Aware Topic Model (IATM) for microblog conversations by integrating network embedding and user attention. A conversation network linking users based on reposting and replying relationship is constructed to mine the dynamic user behaviours. We model dynamic interactions and user attention so as to learn interaction-aware edge embeddings with social context. Then they are incorporated into neural variational inference for generating the more consistent topics. The experiments on three real-world datasets show that our proposed model is effective.

Dependency analysis is vital for spoken language understanding in spoken dialogue systems. However, existing research has mainly focused on western spoken languages, Japanese, and so on. Little research has been done for spoken Chinese in terms of dependency parsing. Therefore, the new spoken corpus, D-ESCSC (Dependency-Expressive Speech Corpus of Standard Chinese) is built by adding new dependency relations special to spoken Chinese based on a written Chinese annotation scheme. Since spoken Chinese contains typical ill-grammatical phenomena, e.g., translocation, repetition, duplication, and omission, the new atom feature related to punctuation and three feature templates are proposed to improve a graph-based dependency parser. Experimental results on spoken Chinese corpus show that the atom feature and three templates really work and the new parser outperforms the baseline parser. To our best knowledge, it is the first work to report dependency parsing results of spoken Chinese.

This paper tackles the task of event detection, which involves identifying and categorizing events. The previous work mainly exist two problems: (1) the traditional feature-based methods apply cross-sentence information, yet need taking a large amount of human effort to design complicated feature sets and inference rules; (2) the representation-based methods though overcome the problem of manually extracting features, while just depend on local sentence representation. Considering local sentence context is insufficient to resolve ambiguities in identifying particular event types, therefore, we propose a novel document level Recurrent Neural Networks (DLRNN) model, which can automatically extract cross-sentence clues to improve sentence level event detection without designing complex reasoning rules. Experiment results show that our approach outperforms other state-of-the-art methods on ACE 2005 dataset without external knowledge base.