Visual questions are often ambiguous: the same image-question pair may admit multiple valid answers depending on which region is referenced.However, current Visual Question Answering (VQA) systems typically collapse this ambiguity, committing to a single interpretation during decoding and evaluation.In this work, we study visual question ambiguity from a grounded, region-centric perspective.We operationalize ambiguity as the existence of multiple distinct answer-supporting regions in an image, each independently yielding a valid answer.This formulation makes ambiguity observable without requiring exhaustive multi-answer annotations.Based on this definition, we conduct a systematic empirical study of state-of-the-art Visual Large Language Models (VLLMs).We find that, under default decoding, VLLMs consistently under-report ambiguity-even when multiple valid visual groundings are present.Importantly, probing model hidden states reveals that ambiguity-related signals are already encoded in their internal representations, despite not being reliably expressed in outputs.Finally, we show that selectively activating multi-focus answering based on these signals can recover additional valid answers while avoiding excessive hallucination.Together, our results suggest that ambiguity in VQA is not merely an annotation artifact or capability limitation, but a property that VLLMs internally recognize yet often fail to surface under standard decoding assumptions.

Event Argument Extraction (EAE) aims to identify event arguments and assign semantic roles under a predefined schema.Recent work formulates EAE with large language models as a structured conditional generation task and applies Reinforcement Learning with Verifiable Rewards (RLVR) to optimize sequence-level event structures.However, RLVR-based EAE supervision is coarse-grained, as a single reward is assigned to the whole event structure, while optimization happens at the token level.This misalignment causes the same reward to be applied to all tokens, including those not related to event roles or arguments, introducing noise into the gradient updates and weakening the signals for decisions critical to argument extraction.To mitigate this misalignment, we propose Hybrid Token Masking RLVR (HTMR), which selectively updates policy gradients on both high-entropy forking tokens and eventcritical tokens that define event structure, along with multi-perspective reasoning.Experiments across multiple benchmarks and models show that HTMR consistently outperforms full-token and high-entropy only RLVR methods.Moreover, HTMR transfers effectively as a plug-andplay approach to other tasks such as named entity recognition and relation classification.The code is publicly available for reproducibility.

Recent advances in auto-regressive transformers have achieved remarkable success in generative modeling. However, text-to-3D generation remains challenging, primarily due to bottlenecks in learning discrete 3D representations. Specifically, existing approaches often suffer from information loss during encoding, causing representational distortion before the quantization process. This effect is further amplified by vector quantization, ultimately degrading the geometric coherence of text-conditioned 3D shapes. Moreover, the conventional two-stage training paradigm induces an objective mismatch between reconstruction and text-conditioned auto-regressive generation. To address these issues, we propose View-aware Auto-Regressive 3D (VAR-3D), which intergrates a view-aware 3D Vector Quantized-Variational AutoEncoder (VQ-VAE) to convert the complex geometric structure of 3D models into discrete tokens. Additionally, we introduce a rendering-supervised training strategy that couples discrete token prediction with visual reconstruction, encouraging the generative process to better preserve visual fidelity and structural consistency relative to the input text. Experiments demonstrate that VAR-3D significantly outperforms existing methods in both generation quality and text-3D alignment.

Multimedia Event Extraction (MEE) aims to identify events and their arguments from documents that contain both text and images. It requires grounding event semantics across different modalities. Progress in MEE is limited by the lack of annotated training data. M2E2 is the only established benchmark, but it provides annotations only for evaluation. This makes direct supervised training impractical. Existing methods mainly rely on cross-modal alignment or inference-time prompting with Vision–Language Models (VLMs). These approaches do not explicitly learn structured event representations and often produce weak argument grounding in multimodal settings. To address these limitations, we propose RMPL, a Relation-aware Multi-task Progressive Learning framework for MEE under low-resource conditions. RMPL incorporates heterogeneous supervision from unimodal event extraction and multimedia relation extraction with stage-wise training. The model is first trained with a unified schema to learn shared event-centric representations across modalities. It is then fine-tuned for event mention identification and argument role extraction using mixed textual and visual data. Experiments on the M2E2 benchmark with multiple VLMs show consistent improvements across different modality settings.

Comparative Opinion Quintuple Extraction (COQE) aims to extract all comparative sentiment quintuples from product review text. Each quintuple comprises five elements: subject, object, aspect, opinion and preference. With the rise of Large Language Models (LLMs), existing work primarily focuses on enhancing the performance of COQE task through data augmentation, supervised fine-tuning and instruction tuning. Instead of the above pre-modeling and in-modeling design techniques, we focus on innovation in the post-processing. We introduce a model-unaware adaptive chain-of-feedback (COF) method from the perspective of inference feedback and extraction revision. This method comprises three core modules: dynamic example selection, self-critique and self-revision. By integrating LLMs, COF enables dynamic iterative self-optimization, making it applicable across different baselines. To validate the effectiveness of our approach, we utilize the outputs of two distinct baselines as inputs for COF: frozen parameters few-shot learning and the SOTA supervised fine-tuned model. We evaluate our approach on three benchmarks: Camera, Car and Ele. Experimental results show that, compared to the few-shot learning method, our approach achieves F1 score improvements of 3.51%, 2.65% and 5.28% for exact matching on the respective dataset. Even more impressively, our method further boosts performance, surpassing the current SOTA results, with additional gains of 0.76%, 6.54%, and 2.36% across the three datasets.

Clarification Question Generation (abbr., CQG) is crucial for ambiguous question answering. It produces the structured "clarification question" to reveal the intention and possible answers. Currently, the existing CQG approach is grounded on a pipeline mode, i.e., predicting possible answers first, and further performing CQG accordingly. This approach is applicable and obtains promising performance. However, it suffers from an unavoidable bottleneck that the distinctions among the answers are difficult to perceive, while such distinctions reveal significantly different intentions of questions. Without the ability of distinction perception and differentiation, the generator easily falls into the hallucination caused by distracting intentions. To address the issues, we construct an end-to-end CQG model using multitask learning. In particular, we propose an Answer-oriented Distinction Perception (ADP) approach to enhance the multi-task learning process. Specifically, ADP conducts comparison between a pair of possible answers, and instructs the Large Language Model (LLM) to summarize their distinction. We integrate ADP into the multi-task learning framework, progressively coupling it with possible answer generation and CQG to form different auxiliary tasks. The goal is to obtain the generalized distinction-aware CQG model. In our experiments, we use LLaMA3 as the backbone of CQG, and fine-tune it by multi-task learning. We leverage ChatGPT to produce the distinction descriptions among possible answers, and use them as observable evidence to fine-tune LLaMA3 for ADP. We evaluate our CQG model on the benchmark dataset CAmbigNQ. The test result shows that our ADP-based end-to-end CQG obtains substantial improvements compared to the pipeline CQG model. In addition, we apply our CQG model to the downstream ambiguous question answering task, and achieve an F1-score of 45.1% with an improvement of 5.9% at best (4.1% at worst).

To mitigate the need for large-scale labeled data and reduce the computational costs for adapting the model to new tasks, this study explores in-context learning (ICL) for aspect term extraction (ATE) with large language models (LLMs). We propose the dynamic sentence type prompting tailored for aspect term extraction tasks. Our hypothesis suggests that LLMs can learn task-specific aspects from demonstrations in ICL.We build upon the exploration of ICL for ATE and introduce a method for constructing explicit and sentence-type based demonstrations. This method shifts from arbitrary example selection to a systematic approach which highlights critical sentence types. We are inspired by human analogical reasoning and introduce the link-of-analogy prompting. This enables LLMs to generalize effectively to new ATE tasks. LLMs can draw analogies to known aspects which improves their performance on unseen categories. Meanwhile, the large language models can refine the extraction performance by iteratively refining the extracted results based on the specified criterion. We conduct experiments using the SemEval benchmark corpora, including R14-16 and L14, focusing on the domains of Restaurant and Laptop. Experiments demonstrate that our approach outperforms existing prompting and few-shot learning methods on token-level ATE datasets.

End-to-end automatic speech recognition (ASR) systems have made significant progress in general scenarios.However, it remains challenging to transcribe contextual named entities (NEs) in the contextual ASR scenario.Previous approaches have attempted to address this by utilizing the NE dictionary.These approaches treat entities as individual tokens and generate them token-by-token, which may result in incomplete transcriptions of entities.In this paper, we treat entities as indivisible wholes and introduce the idea of copying into ASR.We design a systematic mechanism called CopyNE, which can copy entities from the NE dictionary.By copying all tokens of an entity at once, we can reduce errors during entity transcription, ensuring the completeness of the entity.Experiments demonstrate that CopyNE consistently improves the accuracy of transcribing entities compared to previous approaches.Even when based on the strong Whisper, CopyNE still achieves notable improvements.

We tackle Event Argument Extraction (EAE) in the manner of template-based generation. Based on our exploration of generative EAE, it suffers from several issues, such as multiple arguments of one role, generating words out of context and inconsistency with prescribed format. We attribute it to the weakness of following complex input prompts. To address these problems, we propose the demonstration retrieval-augmented generative EAE (DRAGEAE), containing two components: event knowledge-injected generator (EKG) and demonstration retriever (DR). EKG employs event knowledge prompts to capture role dependencies and semantics. DR aims to search informative demonstrations from training data, facilitating the conditional generation of EKG. To train DR, we use the probability-based rankings from large language models (LLMs) as supervised signals. Experimental results on ACE-2005, RAMS and WIKIEVENTS demonstrate that our method outperforms all strong baselines and it can be generalized to various datasets. Further analysis is conducted to discuss the impact of diverse LLMs and prove that our model alleviates the above issues.

Event Detection (ED) is a task of automatically extracting multi-class trigger words. The understanding of word sense is crucial for ED. In this paper, we utilize context-specific commonsense knowledge to strengthen word sense modeling. Specifically, we leverage a Context-specific Knowledge Selector (CKS) to select the exact commonsense knowledge of words from a large knowledge base, i.e., ConceptNet. Context-specific selection is made in terms of the relevance of knowledge to the living contexts. On this basis, we incorporate the commonsense knowledge into the word-level representations before decoding. ChatGPT is an ideal generative CKS when the prompts are deliberately designed, though it is cost-prohibitive. To avoid the heavy reliance on ChatGPT, we train an offline CKS using the predictions of ChatGPT over a small number of examples (about 9% of all). We experiment on the benchmark ACE-2005 dataset. The test results show that our approach yields substantial improvements compared to the BERT baseline, achieving the F1-score of about 78.3%. All models, source codes and data will be made publicly available.

Aspect term extraction (ATE) is an important Natural Language Processing task, which aims to extract aspect terms from reviews. Recently, data augmentation has emerged as a reliable approach for relieving data sparsity in the NLP area. For ATE, self-labeling and semi-generation methods have been proposed to implement effective data augmentation. However, they either rely on external data or a pretrained generation model. In this paper, we propose a simple and self-contained augmentation method, which produces new instances for augmentation by context decoupling and infrequent term refilling, without using external data and generation models. We conduct experiments on four benchmark SemEval datasets. The test results show that our method yields substantial improvements, and performs comparably to the state-of-the-art method which uses external data.

Conversational Question Answering (CQA) aims to provide natural language answers to users in information-seeking dialogues.Existing CQA benchmarks often evaluate models using pre-collected human-human conversations.However, replacing the model-predicted dialogue history with ground truth compromises the naturalness and sustainability of CQA evaluation.While previous studies proposed using predicted history and rewriting techniques to address unresolved coreferences and incoherencies, this approach renders the question self-contained from the conversation.In this paper, we propose a novel automatic evaluation approach, interview evaluation.Specifically, ChatGPT acts as the interviewer (Q agent) with a set of carefully designed prompts, and the CQA model under test serves as the interviewee (A agent).During the interview evaluation, questions are dynamically generated by the Q agent to guide the A agent in predicting the correct answer through an interactive process.We evaluated four different models on QuAC and two models on CoQA in our experiments.The experiment results demonstrate that our interview evaluation has advantages over previous CQA evaluation approaches, particularly in terms of naturalness and coherence.The source code is made publicly available.

Event Detection (ED) is a challenging but valuable task. It aims to identify the words that trigger the events in text and classify them into pre-defined types. The previous works utilize entity information as supplementary clues for ED. However, word semantics in different contexts always have subtle variations. Capturing the connection between entity information and changing semantics is challenging. To tackle the issue, we propose the Mixture of Experts (MoE) technique with context clues to adaptively model semantics. We term the framework as CMoE. Concretely, the CMoE simultaneously performs entity and event detection with the MoE technique which are flexible components embedded transformer block. Furthermore, we apply multi-task learning to mine the shared knowledge between entity and event. We conduct experiments on the public ACE 2005 and KBP 2017 datasets. The results show that our model achieves competitive performance on ACE 2005 without using external knowledge, yielding an improvement of about 2.3% F1-score for ED. More importantly, our model outperforms all State-of-The-Art models on KBP 2017.

We tackle Multi-party Dialogue Reading Comprehension (abbr., MDRC). MDRC stands for an extractive reading comprehension task grounded on a batch of dialogues among multiple interlocutors. It is challenging due to the requirement of understanding cross-utterance contexts and relationships in a multi-turn multi-party conversation. Previous studies have made great efforts on the utterance profiling of a single interlocutor and graph-based interaction modeling. The corresponding solutions contribute to the answer-oriented reasoning on a series of well-organized and thread-aware conversational contexts. However, the current MDRC models still suffer from two bottlenecks. On the one hand, a pronoun like “it” most probably produces multi-skip reasoning throughout the utterances of different interlocutors. On the other hand, an MDRC encoder is potentially puzzled by fuzzy features, i.e., the mixture of inner linguistic features in utterances and external interactive features among utterances. To overcome the bottlenecks, we propose a coreference-aware attention modeling method to strengthen the reasoning ability. In addition, we construct a two-channel encoding network. It separately encodes utterance profiles and interactive relationships, so as to relieve the confusion among heterogeneous features. We experiment on the benchmark corpora Molweni and FriendsQA. Experimental results demonstrate that our approach yields substantial improvements on both corpora, compared to the fine-tuned BERT and ELECTRA baselines. The maximum performance gain is about 2.5%$F\mathbf{1}-\mathbf{score}$. Besides, our MDRC models outperform the state-of-the-art in most cases.

Generative commonsense question answering (GenCQA) is a task of automatically generating a list of answers given a question. The answer list is required to cover all reasonable answers. This presents the considerable challenges of producing diverse answers and ranking them properly. Incorporating a variety of closely-related background knowledge into the encoding of questions enables the generation of different answers. Meanwhile, learning to distinguish positive answers from negative ones potentially enhances the probabilistic estimation of plausibility, and accordingly, the plausibility-based ranking. Therefore, we propose a Knowledge Enhancement and Plausibility Ranking (KEPR) approach grounded on the Generate-Then-Rank pipeline architecture. Specifically, we expand questions in terms of Wiktionary commonsense knowledge of keywords, and reformulate them with normalized patterns. Dense passage retrieval is utilized for capturing relevant knowledge, and different PLM-based (BART, GPT2 and T5) networks are used for generating answers. On the other hand, we develop an ELECTRA-based answer ranking model, where logistic regression is conducted during training, with the aim of approximating different levels of plausibility in a polar classification scenario. Extensive experiments on the benchmark ProtoQA show that KEPR obtains substantial improvements, compared to the strong baselines. Within the experimental models, the T5-based GenCQA with KEPR obtains the best performance, which is up to 60.91% at the primary canonical metric Inc@3. It outperforms the existing GenCQA models on the current leaderboard of ProtoQA.

Leveraging external knowledge to enhance the reasoning ability is crucial for commonsense question answering. However, the existing knowledge bases heavily rely on manual annotation which unavoidably causes deficiency in coverage of world-wide commonsense knowledge. Accordingly, the knowledge bases fail to be flexible enough to support the reasoning over diverse questions. Recently, large-scale language models (LLMs) have dramatically improved the intelligence in capturing and leveraging knowledge, which opens up a new way to address the issue of eliciting knowledge from language models. We propose a Unified Facts Obtaining (UFO) approach. UFO turns LLMs into knowledge sources and produces relevant facts (knowledge statements) for the given question. We first develop a unified prompt consisting of demonstrations that cover different aspects of commonsense and different question styles. On this basis, we instruct the LLMs to generate question-related supporting facts for various commonsense questions via prompting. After facts generation, we apply a dense retrieval-based fact selection strategy to choose the best-matched fact. This kind of facts will be fed into the answer inference model along with the question. Notably, due to the design of unified prompts, UFO can support reasoning in various commonsense aspects (including general commonsense, scientific commonsense, and social commonsense). Extensive experiments on CommonsenseQA 2.0, OpenBookQA, QASC, and Social IQA benchmarks show that UFO significantly improves the performance of the inference model and outperforms manually constructed knowledge sources.

Question Generation (QG) aims to automatically raise questions given contexts. Due to the discrepancy between the training (supervised learning) and inference (autoregressive) modes, the current QG task suffers from exposure bias, i.e., the phenomenon of error propagation induced by mistakenly-generated tokens in the preceding subsequence. To address this issue, we propose to utilize moment matching to progressively alleviate exposure biases, which is referred to EBAMQ for short in this paper. EBAMQ enhances QG by imposing positive mitigation effects on the distributed representation. This is implemented by two components, including (1) adaptive moment matching and (2) assistant-cooperating data augmentation. Within the two components, adaptive moment matching plays the dominant role. It enables the sharing of feature space among different samples that derive from binary channels (i.e., training and inference). More importantly, it provides more similar representations for synonymous or homologous tokens. In addition, we introduce the pseudo-masked technique to generate additional reference questions. It contributes to the reduction of inevitable constraints of ground-truth instances, and therefore, it helps to improve the robustness. Extensive experiments are conducted on the benchmark dataset SQuAD. Experimental results show that EBAMD produces substantial improvements for all the metrics (BLEU-4, ROUGE-L and METEOR) compared to the UniLM-based baseline, and it outperforms the state-of-the-art QG models.

Question Generation (QG) models suffer from exposure bias. Exposure bias refers to the problem that an autoregressive generator is trained given the observable preceding context, though it runs during test conditioned on the self-generated context. Exposure bias unavoidably causes performance degradation. In this paper, we enhance the GAN-based SeqGAN model which slightly contributes to alleviating exposure bias. Instead of RNN and CNN, we leverage UniLMv2 to construct the generator and discriminator in SeqGAN. Different from RNN who generates the current token in terms of the closely adjacent token, UniLMv2 depends on all the preceding contexts to predict the current token. This reduces the risk of severe token-by-token error propagation caused by exposure bias. During the representation learning process, we retain the adversarial mechanism of SeqGAN that drives the generation of human-like languages, as well as the rewarding mechanism which reduces the loss if subsequent tokens are perfectly generated. Though, we carry out a straight rewording using N-grams. The experiments on the benchmark dataset SQuAD show that our model outperforms the state-of-the-art QG models for the metrics BLEU-4, ROUGE-L and METEOR. Besides, our model is less time-consuming compared to the original SeqGAN. The time consumption of 2nd-stage learning is 1h47m on GeForce RTX 2080 Ti.

We tackle Aspect Term Extraction (ATE), a task that automatically recognizes aspect terms conditioned on the under-standing of word-level semantics. Due to the capacity of enriching linguistic phenomena for learning, data augmentation contributes to the establishment of robust ATE models. In this paper, we propose to leverage back translation to augment the training data for ATE. It is grounded on the potential advantages that the back-translated instances generally appear as paraphrases, providing diverse pragmatic modes for learning when semantics remains unchanged. This helps to enhance ATE models in recognizing aspect terms when varied contexts and morphologically-different words occur during test. In our experiments, we apply an off-the-shelf Neural Machine Translation (NMT) model for back translation, using French, Chinese and German as interlanguages, respectively. Besides, word alignment is conducted to designate aspect terms in the back-translated cases. Experimental results on SemEval benchmarks show that retraining with the augmented data produces substantial improvements, reaching up to 3.46% at best. In addition, the experiments suggest that 1) family languages are more beneficial than non-family for the aforementioned data augmentation, and 2) selective sampling produces positive effects in the low-resource settings, It is noteworthy that back translation has been explored for data augmentation in other fields, with the aim to enhance neural language modeling. Nevertheless, it hasn't yet been systematically studied towards the ATE task. Although a vest-pocket method is provided in this paper, the comprehensive analysis is conducted, including that on interlanguage selection, low-resource application, as well as compatibility with both conventional and pretrained neural models, besides that in the common comparison and ablation experiments. All the models and codes in the experiments will be made publicly available to support reproducible research.

Comparative Opinion Quintuple Extraction (COQE) is a task of recognizing comparative relationships in a sentence-level comment. It is additionally required to extract the quintuple constituents that attribute to a specific comparative relation, including a subject and object, as well as comparative aspect, opinion and preference. Previous study employ pipeline models in addressing the COQE task, which tend to suffer from error propagation. To address the issue, in this paper, we propose a BERT-based end-to-end neural model as the alternative. Specifically, we first boil COQE down to a set prediction problem, due to the finding that all the quintuple constituents fail to hold coherent or sequential relationships. In other word, we consider a group of quintuple components as a set, and intent to bag-and-drag them as a whole, instead individually and one-by-one. Furthermore, we leverage Graph Convolutional Network (GCN) to enhance the end-to-end model, which plays the role of perceiving and representing the relevant relations among the quintuple components. We experiment on three benchmark datasets, including Camera-COQE, Car-COQE and Ele-COQE. The experimental results show that our model (GCN-E2E) yields a significant improvement in most cases, compared to the BERT-based pipeline baseline. The performance reaches the F1-scores of 14.10%, 36.46% and 39.29%, with the improvements of 0.74%, 6.71% and 8.56%.

We leverage cross-language data expansion and retraining to enhance neural Event Detection (abbr., ED) on English ACE corpus. Machine translation is utilized for expanding English training set of ED from that of Chinese. However, experimental results illustrate that such strategy actually results in performance degradation. The survey of translations suggests that the mistakenly-aligned triggers in the expanded data negatively influences the retraining process. We refer this phenomenon to “trigger falsification”. To overcome the issue, we apply heuristic rules for regulating the expanded data, fixing the distracting samples that contain the falsified triggers. The supplementary experiments show that the rule-based regulation is beneficial, yielding the improvement of about 1.6% F1-score for ED. We additionally prove that, instead of transfer learning from the translated ED data, the straight data combination by random pouring surprisingly performs better.

Training Neural Machine Translation (NMT) models suffers from sparse parallel data, in the infrequent translation scenarios towards low-resource source languages. The existing solutions primarily concentrate on the utilization of Parent-Child (PC) transfer learning. It transfers well-trained NMT models on high-resource languages (namely Parent NMT) to low-resource languages, so as to produce Child NMT models by fine-tuning. It has been carefully demonstrated that a variety of PC variants yield significant improvements for low-resource NMT. In this paper, we intend to enhance PC-based NMT by a bidirectionally-adaptive learning strategy. Specifically, we divide inner constituents (6 transformers) of Parent encoder into two “teams”, i.e., T1 and T2. During representation learning, T1 learns to encode low-resource languages conditioned on bilingual shareable latent space. Generative adversarial network and masked language modeling are used for space-shareable encoding. On the other hand, T2 is straightforwardly transferred to low-resource languages, and fine-tuned together with T1 for low-resource translation. Briefly, T1 and T2 take actions separately for different goals. The former aims to adapt to characteristics of low-resource languages during encoding, while the latter adapts to translation experiences learned from high-resource languages. We experiment on benchmark corpora SETIMES, conducting low-resource NMT for Albanian (Sq), Macedonian (Mk), Croatian (Hr) and Romanian (Ro). Experimental results show that our method yields substantial improvements, which allows the NMT performance to reach BLEU4-scores of 62.24%, 56.93%, 50.53% and 54.65% for Sq, Mk, Hr and Ro, respectively.

This paper proposes to cast end-to-end span-based SRL as a word-based graph parsing task. The major challenge is how to represent spans at the word level. Borrowing ideas from research on Chinese word segmentation and named entity recognition, we propose and compare four different schemata of graph representation, i.e., BES, BE, BIES, and BII, among which we find that the BES schema performs the best. We further gain interesting insights through detailed analysis. Moreover, we propose a simple constrained Viterbi procedure to ensure the legality of the output graph according to the constraints of the SRL structure. We conduct experiments on two widely used benchmark datasets, i.e., CoNLL05 and CoNLL12. Results show that our word-based graph parsing approach achieves consistently better performance than previous results, under all settings of end-to-end and predicate-given, without and with pre-trained language models (PLMs). More importantly, our model can parse 669/252 sentences per second, without and with PLMs respectively.

In this paper, we focus on the robustness evaluation of Chinese Question Matching (QM) models.Most of the previous work on analyzing robustness issues focus on just one or a few types of artificial adversarial examples.Instead, we argue that a comprehensive evaluation should be conducted on natural texts, which takes into account the fine-grained linguistic capabilities of QM models.For this purpose, we create a Chinese dataset namely DuQM which contains natural questions with linguistic perturbations to evaluate the robustness of QM models.DuQM contains 3 categories and 13 subcategories with 32 linguistic perturbations.The extensive experiments demonstrate that DuQM has a better ability to distinguish different models.Importantly, the detailed breakdown of evaluation by the linguistic phenomenon in DuQM helps us easily diagnose the strength and weakness of different models.Additionally, our experiment results show that the effect of artificial adversarial examples does not work on natural texts.Our baseline codes and a leaderboard are now publicly available.1

We tackle Aspect Term Extraction (ATE), an important natural language processing task that automatically identifies words of domain-specific aspects. Recently, a variety of sophisticated neural models and learning strategies have been explored for enhancing ATE, and significant improvements have been obtained. We intend to strengthen the neural models by involving features of Part-Of-Speech (POS) and syntactic dependency into the encoding process. It is motivated by the empirical findings that features of linguistic structure help to refine the understanding of semantics. Accordingly, we propose a stepwise encoding approach, where POS and syntactic dependency are successively leveraged step by step, including 1) joint encoding over both word and POS sequences using a pretrained language model; 2) BiGRU-based representational refinement conditioned on semantics-aware POS information and POS-aware semantic information; 3) representational augmentation by convolutional encoding of dependency graph. We conduct experiments on the four benchmark datasets of Semantic Evaluation (SemEval) for ATE. Experimental results show that our method obtains substantial improvements on all the considered datasets, and the performance ($F1$-score) reaches 87.69%, 89.76%, 77.94% and 83.96% for L-14, R-14, R-15 and R-16, respectively. All the models and source codes in the experiments will be made publicly available to support reproducible research.

Implicit discourse relation classification refers to a task of automatically determining relationships between arguments. It has been widely proven that, in a neural classification architecture, decoding discourse relations heavily relies on the reliable semantic representations of arguments. In addition, our previous survey shows that, for a target argument, the external semantic information hidden in the accompanying argument benefits the encoding of the target, either wholly or partially. Moreover, dependency structure appears as the crucial feature for synthesizing word senses of the entire words in arguments. Accordingly, we propose a novel method to enhance the current representation learning of pairwise arguments, which takes into consideration both external semantic information and internal dependency structure. In particular, we inject external semantic information into the Long-Short Term Memory (LSTM) unit of Recurrent Neural Network (RNN) through the input and forget gates. Different from the existing one-off interactive learning models, our method allows the neuronal memory of internal argument semantics to be affected by external information at each encoding step. On the basis, we apply the parser-based Graph Convolutional Networks (GCN) over the semantic presentations of words, so as to accumulate the closely-related semantic information in terms of dependency structures. We conduct experiments on Penn Discourse TreeBank Corpus of version 2.0 (PDTB 2.0). The test results illustrate that the proposed method enhances the baseline significantly, and it obtains comparable performance compared to the state of the art.

The goal of Event Detection (ED) task is to identify the words that mark the occurrence of events in text, and classify them into a set of event types. To model informative word semantics, some researchers apply Graph Convolutional Network (GCN) to exploit the syntactic graph transformed from the dependency tree, within one sentence. We are motivated to simultaneously leverage syntactic clues and context information across sentences. To this end, we propose a novel ED model with Cross-Sentence Graph Convolutional Networks (CSGCN). The CSGCN contains two main components, including a tree extension module and the syntax-aware graph convolution. Each sentence is parsed to a dependency tree by an automatic toolkit. The first module merges entity-specific subtrees from neighbor sentences into the dependency tree of current sentence, which constructs a cross-sentence dependency tree. On this basis, we transform the tree into an undirected graph. After that, a syntax-aware attention mechanism is employed in the computation of graph convolution. This mechanism dynamically captures syntax-relevant information from neighbor nodes via the graph structure. Finally, we devise an entity aggregation module to aggregate key entity information for trigger candidates. We conduct experiments on the ACE 2005 and KBP 2017 datasets. The results show that our model achieves satisfactory and competitive performance on ACE 2005, and outperforms all State-of-The-Art models on KBP 2017.

We tackle Open-Domain Conversational Question Answering (abbr., ODCQA), a task of answering questions in multi-turn conversations by mining clues from a large passage collection. Recent progress in deep learning and large-scale Pre-trained Language Model (PLM) is driving fast-paced advances in ODCQA. However, the typical retriever-reranker-reader (3R) pipeline framework of the existing ODCQA systems suffers from a potential flaw, which is referred to the unavoidable interference from the reader to reranker. Briefly, there are a large number of shareable parameters between the reader and reranker because they share the same PLM-based encoder. During the gradient backpropagation, their directions of updating PLM parameters are most probably inconsistent, which leads to interference in seeking for the optimal solution. In addition, the recent retriever in 3R framework merely utilizes dense representation. Though, dense retrievers are generally weaker than sparse retrievers in lexical matching for rare entities. To address the aforementioned two issues, we first propose to utilize a weak reader to alleviate the interference, and explore three different methods to pursue the goal, including (1) masking gradient, (2) dropping inputs and (3) early stopping. Besides, we propose to hybridize a semantic-sensitive dense retriever and a keyword -sensitive sparse retriever, so as to enhance the robustness of retriever in dealing with rare entities. We conduct extensive experiments on the benchmark corpus OR-QUAC. Experimental results show that our approach outperforms the State-of-The-Art (SoTA) models, yeilding an improvement of 6.1 % F1-score.

Aspect Term Extraction (ATE) is a natural language processing task, which identifies the languages describing product attributes. Such languages (words) are referred to aspect terms in this field. The current neural ATE models suffer from sparsity of available training data. As a result, they fail to be robust in real applications due to overfitting. More seriously, the distinguishable underlying features cannot be learned sufficiently by neural networks, which causes high misjudgement rates. Deliberate data expansion by human annotation undoubtedly helps to alleviate the problem. However, it is time-consuming. In order to overcome the bottleneck, we utilize the Regularized Dropout (R-Drop) approach to implement self data augmentation, creating variant distributed representations for learning in the real-time computation process of neural networks. More importantly, we propose to conduct contrastive learning over the self-augmented data, which sufficiently leverages the variant distributed representations to explore the distinguishable features. We experiment on four widely-used benchmark datasets (R14-16 and L14) in the shared tasks of Semantic Evaluation (SemEval). Experimental results show that contrastive learning over self-augmented data yields significant performance gains, where the improvement is up to 1.9% F1-score at best. In addition, it is demonstrated in the experiments that our method outperforms the state of the art on two test sets (R14 and R16), and meanwhile achieves competitive performance on the rest test sets.

We propose a novel evaluation method for Question Generation (QG) task. It is designed to verify the quality of the generated questions in terms of different references, including not only the manually-written questions (i.e., ground truth) but also their variants. Back translation is utilized to obtain the variants, and accordingly, they generally appear as paraphrases of the ground-truth examples. In particular, an Asymmetrical Twin Gain (ATG) is proposed for binary-perspective evaluation using the existing metrics, such as BLEU and ROUGE-L, respectively. It enables both the metrics to be observed from two perspectives, including the consistency between QG results and ground-truth examples, as well as that of variants. The experiments on the publicly-available benchmark SQuAD demonstrate the reliability of ATG. More importantly, ATG is proven effective for indicating the stable QG performance. It is noteworthy that the proposed binary-perspective evaluation is explored for assisting the conventional evaluation methods, instead of replacing them. The contribute can be identified as the additional insight into the robustness of QG when some slightly-different references (e.g., paraphrases) are offered for evaluation. All the models and source codes in the experiments will be made publicly available to support reproducible research.

The current aspect extraction methods suffer from boundary errors.These errors lead to a relatively minor difference between the extracted aspects and the ground-truth.However, they hurt the performance severely.In this paper, we propose to utilize a pointer network for repositioning the boundaries.Recycling mechanism is used which enables the training data to be collected without manual intervention.We conduct the experiments on the benchmark datasets SE14 of laptop and SE14-16 of restaurant.Experimental results show that our method achieves substantial improvements over the baseline, and outperforms stateof-the-art methods.

Reading comprehension (RC) on social media such as Twitter is a critical and challenging task due to its noisy, informal, but informative nature. Most existing RC models are developed on formal datasets such as news articles and Wikipedia documents, which severely limit their performances when directly applied to the noisy and informal texts in social media. Moreover, these models only focus on a certain type of RC, extractive or generative, but ignore the integration of them. To well address these challenges, we come up with a noisy user-generated text-oriented RC model. In particular, we first introduce a set of text normalizers to transform the noisy and informal texts to the formal ones. Then, we integrate the extractive and the generative RC model by a multi-task learning mechanism and an answer selection module. Experimental results on TweetQA demonstrate that our NUT-RC model significantly outperforms the state-of-the-art social media-oriented RC models.

We tackle implicit discourse relation recognition.Both self-attention and interactive-attention mechanisms have been applied for attention-aware representation learning, which improves the current discourse analysis models.To take advantages of the two attention mechanisms simultaneously, we develop a propagative attention learning model using a cross-coupled two-channel network.We experiment on Penn Discourse Treebank.The test results demonstrate that our model yields substantial improvements over the baselines (BiLSTM and BERT).

Longxiang Shen, Bowei Zou, Yu Hong, Guodong Zhou, Qiaoming Zhu, AiTi Aw. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

Due to the ability of encoding and mapping semantic information into a highdimensional latent feature space, neural networks have been successfully used for detecting events to a certain extent.However, such a feature space can be easily contaminated by spurious features inherent in event detection.In this paper, we propose a self-regulated learning approach by utilizing a generative adversarial network to generate spurious features.On the basis, we employ a recurrent network to eliminate the fakes.Detailed experiments on the ACE 2005 and TAC-KBP 2015 corpora show that our proposed method is highly effective and adaptable.

Event relation recognition is a challenging language processing task. It is required to determine the relation class of a pair of query events, such as causality, under the condition that there isn’t any reliable clue for use. We follow the traditional statistical approach in this paper, speculating the relation class of the target events based on the relation-class distributions on the similar events. There is minimal supervision used during the speculation process. In particular, we incorporate image processing into the acquisition of similar event instances, including the utilization of images for visually representing event scenes, and the use of the neural network based image matching for approximate calculation between events. We test our method on the ACE-R2 corpus and compared our model with the fully-supervised neural network models. Experimental results show that we achieve a comparable performance to CNN while slightly better than LSTM.

Relation Classification aims to classify the semantic relationship between two marked entities in a given sentence. It plays a vital role in a variety of natural language processing applications. Most existing methods focus on exploiting mono-lingual data, e.g., in English, due to the lack of annotated data in other languages. In this paper, we come up with a feature adaptation approach for cross-lingual relation classification, which employs a generative adversarial network (GAN) to transfer feature representations from one language with rich annotated data to another language with scarce annotated data. Such a feature adaptation approach enables feature imitation via the competition between a relation classification network and a rival discriminator. Experimental results on the ACE 2005 multilingual training corpus, treating English as the source language and Chinese the target, demonstrate the effectiveness of our proposed approach, yielding an improvement of 5.7% over the state-of-the-art.

We tackle discourse-level relation recognition, a problem of determining semantic relations between text spans.Implicit relation recognition is challenging due to the lack of explicit relational clues.The increasingly popular neural network techniques have been proven effective for semantic encoding, whereby widely employed to boost semantic relation discrimination.However, learning to predict semantic relations at a deep level heavily relies on a great deal of training data, but the scale of the publicly available data in this field is limited.In this paper, we follow Rutherford and Xue (2015) to expand the training data set using the corpus of explicitly-related arguments, by arbitrarily dropping the overtly presented discourse connectives.On the basis, we carry out an experiment of sampling, in which a simple active learning approach is used, so as to take the informative instances for data expansion.The goal is to verify whether the selective use of external data not only reduces the time consumption of retraining but also ensures a better system performance.Using the expanded training data, we retrain a convolutional neural network (CNN) based classifer which is a simplified version of Qin et al. (2016)'s stacking gated relation recognizer.Experimental results show that expanding the training set with small-scale carefully-selected external data yields substantial performance gain, with the improvements of about 4% for accuracy and 3.6% for F-score.This allows a weak classifier to achieve a comparable performance against the state-of-the-art systems.

Motivated by the critical importance of connectives in recognizing discourse relations, we present an unsupervised cross-argument inference mechanism to implicit discourse relation recognition. The basic idea is to infer the implicit discourse relation of an argument pair from a large number of comparable argument pairs, which are automatically retrieved from the web in an unsupervised way. In this way, the inference proceeds from explicit relations to implicit ones via connective as bridge. This kind of pair-to-pair inference is based on the assumption that two argument pairs with high content similarity (i.e. comparable argument pairs) should have similar discourse relationship. Evaluation on PDTB proves the effectiveness of our inference mechanism in implicit relation recognition to the four level-1 relations. It also shows that our mechanism significantly outperforms other alternatives.

Semi-supervised sentiment classification aims to train a classifier with a small number of labeled data (called seed data) and a large amount of unlabeled data. a big advantage of this approach is its saving of annotation effort by using the unlabeled data which is usually freely available. In this paper, we propose an approach to further minimize the annotation effort of semi-supervised sentiment classification by actively selecting the seed data. Specifically, a novel selection strategy is proposed to simultaneously select good words and documents for manual annotation by considering both of their annotation costs and informativeness. Experimental results demonstrate the effectiveness of our approach.

This paper focuses on exploring the features of product reviews that satisfy users, by which to improve the automatic helpfulness voting for the reviews on commercial websites. Compared to the previous work, which single-mindedly adopts the textual features to assess the review helpfulness, we propose that user preferences are more explicit clues to infer the opinions of users on the review helpfulness. By using the user-preference based features, we firstly implement a binary helpfulness based review classification system to divide helpful reviews and useless, and on the basis, we secondly build a Ranking SVM based automatic helpfulness voting system (AHV) which rank reviews based on their helpfulness. Experiments used a large scale dataset containing over 34,266 reviews on 1289 products to test the systems, which achieves promising performances with accuracy of up to 0.72 and [email protected] of 0.25, and at least 9% accuracy improvement compared to the textual-feature based helpfulness assessment.