Grammatical error correction systems improve written communication by detecting and correcting language mistakes.To help language learners better understand why the GEC system makes a certain correction, the causes of errors (evidence words) and the corresponding error types are two key factors.To enhance GEC systems with explanations, we introduce EXPECT, a large dataset annotated with evidence words and grammatical error types.We propose several baselines and analysis to understand this task.Furthermore, human evaluation verifies our explainable GEC system's explanations can assist second-language learners in determining whether to accept a correction suggestion and in understanding the associated grammar rule.

Our physical world is constantly evolving over time, rendering challenges for pre-trained language models to understand and reason over the temporal contexts of texts.Existing work focuses on strengthening the direct association between a piece of text and its time-stamp.However, the knowledge-time association is usually insufficient for the downstream tasks that require reasoning over temporal dependencies between knowledge.In this work, we make use of the underlying nature of time, all temporally-scoped sentences are strung together through a one-dimensional time axis, and suggest creating a graph structure based on the relative placements of events along the time axis.Inspired by the graph view, we propose REMEMO (Relative Time Modeling), which explicitly connects all temporally-scoped facts by modeling the time relations between any two sentences.Experimental results show that REMEMO outperforms the baseline T5 on multiple temporal question answering datasets under various settings.Further analysis suggests that REMEMO is especially good at modeling long-range complex temporal dependencies.We release our code and pretrained checkpoints at https://github.com/ DAMO-NLP-SG/RemeMo.

Thanks to the strong representation power of neural encoders, neural chart-based parsers have achieved highly competitive performance by using local features.Recently, it has been shown that non-local features in CRF structures lead to improvements.In this paper, we investigate injecting non-local features into the training process of a local span-based parser, by predicting constituent n-gram non-local patterns and ensuring consistency between non-local patterns and local constituents.Results show that our simple method gives better results than the self-attentive parser on both PTB and CTB.Besides, our method achieves state-of-the-art BERT-based performance on PTB (95.92 F1) and strong performance on CTB (92.31 F1).Our parser also achieves better or competitive performance in multilingual and zero-shot cross-domain settings compared with the baseline.

Meaning Representation (AMR) parsing aims to predict an AMR graph from textual input.Recently, there has been notable growth in AMR parsing performance.However, most existing work focuses on improving the performance in the specific domain, ignoring the potential domain dependence of AMR parsing systems.To address this, we extensively evaluate five representative AMR parsers on five domains and analyze challenges to cross-domain AMR parsing.We observe that challenges to cross-domain AMR parsing mainly arise from the distribution shift of words and AMR concepts.Based on our observation, we investigate two approaches to reduce the domain distribution divergence of text and AMR features, respectively.Experimental results on two out-of-domain test sets show the superiority of our method.

Sentiment analysis provides a useful overview of customer review contents.Many review websites allow a user to enter a summary in addition to a full review.Intuitively, summary information may give additional benefit for review sentiment analysis.In this paper, we conduct a study to exploit methods for better use of summary information.We start by finding out that the sentimental signal distribution of a review and that of its corresponding summary are in fact complementary to each other.We thus explore various architectures to better guide the interactions between the two and propose a hierarchically-refined review-centric attention model.Empirical results show that our review-centric model can make better use of user-written summaries for review sentiment analysis, and is also more effective compared to existing methods when the user summary is replaced with summary generated by an automatic summarization system.

Deep learning has led to significant improvement in text summarization with various methods investigated and improved ROUGE scores reported over the years.However, gaps still exist between summaries produced by automatic summarizers and human professionals.Aiming to gain more understanding of summarization systems with respect to their strengths and limits on a fine-grained syntactic and semantic level, we consult the Multidimensional Quality Metric 1 (MQM) and quantify 8 major sources of errors on 10 representative summarization models manually.Primarily, we find that 1) under similar settings, extractive summarizers are in general better than their abstractive counterparts thanks to strength in faithfulness and factual-consistency; 2) milestone techniques such as copy, coverage and hybrid extractive/abstractive methods do bring specific improvements but also demonstrate limitations; 3) pre-training techniques, and in particular sequence-to-sequence pre-training, are highly effective for improving text summarization, with BART giving the best results.