The relationship between the quality of a string, as judged by a human reader, and its probability, p(y) under a language model undergirds the development of better language models.For example, many popular algorithms for sampling from a language model have been conceived with the goal of manipulating p(y) to place higher probability on strings that humans deem of high quality (Fan et al., 2018;Holtzman et al., 2020).In this article, we examine the probability-quality relationship in language models explicitly aligned to human preferences, e.g., through reinforcement learning through human feedback.We show that, when sampling corpora from an aligned language model, there exists a trade-off between the strings' average reward and average log-likelihood under the prior language model, i.e., the same model before alignment with human preferences.We provide a formal treatment of this phenomenon and demonstrate how a choice of sampling adaptor allows for a selection of how much likelihood we exchange for the reward.https://github.com/tanyjnaaman/ probability-quality-paradox

We present ACL OCL, a scholarly corpus derived from the ACL Anthology to assist Open scientific research in the Computational Linguistics domain.Integrating and enhancing the previous versions of the ACL Anthology, the ACL OCL contributes metadata, PDF files, citation graphs and additional structured full texts with sections, figures, and links to a large knowledge resource (Semantic Scholar).The ACL OCL spans seven decades, containing 73K papers, alongside 210K figures.We spotlight how ACL OCL applies to observe trends in computational linguistics.By detecting paper topics with a supervised neural model, we note that interest in "Syntax: Tagging, Chunking and Parsing" is waning and "Natural Language Generation" is resurging.Our dataset is available from HuggingFace 1 .

Recently, character information has been successfully introduced into the encoder-decoder event detection model to relieve the trigger-word mismatch problem, thus achieving impressive results in the languages without natural delimiters (i.e., Chinese). However, it is introduced into the encoder or the decoder separately, which makes the advantage of character information not be captured and represented adequately for event detection. In this article, we proposed a novel method to model character information in both the encoding and decoding stages to advance the neural event detection model. In particular, the proposed method can encode both words and characters and predict their event types jointly and further leverage interactions between word and its characters to optimize the inference. Experimental results show that the proposed model outperforms previous event detection methods on the ACE2005 Chinese benchmark. We release our code at Github. 1

The task of event extraction contains subtasks including detections for entity mentions, event triggers and argument roles. Traditional methods solve them as a pipeline, which does not make use of task correlation for their mutual benefits. There have been recent efforts towards building a joint model for all tasks. However, due to technical challenges, there has not been work predicting the joint output structure as a single task. We build a first model to this end using a neural transition-based framework, incrementally predicting complex joint structures in a state-transition process. Results on standard benchmarks show the benefits of the joint model, which gives the best result in the literature.