Whereas there is a growing literature that probes neural language models to assess the degree to which they have latently acquired grammatical knowledge, little if any research has investigated their acquisition of discourse modeling ability.We address this question by drawing on a rich psycholinguistic literature that has established how different contexts affect referential biases concerning who is likely to be referred to next.The results reveal that, for the most part, the prediction behavior of neural language models does not resemble that of human language users.

We present a classification model that predicts the presence or omission of a lexical connective between two clauses, based upon linguistic features of the clauses and the type of discourse relation holding between them.The model is trained on a set of high frequency relations extracted from the Penn Discourse Treebank and achieves an accuracy of 86.6%.Analysis of the results reveals that the most informative features relate to the discourse dependencies between sequences of coherence relations in the text.We also present results of an experiment that provides insight into the nature and difficulty of the task.

As evaluations of computational linguistics technology progress toward higher-level interpretation tasks, the problem of determining alignments between system responses and answer key entries may become less straightforward. We present an extensive analysis of the alignment procedure used in the MUC-6 evaluation of information extraction technology, which reveals effects that interfere with the stated goals of the evaluation. These effects are shown to be pervasive enough that they have the potential to adversely impact the technology development process. These results argue strongly for the use of accurate alignment criteria in natural language evaluations, and for maintaining the independence of alignment criteria and mechanisms used to calculate scores.

We provide a general account of parallelism in discourse, and apply it to the special case of resolving possible readings for instances of VP ellipsis. We show how several problematic examples are accounted for in a natural and straightforward fashion. The generality of the approach makes it directly applicable to a variety of other types of ellipsis and reference.

It is claimed that a variety of facts concerning ellipsis, event reference, and interclausal coherence can be explained by two features of the linguistic form in question: (1) whether the form leaves behind an empty constituent in the syntax, and (2) whether the form is anaphoric in the semantics.It is proposed that these features interact with one of two types of discourse inference, namely Common Topic inference and Coherent Situation inference.The differing ways in which these types of inference utilize syntactic and semantic representations predicts phenomena for which it is otherwise difficult to account.

The temporal relations that hold between events described by successive utterances are often left implicit or underspecified. We address the role of two phenomena with respect to the recovery of these relations: (1) the referential properties of tense, and (2) the role of temporal constraints imposed by coherence relations. We account for several facets of the identification of temporal relations through an integration of these.

This paper presents a new model of anaphoric processing that utilizes the establishment of coherence relations between clauses in a discourse. We survey data that comprises a currently stalemated argument over whether VP-ellipsis is an inherently syntactic or inherently semantic phenomenon, and show that the data can be handled within a uniform discourse processing architecture. This architecture, which revises the dichotomy between ellipsis vs. Model Interpretive Anaphora given by Sag and Hankamer (1984), is also able to accommodate divergent theories and data for pronominal reference resolution. The resulting architecture serves as a baseline system for modeling the role of cohesive devices in natural language.

NLP systems, in order to be robust, must handle novel and ill-formed input. One common type of error involves the use of non-standard prepositions to mark arguments. In this paper, we argue that such errors can be handled in a systematic fashion, and that a system designed to handle them offers other advantages. We offer a classification scheme for preposition usage errors. Further, we show how the knowledge representation employed in the SRA NLP system facilitates handling these data.