This paper presents a fine-grained test suite for the language pair German–English. The test suite is based on a number of linguistically motivated categories and phenomena and the semi-automatic evaluation is carried out with regular expressions. We describe the creation and implementation of the test suite in detail, providing a full list of all categories and phenomena. Furthermore, we present various exemplary applications of our test suite that have been implemented in the past years, like contributions to the Conference of Machine Translation, the usage of the test suite and MT outputs for quality estimation, and the expansion of the test suite to the language pair Portuguese–English. We describe how we tracked the development of the performance of various systems MT systems over the years with the help of the test suite and which categories and phenomena are prone to resulting in MT errors. For the first time, we also make a large part of our test suite publicly available to the research community.

It is with great sadness that we report the passing of Martin Kay in August 2021. Martin was a pioneer and intellectual trailblazer in computational linguistics. He was also a close friend and colleague of many years.Martin was a polyglot undergraduate student of modern and medieval languages at Cambridge University, with a particular interest in translation. He was not (yet) a mathematician or engineer, but idle speculation in 1958 about the possibilities of automating the translation process led him to Margaret Masterman at the Cambridge Language Research Unit, and a shift to a long and productive career.In 1960 he was offered an internship with Dave Hays and the Linguistics Project at The RAND Corporation in California, another early center of research in our emerging discipline. He stayed at RAND for more than a decade, working on basic technologies that are needed for machine processing of natural language. Among his contributions during that period was the development of the first so-called chart parser (Kay 1967), a computationally effective mechanism for dealing systematically with linguistic dependencies that cannot be expressed in context-free grammars. The chart architecture could be deployed for language generation as well as parsing, an important property for Martin’s continuing interest in translation.It was during the years at RAND that Martin found his second calling, as a teacher of computational linguistics, initially at UCLA and then in many other settings. He was a gifted and entertaining speaker and lecturer, able to present complex material with clarity and precision. He took great pleasure in the interactions with his students and the role that he played in helping to advance their careers. He left RAND in 1972 to become a full-time professor and chair of the Computer Science Department at the University of California at Irvine.His time at Irvine was short-lived, as he was attracted back to an open-ended research environment. In 1974 he joined with Danny Bobrow, Ron Kaplan, and Terry Winograd to form the Language Understander project at the recently created Palo Alto Research Center (PARC) of the Xerox Corporation. The group took as a first goal the construction of a mixed-initiative dialog system using state-of-the-art components for knowledge representation and reasoning, language understanding, language production, and dialog management (Bobrow et al. 1977). Martin took responsibility for the language production module, which was initially based on the quite rudimentary technology of the time.That was the beginning of his focus on “reversible grammars,” grammatical rules and representations that could be applied to parse strings into their underlying syntactic representations but also convert underlying representations back to the strings that express them. He and his colleagues at PARC developed the idea of hierarchical attribute-value structures (feature/functional structures) as underlying representations that could be characterized by the primitive predicates of equality and unification. This insight took form in his Functional Unification Grammar (Kay 1979) and in Lexical Functional Grammar (Kaplan and Bresnan 1982), and it also surfaced in the design of Head Driven Phrase Structure Grammar (Pollard and Sag 1987).Reversibility, for translation as well as dialog, was also the motivation at PARC for developing the mathematical, linguistic, and computational concepts that led to the use of bi-directional finite-state transducers for phonological and morphological description (Kaplan and Kay 1994). This technology is still being applied to a wide variety of language processing problems. But for Martin translation was always a central theme, bracketed by his early article “The Proper Place of Men and Machines in Language Translation” (which circulated in research for quite some time before it was finally published [Kay 1997]) and his most recent book (Kay 2017).In 1985 Martin struck a new balance between his commitment to research and his love of teaching by officially dividing his time between his prestigious role as a Research Fellow at PARC and a professorship in the Linguistics Department at Stanford. In addition to his Stanford professorship, he also taught (1998–2014) as an Honorary Professor at Saarland University, offering one or two courses every year. During his stays in Germany, he also advised on ongoing research, and his lectures and discussions helped in the gradual integration of programs in linguistics, computational linguistics, and translation studies.Martin contributed in many other ways to international progress in computational linguistics. In the 1970s and later he was a mainstay lecturer in the International Summer Schools in Computational Linguistics in Italy, and the Nordic summer schools in Scandinavia (actually, he and his wife Iris hosted one Nordic summer school at their home in Menlo Park). He advised research organizations and projects in several countries. He was a specialist advisor to the German Ministry of Education and Research, a reviewer for the two largest European projects in automatic translation, Eurotra and Verbmobil, and a valued advisor for projects at the German Research Center for Artificial Intelligence (DFKI). He also served for many years as chairman of the International Committee for Computational Linguistics (ICCL).Martin received many honors during his lifetime. He is a past President of the Association for Computational Linguistics (ACL). In 2005 he received the ACL Lifetime Achievement Award (Kay 2006). He was awarded honorary doctorate degrees from the University of Gothenburg (1982) and the University of Geneva (2008). He was the recipient of the Okawa Prize in 2019.Martin’s quiet and modest style of personal interaction stood only in apparent contrast to his widely recognized fame as an intellectual leader. His impressive expertise in several disciplines and his diverse intellectual interests made him a wonderful conversation partner for colleagues and friends who were lucky enough to be able to spend time with him. All students and colleagues remember him as a gifted speaker who was able to captivate and convince his audience with excellent didactics, rhetorical sharpness, and his very own sense of humor.We also remember Martin’s wife, Iris Kay, who predeceased him by a few months. Iris was a warm and psychologically insightful figure who played a prominent role in the early social history of computational linguistics, when personal relationships were more immediate and so important. They will both be sorely missed.

Coreference resolution for event mentions enables extraction systems to process document-level information.Current systems in this area base their decisions on rich semantic features from various knowledge bases, thus restricting them to domains where such external sources are available.We propose a model for this task which does not rely on such features but instead utilizes sentential features coming from convolutional neural networks.Two such networks first process coreference candidates and their respective context, thereby generating latent-feature representations which are tuned towards event aspects relevant for a linking decision.These representations are augmented with lexicallevel and pairwise features, and serve as input to a trainable similarity function producing a coreference score.Our model achieves state-of-the-art performance on two datasets, one of which is publicly available.An error analysis points out directions for further research.

To alleviate the error propagation in the traditional pipelined models for Abstract Meaning Representation (AMR) parsing, we formulate AMR parsing as a joint task that performs the two subtasks: concept identification and relation identification simultaneously.To this end, we first develop a novel componentwise beam search algorithm for relation identification in an incremental fashion, and then incorporate the decoder into a unified framework based on multiple-beam search, which allows for the bi-directional information flow between the two subtasks in a single incremental model.Experiments on the public datasets demonstrate that our joint model significantly outperforms the previous pipelined counterparts, and also achieves better or comparable performance than other approaches to AMR parsing, without utilizing external semantic resources.

In this paper, we present a novel approach to joint word sense disambiguation (WSD) and entity linking (EL) that combines a set of complementary objectives in an extensible multi-objective formalism. During disambiguation the system performs continuous optimization to find optimal probability distributions over candidate senses. The performance of our system on nominal WSD as well as EL improves state-ofthe-art results on several corpora. These improvements demonstrate the importance of combining complementary objectives in a joint model for robust disambiguation.

The article demonstrates how generic parsers in a minimally supervised information extraction framework can be adapted to a given task and domain for relation extraction (RE). For the experiments, two parsers that deliver n-best readings are included: (1) a generic deep-linguistic parser (PET) with a largely hand-crafted head-driven phrase structure grammar for English (ERG); (2) a generic statistical parser (Stanford Parser) trained on the Penn Treebank. It will be shown how the estimated confidence of RE rules learned from the n-best parses can be exploited for parse reranking for both parsers. The acquired reranking model improves the performance of RE in both training and test phases with the new first parses. The obtained significant boost of recall does not come from an overall gain in parsing performance but from an application-driven selection of parses that are best suited for the RE task. Since the readings best suited for the successful extraction of rules and instances are often not the readings favoured by a regular parser evaluation, generic parsing accuracy actually decreases. The novel method for task-specific parse reranking does not require any annotated data beyond the semantic seed, which is needed anyway for the RE task.

This paper presents the extension of an existing mimimally supervised rule acquisition method for relation extraction by coreference resolution (CR). To this end, a novel approach to CR was designed and tested. In comparison to state-of-the-art methods for CR, our strategy is driven by the target semantic relation and utilizes domain-specific ontological and lexical knowledge in addition to the learned relation extraction rules. An empirical investigation reveals that newswire texts in our selected domains contain more coreferring noun phrases than prononimal coreferences. This means that existing methods for CR would not suffice and a semantic approach is needed. Our experiments show that the utilization of domain knowledge can boost CR. In our approach, the tasks of relation extraction and CR support each other. On the one hand, reference resolution is needed for the detection of arguments of the target relation. On the other hand, domain modelling for the IE task is used for semantic classification of the referring nouns. Moreover, the application of the learned relation extraction rules often narrows down the number of candidates for CR.

In this interactive presentation, a Chinese named entity and relation identification system is demonstrated.The domainspecific system has a three-stage pipeline architecture which includes word segmentation and part-of-speech (POS) tagging, named entity recognition, and named entity relation identitfication.The experimental results have shown that the average F-measure for word segmentation and POS tagging after correcting errors achieves 92.86 and 90.01 separately.Moreover, the overall average F-measure for 6 kinds of name entities and 14 kinds of named entity relations is 83.08% and 70.46% respectively.

We present an architecture for the integration of shallow and deep NLP components which is aimed at flexible combination of different language technologies for a range of practical current and future applications. In particular, we describe the integration of a high-level HPSG parsing system with different high-performance shallow components, ranging from named entity recognition to chunk parsing and shallow clause recognition. The NLP components enrich a representation of natural language text with layers of new XML meta-information using a single shared data structure, called the text chart. We describe details of the integration methods, and show how information extraction and language checking applications for realworld German text benefit from a deep grammatical analysis.

Linear precedence (LP) rules are widley used for stating word order principles. They have been adopted as constraints by HPSG but no encoding in the formalism has been provided. Since they only order siblings, they are not quite adequate, at least not for German. We propose a notion of LP constraints that applies to linguistically motivated branching domains such as head domains. We show a type-based encoding in an HPSG-style formalism that supports processing. The encoding can be achieved by a compilation step.

Strategies are proposed for combining different kinds of constraints in declarative grammars with a detachable layer of control information. The added control information is the basis for parametrized dynamically controlled linguistic deduction, a form of linguistic processing that permits the implementation of plausible linguistic performance models without giving up the declarative formulation of linguistic competence. The information can be used by the linguistic processor for ordering the sequence in which conjuncts and disjuncts are processed, for mixing depth-first and breadth-first search, for cutting off undesired derivations, and for constraint-relaxation.

Metagrammatical formalisms that combine context-free phrase structure rules and metarules (MPS grammars) allow concise statement of generalizations about the syntax of natural languages. Unconstrained MPS grammars, unfortunately, are not computationally "safe." We evaluate several proposals for constraining them, basing our assessment on computational tractability and explanatory adequacy. We show that none of them satisfies both criteria, and suggest new directions for research on alternative metagrammatical formalisms.

The partially free word order in German belongs to the class of phenomena in natural language that require a close interaction between syntax and pragmatics. Several competing principles, which are based on syntactic and on discourse information, determine the linear order of noun phrases. A solution to problems of this sort is a prerequisite for high-quality language generation. The linguistic framework of Generalized Phrase Structure Grammar offers tools for dealing with word order variation. Some slight modifications to the framework allow for an analysis of the German data that incorporates just the right degree of interaction between syntactic and pragmatic components and that can account for conflicting ordering statements.