The paper explores the performance of LLMs in the context of multi-dimensional analytic writing assessments, i.e. their ability to provide both scores and comments based on multiple assessment criteria.Using a corpus of literature reviews written by L2 graduate students and assessed by human experts against 9 analytic criteria, we prompt several popular LLMs to perform the same task under various conditions.To evaluate the quality of feedback comments, we apply a novel feedback comment quality evaluation framework.This framework is interpretable, cost-efficient, scalable, and reproducible, compared to existing methods that rely on manual judgments.We find that LLMs can generate reasonably good and generally reliable multi-dimensional analytic assessments.We release our corpus and code 1 for reproducibility.

Explicit linguistic knowledge, encoded by resources such as rule-based morphological analyzers, continues to prove useful in downstream NLP tasks, especially for low-resource languages and dialects.Rules are an important asset in descriptive linguistic grammars.However, creating such resources is usually expensive and non-trivial, especially for spoken varieties with no written standard.In this work, we present a novel approach for automatically learning morphophonological rules of Arabic from a corpus.Motivated by classic cognitive models for rule learning, rules are generalized cautiously.Rules that are memorized for individual items are only allowed to generalize to unseen forms if they are sufficiently reliable in the training data.The learned rules are further examined to ensure that they capture true linguistic phenomena described by domain experts.We also investigate the learnability of rules in low-resource settings across different experimental setups and dialects

Morphological inflection is a popular task in sub-word NLP with both practical and cognitive applications.For years now, state-of-theart systems have reported high, but also highly variable, performance across data sets and languages.We investigate the causes of this high performance and high variability; we find several aspects of data set creation and evaluation which systematically inflate performance and obfuscate differences between languages.To improve generalizability and reliability of results, we propose new data sampling and evaluation strategies that better reflect likely usecases.Using these new strategies, we make new observations on the generalization abilities of current inflection systems.

Modern work on the cross-linguistic computational modeling of morphological inflection has typically employed language-independent data splitting algorithms.In this paper, we supplement that approach with language-specific probes designed to test aspects of morphological generalization.Testing these probes on three morphologically distinct languages, English, Spanish, and Swahili, we find evidence that three leading morphological inflection systems employ distinct generalization strategies over conjugational classes and feature sets on both orthographic and phonologically transcribed inputs.

With the advent of powerful neural language models over the last few years, research attention has increasingly focused on what aspects of language they represent that make them so successful.Several testing methodologies have been developed to probe models' syntactic representations.One popular method for determining a model's ability to induce syntactic structure trains a model on strings generated according to a template then tests the model's ability to distinguish such strings from superficially similar ones with different syntax.We illustrate a fundamental problem with this approach by reproducing positive results from a recent paper with two non-syntactic baseline language models: an n-gram model and an LSTM model trained on scrambled inputs.

This paper describes a language-independent model for fully unsupervised morphological analysis that exploits a universal framework leveraging morphological typology.By modeling morphological processes including suffixation, prefixation, infixation, and full and partial reduplication with constrained stem change rules, our system effectively constrains the search space and offers a wide coverage in terms of morphological typology.The system is tested on nine typologically and genetically diverse languages, and shows superior performance over leading systems.We also investigate the effect of an oracle that provides only a handful of bits per language to signal morphological type.

This paper describes a new morphology resource created by Linguistic Data Consortium and the University of Pennsylvania for the DARPA LORELEI Program. The data consists of approximately 2000 tokens annotated for morphological segmentation in each of 9 low resource languages, along with root information for 7 of the languages. The languages annotated show a broad diversity of typological features. A minimal annotation scheme for segmentation was developed such that it could capture the patterns of a wide range of languages and also be performed reliably by non-linguist annotators. The basic annotation guidelines were designed to be language-independent, but included language-specific morphological paradigms and other specifications. The resulting annotated corpus is designed to support and stimulate the development of unsupervised morphological segmenters and analyzers by providing a gold standard for their evaluation on a more typologically diverse set of languages than has previously been available. By providing root annotation, this corpus is also a step toward supporting research in identifying richer morphological structures than simple morpheme boundaries.

Language variation and change are driven both by individuals' internal cognitive processes and by the social structures through which language propagates.A wide range of computational frameworks have been proposed to connect these drivers.We compare the strengths and weaknesses of existing approaches and propose a new analytic framework which combines previous network models' ability to capture realistic social structure with practically and more elegant computational properties.The framework privileges the process of language acquisition and embeds learners in a social network but is modular so that population structure can be combined with different acquisition models.We demonstrate two applications for the framework: a test of practical concerns that arise when modeling acquisition in a population setting and an application of the framework to recent work on phonological mergers in progress.

Generating the English transliteration of a name written in a foreign script is an important and challenging step in multilingual knowledge acquisition and information extraction.Existing approaches to transliteration generation require a large (>5000) number of training examples.This difficulty contrasts with transliteration discovery, a somewhat easier task that involves picking a plausible transliteration from a given list.In this work, we present a bootstrapping algorithm that uses constrained discovery to improve generation, and can be used with as few as 500 training examples, which we show can be sourced from annotators in a matter of hours.This opens the task to languages for which large number of training examples are unavailable.We evaluate transliteration generation performance itself, as well the improvement it brings to crosslingual candidate generation for entity linking, a typical downstream task.We present a comprehensive evaluation of our approach on nine languages, each written in a unique script. 1