Demo Data from SIGIR 2026 / reviewers in the wild / expert
Garrett Nicolai
dblp:51/5461
· DBLP profile ↗
16ranked-venue papers
7as first author
5since 2021 · last 2024
—ORCID · none
Domains — the database's venue-derived domains; a paper can count in several
Artificial intelligence and machine learning · 16 · 7 first-author · 5 since 2021
Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.
| Artificial intelligence
4 papers |
Language models and text generation · 45% Information extraction and text analysis · 38% Machine translation · 17% |
Topics — the 7 heaviest of 8, each with the papers that count most for it
| Topic | Weight | Papers | Last | Evidence papers |
|---|---|---|---|---|
Natural language and speech › Language models and text generation
low-resource languages |
0.8 | 1 | 2024 | Multiple Sources are Better Than One: Incorporating External Knowledge in Low-Resource Glossing · EMNLP 2024 |
Natural language and speech › Information extraction and text analysis
morphological analysis |
0.6 | 2 | 2019 | Learning Morphosyntactic Analyzers from the Bible via Iterative Annotation Projection across 26 Languages · ACL (1) 2019 Leveraging Inflection Tables for Stemming and Lemmatization · ACL (1) 2016 |
Natural language and speech › Language models and text generation › evaluation of language models › multilingual evaluation
cross-lingual evaluation |
0.4 | 1 | 2020 | Cross-Linguistic Syntactic Evaluation of Word Prediction Models · ACL 2020 |
Natural language and speech › Language models and text generation › large language model evaluation
syntactic evaluation |
0.4 | 1 | 2020 | Cross-Linguistic Syntactic Evaluation of Word Prediction Models · ACL 2020 |
Natural language and speech › Machine translation
annotation projection |
0.4 | 1 | 2019 | Learning Morphosyntactic Analyzers from the Bible via Iterative Annotation Projection across 26 Languages · ACL (1) 2019 |
Natural language and speech › Information extraction and text analysis › morphological analysis
lemmatization |
0.4 | 1 | 2019 | Learning Morphosyntactic Analyzers from the Bible via Iterative Annotation Projection across 26 Languages · ACL (1) 2019 |
Natural language and speech › Information extraction and text analysis › morphological analysis
morphosyntactic analysis |
0.4 | 1 | 2019 | Learning Morphosyntactic Analyzers from the Bible via Iterative Annotation Projection across 26 Languages · ACL (1) 2019 |
Methods — techniques the papers use, named apart from their topics
large language model · 0.8dictionary resources · 0.8challenge set evaluation · 0.4reranking · 0.4ensembling · 0.4annotation projection · 0.4unsupervised learning · 0.2discriminative string transduction · 0.2
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2024 | Multiple Sources are Better Than One: Incorporating External Knowledge in Low-Resource GlossingabstractIn this paper, we address the data scarcity problem in automatic data-driven glossing for low-resource languages by coordinating multiple sources of linguistic expertise.We enhance models by incorporating both tokenlevel and sentence-level translations, utilizing the extensive linguistic capabilities of modern LLMs, and incorporating available dictionary resources.Our enhancements lead to an average absolute improvement of 5%-points in word-level accuracy over the previous state of the art on a typologically diverse dataset spanning six low-resource languages.The improvements are particularly noticeable for the lowest-resourced language Gitksan, where we achieve a 10%-point improvement.Furthermore, in a simulated ultra-low resource setting for the same six languages, training on fewer than 100 glossed sentences, we establish an average 10%-point improvement in word-level accuracy over the previous state-of-the-art system. Changbing Yang, Garrett Nicolai, Miikka Silfverberg |
EMNLP | 2 |
| 2022 | Penalizing Divergence: Multi-Parallel Translation for Low-Resource Languages of North AmericaabstractThis paper explores a special case in multilingual machine translation: so called multi-parallel translation, where the target data for all language pairs are identical. While multi-parallelism offers benefits which are not available in a standard translation setting, translation models can easily overfit when training data are limited. We introduce a regularizer, the divergence penalty, which penalizes the translation model when it represents source sentences with identical target translations in divergent ways. Experiments on very low-resourced Indigenous North American languages show that an initially deficient multilingual translator can improve by 4.9 BLEU through mBART pre-training, and 5.5 BLEU points with the strategic addition of monolingual data, and that a divergence penalty leads to further increases of 0.4 BLEU. Further experiments on Germanic languages demonstrate a improvement of 0.5 BLEU when applying the divergence penalty. An investigation of the neural encoder representations learned by our translation models shows that the divergence penalty encourages models to learn a unified neural interlingua. Garrett Nicolai, Changbing Yang, Miikka Silfverberg |
COLING | 1 |
| 2022 | UniMorph 4.0: Universal MorphologyabstractThe Universal Morphology (UniMorph) project is a collaborative effort providing broad-coverage instantiated normalized morphological inflection tables for hundreds of diverse world languages. The project comprises two major thrusts: a language-independent feature schema for rich morphological annotation, and a type-level resource of annotated data in diverse languages realizing that schema. This paper presents the expansions and improvements on several fronts that were made in the last couple of years (since McCarthy et al. (2020)). Collaborative efforts by numerous linguists have added 66 new languages, including 24 endangered languages. We have implemented several improvements to the extraction pipeline to tackle some issues, e.g., missing gender and macrons information. We have amended the schema to use a hierarchical structure that is needed for morphological phenomena like multiple-argument agreement and case stacking, while adding some missing morphological features to make the schema more inclusive. In light of the last UniMorph release, we also augmented the database with morpheme segmentation for 16 languages. Lastly, this new release makes a push towards inclusion of derivational morphology in UniMorph by enriching the data and annotation schema with instances representing derivational processes from MorphyNet. Khuyagbaatar Batsuren, Omer Goldman, Salam Khalifa, Nizar Habash, Witold Kieras, Gábor Bella, Brian Leonard, Garrett Nicolai, Kyle Gorman, Yustinus Ghanggo Ate, Maria Ryskina, Sabrina J. Mielke, Elena Budianskaya, Charbel El-Khaissi, Tiago Pimentel, Michael Gasser, William Lane 0002, Mohit Raj, Matt Coler, Jaime Rafael Montoya Samame, Delio Siticonatzi Camaiteri, Esaú Zumaeta Rojas, Didier López Francis, Arturo Oncevay, Juan López Bautista, Gema Celeste Silva Villegas, Lucas Torroba Hennigen, Adam Ek, David Guriel, Peter Dirix, Jean-Philippe Bernardy, Andrey Scherbakov, Aziyana Bayyr-ool, Antonios Anastasopoulos, Roberto Zariquiey, Karina Sheifer, Sofya Ganieva, Hilaria Cruz, Ritván Karahóga, Stella Markantonatou, George Pavlidis, Matvey Plugaryov, Elena Klyachko, Ali Salehi, Candy Angulo, Jatayu Baxi, Andrew Krizhanovsky, Natalia Krizhanovskaya, Elizabeth Salesky, Clara Vania, Sardana Ivanova, Jennifer C. White, Rowan Hall Maudslay, Josef Valvoda, Ran Zmigrod, Paula Czarnowska, Irene Nikkarinen, Aelita Salchak, Brijesh Bhatt, Christopher Straughn, Zoey Liu, Jonathan Washington, Yuval Pinter, Duygu Ataman, Marcin Wolinski, Totok Suhardijanto, Anna Yablonskaya, Niklas Stoehr, Hossep Dolatian, Zahroh Nuriah, Shyam Ratan, Francis M. Tyers, Edoardo Maria Ponti, Grant Aiton, Aryaman Arora, Richard J. Hatcher, Ritesh Kumar 0002, Jeremiah Young, Daria Rodionova, Anastasia Yemelina, Taras Andrushko, Igor Marchenko, Polina Mashkovtseva, Alexandra Serova, Emily Tucker Prud'hommeaux, Maria Nepomniashchaya, Fausto Giunchiglia, Eleanor Chodroff, Mans Hulden, Miikka Silfverberg, Arya McCarthy, David Yarowsky, Ryan Cotterell, Reut Tsarfaty, Ekaterina Vylomova |
LREC | 8 |
| 2022 | An Inflectional Database for GitksanabstractThis paper presents a new inflectional resource for Gitksan, a low-resource Indigenous language of Canada. We use Gitksan data in interlinear glossed format, stemming from language documentation efforts, to build a database of partial inflection tables. We then enrich this morphological resource by filling in blank slots in the partial inflection tables using neural transformer reinflection models. We extend the training data for our transformer reinflection models using two data augmentation techniques: data hallucination and back-translation. Experimental results demonstrate substantial improvements from data augmentation, with data hallucination delivering particularly impressive gains. We also release reinflection models for Gitksan. Bruce Harold Oliver, Clarissa Forbes, Changbing Yang, Farhan Samir, Edith Coates, Garrett Nicolai, Miikka Silfverberg |
LREC | 6 |
| 2021 | Do RNN States Encode Abstract Phonological Alternations?abstractMiikka Silfverberg, Francis Tyers, Garrett Nicolai, Mans Hulden. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021. Miikka Silfverberg, Francis M. Tyers, Garrett Nicolai, Mans Hulden |
NAACL-HLT | 3 |
| 2020 | Cross-Linguistic Syntactic Evaluation of Word Prediction ModelsabstractA range of studies have concluded that neural word prediction models can distinguish grammatical from ungrammatical sentences with high accuracy.However, these studies are based primarily on monolingual evidence from English.To investigate how these models' ability to learn syntax varies by language, we introduce CLAMS (Cross-Linguistic Assessment of Models on Syntax), a syntactic evaluation suite for monolingual and multilingual models.CLAMS includes subject-verb agreement challenge sets for English, French, German, Hebrew and Russian, generated from grammars we develop.We use CLAMS to evaluate LSTM language models as well as monolingual and multilingual BERT.Across languages, monolingual LSTMs achieved high accuracy on dependencies without attractors, and generally poor accuracy on agreement across object relative clauses.On other constructions, agreement accuracy was generally higher in languages with richer morphology.Multilingual models generally underperformed monolingual models.Multilingual BERT showed high syntactic accuracy on English, but noticeable deficiencies in other languages. Aaron Mueller, Garrett Nicolai, Panayiota Petrou-Zeniou, Natalia Talmina, Tal Linzen |
ACL | 2 |
| 2020 | Noise Isn't Always Negative: Countering Exposure Bias in Sequence-to-Sequence Inflection ModelsabstractMorphological inflection, like many sequence-to-sequence tasks, sees great performance from recurrent neural architectures when data is plentiful, but performance falls off sharply in lowerdata settings.We investigate one aspect of neural seq2seq models that we hypothesize contributes to overfitting -teacher forcing.By creating different training and test conditions, exposure bias increases the likelihood that a system too closely models its training data.Experiments show that teacher-forced models struggle to recover when they enter unknown territory.However, a simple modification to the training algorithm to more closely mimic test conditions creates models that are better able to generalize to unseen environments. Garrett Nicolai, Miikka Silfverberg |
COLING | 1 |
| 2020 | UniMorph 3.0: Universal MorphologyabstractThe Universal Morphology (UniMorph) project is a collaborative effort providing broad-coverage instantiated normalized morphological paradigms for hundreds of diverse world languages. The project comprises two major thrusts: a language-independent feature schema for rich morphological annotation and a type-level resource of annotated data in diverse languages realizing that schema. We have implemented several improvements to the extraction pipeline which creates most of our data, so that it is both more complete and more correct. We have added 66 new languages, as well as new parts of speech for 12 languages. We have also amended the schema in several ways. Finally, we present three new community tools: two to validate data for resource creators, and one to make morphological data available from the command line. UniMorph is based at the Center for Language and Speech Processing (CLSP) at Johns Hopkins University in Baltimore, Maryland. This paper details advances made to the schema, tooling, and dissemination of project resources since the UniMorph 2.0 release described at LREC 2018. Arya McCarthy, Christo Kirov, Matteo Grella, Amrit Nidhi, Patrick Xia 0002, Kyle Gorman, Ekaterina Vylomova, Sabrina J. Mielke, Garrett Nicolai, Miikka Silfverberg, Timofey Arkhangelskiy, Nataly Krizhanovsky, Andrew Krizhanovsky, Elena Klyachko, Alexey Sorokin, John Mansfield, Valts Ernstreits, Yuval Pinter, Cassandra L. Jacobs, Ryan Cotterell, Mans Hulden, David Yarowsky |
LREC | 9 |
| 2020 | The Johns Hopkins University Bible Corpus: 1600+ Tongues for Typological ExplorationabstractWe present findings from the creation of a massively parallel corpus in over 1600 languages, the Johns Hopkins University Bible Corpus (JHUBC). The corpus consists of over 4000 unique translations of the Christian Bible and counting. Our data is derived from scraping several online resources and merging them with existing corpora, combining them under a common scheme that is verse-parallel across all translations. We detail our effort to scrape, clean, align, and utilize this ripe multilingual dataset. The corpus captures the great typological variety of the world’s languages. We catalog this by showing highly similar proportions of representation of Ethnologue’s typological features in our corpus. We also give an example application: projecting pronoun features like clusivity across alignments to richly annotate languages which do not mark the distinction. Arya McCarthy, Rachel Wicks, Dylan Lewis, Aaron Mueller, Winston Wu, Oliver Adams, Garrett Nicolai, Matt Post, David Yarowsky |
LREC | 7 |
| 2020 | An Analysis of Massively Multilingual Neural Machine Translation for Low-Resource LanguagesabstractIn this work, we explore massively multilingual low-resource neural machine translation. Using translations of the Bible (which have parallel structure across languages), we train models with up to 1,107 source languages. We create various multilingual corpora, varying the number and relatedness of source languages. Using these, we investigate the best ways to use this many-way aligned resource for multilingual machine translation. Our experiments employ a grammatically and phylogenetically diverse set of source languages during testing for more representative evaluations. We find that best practices in this domain are highly language-specific: adding more languages to a training set is often better, but too many harms performance—the best number depends on the source language. Furthermore, training on related languages can improve or degrade performance, depending on the language. As there is no one-size-fits-most answer, we find that it is critical to tailor one’s approach to the source language and its typology. Aaron Mueller, Garrett Nicolai, Arya McCarthy, Dylan Lewis, Winston Wu, David Yarowsky |
LREC | 2 |
| 2020 | Fine-grained Morphosyntactic Analysis and Generation Tools for More Than One Thousand LanguagesabstractExploiting the broad translation of the Bible into the world’s languages, we train and distribute morphosyntactic tools for approximately one thousand languages, vastly outstripping previous distributions of tools devoted to the processing of inflectional morphology. Evaluation of the tools on a subset of available inflectional dictionaries demonstrates strong initial models, supplemented and improved through ensembling and dictionary-based reranking. Likewise, a novel type-to-token based evaluation metric allows us to confirm that models generalize well across rare and common forms alike Garrett Nicolai, Dylan Lewis, Arya McCarthy, Aaron Mueller, Winston Wu, David Yarowsky |
LREC | 1 |
| 2020 | Multilingual Dictionary Based Construction of Core VocabularyabstractWe propose a new functional definition and construction method for core vocabulary sets for multiple applications based on the relative coverage of a target concept in thousands of bilingual dictionaries. Our newly developed core concept vocabulary list derived from these dictionary consensus methods achieves high overlap with existing widely utilized core vocabulary lists targeted at applications such as first and second language learning or field linguistics. Our in-depth analysis illustrates multiple desirable properties of our newly proposed core vocabulary set, including their non-compositionality. We employ a cognate prediction method to recover missing coverage of this core vocabulary in massively multilingual dictionary construction, and we argue that this core vocabulary should be prioritized for elicitation when creating new dictionaries for low-resource languages for multiple downstream tasks including machine translation and language learning. Winston Wu, Garrett Nicolai, David Yarowsky |
LREC | 2 |
| 2019 | Learning Morphosyntactic Analyzers from the Bible via Iterative Annotation Projection across 26 LanguagesabstractA large percentage of computational tools are concentrated in a very small subset of the planet’s languages. Compounding the issue, many languages lack the high-quality linguistic annotation necessary for the construction of such tools with current machine learning methods. In this paper, we address both issues simultaneously: leveraging the high accuracy of English taggers and parsers, we project morphological information onto translations of the Bible in 26 varied test languages. Using an iterative discovery, constraint, and training process, we build inflectional lexica in the target languages. Through a combination of iteration, ensembling, and reranking, we see double-digit relative error reductions in lemmatization and morphological analysis over a strong initial system. Garrett Nicolai, David Yarowsky |
ACL (1) | 1 |
| 2016 | Leveraging Inflection Tables for Stemming and LemmatizationabstractWe present several methods for stemming and lemmatization based on discriminative string transduction. We exploit the paradigmatic regularity of semi-structured inflection tables to identify stems in an unsupervised manner with over 85% accuracy. Experiments on English, Dutch and German show that our stemmers substantially outperform Snowball and Morfessor, and approach the accuracy of a supervised model. Furthermore, the generated stems are more consistent than those annotated by experts. Our direct lemmatization model is more accurate than Morfette and Lemming on most datasets. Finally, we test our methods on the data from the shared task on morphological reinflection. Garrett Nicolai, Grzegorz Kondrak |
ACL (1) | 1 |
| 2015 | Inflection Generation as Discriminative String TransductionabstractWe approach the task of morphological inflection generation as discriminative string transduction.Our supervised system learns to generate word-forms from lemmas accompanied by morphological tags, and refines them by referring to the other forms within a paradigm.Results of experiments on six diverse languages with varying amounts of training data demonstrate that our approach improves the state of the art in terms of predicting inflected word-forms. Garrett Nicolai, Colin Cherry, Grzegorz Kondrak |
HLT-NAACL | 1 |
| 2015 | English orthography is not "close to optimal"abstractIn spite of the apparent irregularity of the English spelling system, Chomsky and Halle (1968) characterize it as "near optimal".We investigate this assertion using computational techniques and resources.We design an algorithm to generate word spellings that maximize both phonemic transparency and morphological consistency.Experimental results demonstrate that the constructed system is much closer to optimality than the traditional English orthography. Garrett Nicolai, Grzegorz Kondrak |
HLT-NAACL | 1 |