How do we roll out language technologies across a world with 7,000 languages?In one story, we scale the successes of NLP further into 'low-resource' languages, doing ever more with less.However, this approach does not recognise the fact that -beyond the 500 institutional languages -the remaining languages are oral vernaculars.These speech communities interact with the outside world using a 'contact language'.I argue that contact languages are the appropriate target for technologies like speech recognition and machine translation, and that the 6,500 oral vernaculars should be approached differently.I share stories from an Indigenous community where local people reshaped an extractive agenda to align with their relational agenda.I describe the emerging paradigm of Relational NLP and explain how it opens the way to non-extractive methods and to solutions that enhance human agency.

How can NLP/AI practitioners engage with oral societies and develop locally appropriate language technologies?We report on our experience of working together over five years in a remote community in the far north of Australia, and how we prototyped simple language technologies to support our collaboration.We navigated different understandings of language, the functional differentiation of institutional vs oral languages, and the distinct technology opportunities for each.Our collaboration unsettled the first author's western framing of language as data for exploitation by machines, and we devised a design pattern that seems better aligned with local interests and aspirations.We call for new collaborations on the design of appropriate technologies for oral languages.

How can language technology address the diverse situations of the world's languages?In one view, languages exist on a resource continuum and the challenge is to scale existing solutions, bringing under-resourced languages into the high-resource world.In another view, presented here, the world's language ecology includes standardised languages, local languages, and contact languages.These are often subsumed under the label of 'under-resourced languages' even though they have distinct functions and prospects.I explore this position and propose some ecologically-aware language technology agendas.

Most low resource language technology development is premised on the need to collect data for training statistical models.When we follow the typical process of recording and transcribing text for small Indigenous languages, we hit up against the so-called "transcription bottleneck."Therefore it is worth exploring new ways of engaging with speakers which generate data while avoiding the transcription bottleneck.We have deployed a prototype app for speakers to use for confirming system guesses in an approach to transcription based on word spotting.However, in the process of testing the app we encountered many new problems for engagement with speakers.This paper presents a close-up study of the process of deploying data capture technology on the ground in an Australian Aboriginal community.We reflect on our interactions with participants and draw lessons that apply to anyone seeking to develop methods for language data collection in an Indigenous community.

An increasing number of papers have been addressing issues related to low-resource languages and the transcription bottleneck paradigm. After several years spent in Northern Australia, where some of the strongest Aboriginal languages are spoken, we could observe a gap between the motivations depicted in research contributions in this space and the Northern Australian context. In this paper, we address this gap in research by exploring the potential of speech recognition in an Aboriginal community. We describe our work from training a spoken term detection system to its implementation in an activity with Aboriginal participants. We report here on one side how speech recognition technologies can find their place in an Aboriginal context and, on the other, methodological paths that allowed us to reach better comprehension and engagement from Aboriginal participants.

Human expertise and the participation of speech communities are essential factors in the success of technologies for low-resource languages.Accordingly, we propose a new computational task which is tuned to the available knowledge and interests in an Indigenous community, and which supports the construction of high quality texts and lexicons.The task is illustrated for Kunwinjku, a morphologicallycomplex Australian language.We combine a finite state implementation of a published grammar with a partial lexicon, and apply this to a noisy phone representation of the signal.We locate known lexemes in the signal and use the morphological transducer to build these out into hypothetical, morphologicallycomplex words for human validation.We show that applying a single iteration of this method results in a relative transcription density gain of 17%.Further, we find that 75% of breath groups in the test set receive at least one correct partial or full-word suggestion.

Polysynthetic languages have exceptionally large and sparse vocabularies, thanks to the number of morpheme slots and combinations in a word.This complexity, together with a general scarcity of written data, poses a challenge to the development of natural language technologies.To address this challenge, we offer linguistically-informed approaches for bootstrapping a neural morphological analyzer, and demonstrate its application to Kunwinjku, a polysynthetic Australian language.We generate data from a finite state transducer to train an encoderdecoder model.We improve the model by "hallucinating" missing linguistic structure into the training data, and by resampling from a Zipf distribution to simulate a more natural distribution of morphemes.The best model accounts for all instances of reduplication in the test set and achieves an accuracy of 94.7% overall, a 10 percentage point improvement over the FST baseline.This process demonstrates the feasibility of bootstrapping a neural morph analyzer from minimal resources.

After generations of exploitation, Indigenous people often respond negatively to the idea that their languages are data ready for the taking.By treating Indigenous knowledge as a commodity, speech and language technologists risk disenfranchising local knowledge authorities, reenacting the causes of language endangerment.Scholars in related fields have responded to calls for decolonisation, and we in the speech and language technology community need to follow suit, and explore what this means for our practices that involve Indigenous languages and the communities who own them.This paper reviews colonising discourses in speech and language technology, and suggests new ways of working with Indigenous communities, and seeks to open a discussion of a postcolonial approach to computational methods for supporting language vitality.

We propose a novel transcription workflow which combines spoken term detection and humanin-the-loop, together with a pilot experiment.This work is grounded in an almost zero-resource scenario where only a few terms have so far been identified, involving two endangered languages.We show that in the early stages of transcription, when the available data is insufficient to train a robust ASR system, it is possible to take advantage of the transcription of a small number of isolated words in order to bootstrap the transcription of a speech collection.

Text input technologies for low-resource languages support literacy, content authoring, and language learning.However, tasks such as word completion pose a challenge for morphologically complex languages thanks to the combinatorial explosion of possible words.We have developed a method for morphologically-aware text input in Kunwinjku, a polysynthetic language of northern Australia.We modify an existing finite state recognizer to map input morph prefixes to morph completions, respecting the morphosyntax and morphophonology of the language.We demonstrate the portability of the method by applying it to Turkish.We show that the space of proximal morph completions is many orders of magnitude smaller than the space of full word completions for Kunwinjku.We provide a visualization of the morph completion space to enable the text completion parameters to be fine-tuned.Finally, we report on a web services deployment, along with a web interface which helps users enter morphologically complex words and which retrieves corresponding entries from the lexicon.

Abstract The transcription bottleneck is often cited as a major obstacle for efforts to document the world’s endangered languages and supply them with language technologies. One solution is to extend methods from automatic speech recognition and machine translation, and recruit linguists to provide narrow phonetic transcriptions and sentence-aligned translations. However, I believe that these approaches are not a good fit with the available data and skills, or with long-established practices that are essentially word-based. In seeking a more effective approach, I consider a century of transcription practice and a wide range of computational approaches, before proposing a computational model based on spoken term detection that I call “sparse transcription.” This represents a shift away from current assumptions that we transcribe phones, transcribe fully, and transcribe first. Instead, sparse transcription combines the older practice of word-level transcription with interpretive, iterative, and interactive processes that are amenable to wider participation and that open the way to new methods for processing oral languages.

Oliver Adams, Adam Makarucha, Graham Neubig, Steven Bird, Trevor Cohn. Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers. 2017.

Long Duong, Hiroshi Kanayama, Tengfei Ma, Steven Bird, Trevor Cohn. Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers. 2017.

A new computer science curriculum has been developed for the Victorian Certificate of Education. It gives students direct entry into second year University computer science. The curriculum focuses on data structures and algorithms, with an emphasis on the graph abstract data type and graph algorithms. We taught a pilot course during 2014 involving students from seven schools, and administered an algorithmic thinking quiz on entry and exit, and also tested a first year university reference group. In this paper we present the curriculum and report on the evaluation. We discuss the effectiveness of our approach and make recommendations for improving future versions of the course. Pedagogical issues are discussed in relation to the cognitive education literature on the teaching of algorithmic thinking.

Language documentation begins by gathering speech.Manual or automatic transcription at the word level is typically not possible because of the absence of an orthography or prior lexicon, and though manual phonemic transcription is possible, it is prohibitively slow.On the other hand, translations of the minority language into a major language are more easily acquired.We propose a method to harness such translations to improve automatic phoneme recognition.The method assumes no prior lexicon or translation model, instead learning them from phoneme lattices and translations of the speech being transcribed.Experiments demonstrate phoneme error rate improvements against two baselines and the model's ability to learn useful bilingual lexical entries.

Crosslingual word embeddings represent lexical items from different languages in the same vector space, enabling transfer of NLP tools. However, previous attempts had expensive resource requirements, difficulty incorporating monolingual data or were unable to handle polysemy. We address these drawbacks in our method which takes advantage of a high coverage dictionary in an EM style training algorithm over monolingual corpora in two languages. Our model achieves state-of-the-art performance on bilingual lexicon induction task exceeding models using large bilingual corpora, and competitive results on the monolingual word similarity and cross-lingual document classification task.

Translation models have been used to improve automatic speech recognition when speech input is paired with a written translation, primarily for the task of computer-aided translation. Existing approaches require large amounts of parallel text for training the translation models, but for many language pairs this data is not available. We propose a model for learning lexical translation parameters directly from the word lattices for which a transcription is sought. The model is expressed through composition of each lattice with a weighted finite-state transducer representing the translation model, where inference is performed by sampling paths through the composed finitestate transducer. We show consistent word error rate reductions in two datasets, using between just 20 minutes and 4 hours of speech input, additionally outperforming a translation model trained on the 1-best path.

Long Duong, Antonios Anastasopoulos, David Chiang, Steven Bird, Trevor Cohn. Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2016.

Cross-lingual transfer has been shown to produce good results for dependency parsing of resource-poor languages.Although this avoids the need for a target language treebank, most approaches have still used large parallel corpora.However, parallel data is scarce for low-resource languages, and we report a new method that does not need parallel data.Our method learns syntactic word embeddings that generalise over the syntactic contexts of a bilingual vocabulary, and incorporates these into a neural network parser.We show empirical improvements over a baseline delexicalised parser on both the CoNLL and Universal Dependency Treebank datasets.We analyse the importance of the source languages, and show that combining multiple source-languages leads to a substantial improvement.

Accurate dependency parsing requires large treebanks, which are only available for a few languages. We propose a method that takes advantage of shared structure across languages to build a mature parser using less training data. We propose a model for learning a shared “univer-sal ” parser that operates over an inter-lingual continuous representation of lan-guage, along with language-specific map-ping components. Compared with super-vised learning, our methods give a con-sistent 8-10 % improvement across several treebanks in low-resource simulations. 1

by stating that they required an external tag dictionary.We have corrected these inaccuracies to reflect their modest data requirements.

469

The Natural Language Toolkit is a suite of program modules, data sets and tutorials supporting research and teaching in computational linguistics and natural language processing. NLTK is written in Python and distributed under the GPL open source license. Over the past year the toolkit has been rewritten, simplifying many linguistic data structures and taking advantage of recent enhancements in the Python language. This paper reports on the simplified toolkit and explains how it is used in teaching NLP.

Linguistic research and natural language processing employ large repositories of ordered trees. XML, a standard ordered tree model, and XPath, its associated language, are natural choices for linguistic data and queries. However, several important expressive features required for linguistic queries are missing or hard to express in XPath. In this paper, we motivate and illustrate these features with a variety of linguistic queries. Then we propose extensions to XPath to support linguistic queries, and design an efficient query engine based on a novel labeling scheme. Experiments demonstrate that our language is not only sufficiently expressive for linguistic trees but also efficient for practical usage.

Search engines pervade the digital world, mediating most access to information instantaneously. We have found that students can build search engine components, and even entire search engines, in the context of problem-based learning in introductory and intermediate computer science courses. The courses cover a broad range of topics in algorithms, data structures, and web design, with a heavy emphasis on programming. Additionally, the internet is coupled with the syllabus at many places, from web design and HTML to graph algorithms and pattern matching. This connection enlivens the discussion of otherwise dry topics like searching, sorting, indexing and hashing. Moreover, the challenge of web-scale computing motivates the continuing students in their later study of formal topics like algorithmic complexity, while non-continuing students acquire transferable analytical skills. We report on the experience in search engine projects for driving problem-based learning in computer science courses, for both high school and university students. Our experience shows that such projects are effective in both introductory and intermediate courses, and readily encompass student groups with diverse programming abilities.