As the quality of contemporary speech synthesis improves, so too does the interest from language communities in developing text-to-speech (TTS) systems for a variety of real-world applications. Much of the work on TTS has focused on high-resource languages, resulting in implicitly resource-intensive paths to building such systems. The goal of this paper is to provide signposts and points of reference for future low-resource speech synthesis efforts, with insights drawn from the Speech Generation for Indigenous Language Education (SGILE) project. Funded and coordinated by the National Research Council of Canada (NRC), this multi-year, multi-partner project has the goal of producing high-quality text-to-speech systems that support the teaching of Indigenous languages in a variety of educational contexts. We provide background information and motivation for the project, as well as details about our approach and project structure, including results from a multi-day requirements-gathering session. We discuss some of our key challenges, including building models with appropriate controls for educators, improving model data efficiency, and strategies for low-resource transfer learning and evaluation. Finally, we provide a detailed survey of existing speech synthesis software and introduce EveryVoice TTS, a toolkit designed specifically for low-resource speech synthesis. • We provide background and points of reference for future low-resource TTS projects • We describe four main technical challenges for low-resource speech synthesis • We introduce the EveryVoice TTS Toolkit designed specifically for low-resource TTS • We compare the EveryVoice TTS Toolkit with six other existing toolkits

We introduce Gramble, a domain-specific programming language for linguistic parsing and generation, in the tradition of XFST, TWOLC, and Kleene. Gramble features an intuitive tabular syntax and supports live group programming, allowing community experts to participate more directly in system development without having to be programmers themselves. A cross-platform interpreter is available for Windows, MacOS, and UNIX, supports collaborative programming on the web via Google Sheets, and is released open-source under the MIT license.

Roland Kuhn, Fineen Davis, Alain Désilets, Eric Joanis, Anna Kazantseva, Rebecca Knowles, Patrick Littell, Delaney Lothian, Aidan Pine, Caroline Running Wolf, Eddie Santos, Darlene Stewart, Gilles Boulianne, Vishwa Gupta, Brian Maracle Owennatékha, Akwiratékha’ Martin, Christopher Cox, Marie-Odile Junker, Olivia Sammons, Delasie Torkornoo, Nathan Thanyehténhas Brinklow, Sara Child, Benoît Farley, David Huggins-Daines, Daisy Rosenblum, Heather Souter. Proceedings of the 28th International Conference on Computational Linguistics. 2020.

The Inuktitut language, a member of the Inuit-Yupik-Unangan language family, is spoken across Arctic Canada and noted for its morphological complexity. It is an official language of two territories, Nunavut and the Northwest Territories, and has recognition in additional regions. This paper describes a newly released sentence-aligned Inuktitut–English corpus based on the proceedings of the Legislative Assembly of Nunavut, covering sessions from April 1999 to June 2017. With approximately 1.3 million aligned sentence pairs, this is, to our knowledge, the largest parallel corpus of a polysynthetic language or an Indigenous language of the Americas released to date. The paper describes the alignment methodology used, the evaluation of the alignments, and preliminary experiments on statistical and neural machine translation (SMT and NMT) between Inuktitut and English, in both directions.

In this article, we discuss which text, speech, and image technologies have been developed, and would be feasible to develop, for the approximately 60 Indigenous languages spoken in Canada. In particular, we concentrate on technologies that may be feasible to develop for most or all of these languages, not just those that may be feasible for the few most-resourced of these. We assess past achievements and consider future horizons for Indigenous language transliteration, text prediction, spell-checking, approximate search, machine translation, speech recognition, speaker diarization, speech synthesis, optical character recognition, and computer-aided language learning.

Today's consumer is faced with the task of navigating through a bewildering array of entertainment choices in the home. The traditional interfaces for accessing entertainment content (e.g., the handheld remote TV control) were not designed to cope with such a complex navigation task. Furthermore, such traditional interfaces present choices in a way that is quite different from the way in which the user thinks about the choices. We describe a prototype system for interactive TV control that illustrates a new paradigm for human-centered access to entertainment. The system is aided by knowledge about the consumer's preferences and past behavior, and by its access to information available on the Internet. We show how, by putting together a number of enabling technologies, a natural dialogue between the system and the consumer is achieved. The dialogue is mixed-mode; the system supports spoken input and output, and graphical and text output. Test users greatly preferred this mixed interface to a speech-only one, mainly because it leaves them in no doubt about the current state of the system. We believe the success of the approach adopted here shows how accessibility requirements may be met for many different types of consumer devices.

Multimedia content cannot be retrieved effectively unless metadata describing it is generated. However, metadata generation tends to be time-consuming and expensive, since it typically involves human beings going through the content and manually tagging it. The paper shows how automatic speech recognition (ASR) technology can be used to carry out metadata generation with significantly less expenditure of human effort. The paper describes two different approaches: voice tagging, whereby human beings tag the data but this process is speeded up by applying ASR to the tagging process; audio indexing, whereby much of the tagging process is automated by applying ASR to the content itself.

Biometrics is an emerging topic in the field of signal processing. While technologies (e.g. audio, video) for biometrics have mostly been studied separately, ultimately, biometric technologies could find their strongest role as interwined and complementary pieces of a multi-modal authentication system. In this paper, a short overview of voice, fingerprint, and face authentication algorithms is provided.

The "eigenvoice" technique achieves rapid speaker adaptation by employing prior knowledge of speaker space obtained from reference speakers to place strong constraints on the initial model for each new speaker. It has previously been shown to yield very fast adaptation for a large-vocabulary system. In this paper, we describe a new way of applying the eigenvoice technique to context-dependent acoustic modeling, called the "eigencentroid plus delta trees" (EDT) model. Here, the context-dependent model is defined so that it consists of a speaker-dependent component with a small number of parameters linked to a speaker-independent component with far more parameters. The eigenvoice technique can then be applied to the speaker-dependent component alone to attain very fast adaptation of the entire context-dependent model (e.g., 10% relative reduction in error rate after 3 sentences). EDT requires only a small number of parameters to represent speaker space and works even if only a small amount of data is available per reference speaker.

Gaussian Mixture Models (GMMs) have been successfully applied to the tasks of speaker ID and verification when a large amount of enrolment data is available to characterize client speakers ([1],[10],[11]). However, there are many applications where it is unreasonable to expect clients to spend this much time training the system. Thus, we have been exploring the performance of various methods when only a sparse amount of enrolment data is available. Under such conditions, the performance of GMMs deteriorates drastically. A possible solution is the “eigenvoice” approach, in which client and test speaker models are confined to a low-dimensional linear subspace obtained previously from a different set of training data. One advantage of the approach is that it does away with the need for impostor models for speaker verification. After giving a detailed description of the eigenvoice approach, the paper compares the performance of conventional GMMs on speaker ID and verification with that of GMMs obtained by means of the eigenvoice approach. Experimental results are presented to show that conventional GMMs perform better if there are abundant enrolment data, while eigenvoice GMMs perform better if enrolment data are sparse. The paper also gives experimental results for the case where the eigenspace is trained on one database (TIMIT), but client enrolment and testing involve another (YOHO). For this case, we show that performance improves if an environment adaptation technique is applied to the eigenspace. Finally, we discuss priorities for future work. training data client data train eigenspace estimate clients’ coordinates SPEAKER ID/VERIFICATION 1. use distance in eigenspace-> ‘‘eigendistance decoding’’ OR test data 2. use P(test data|client GMMs)-> ‘‘eigenGMM decoding’’ optional MLES re-estimation optional MLLR adaptation

This paper describes a new model-based speaker adaptation algorithm called the eigenvoice approach. The approach constrains the adapted model to be a linear combination of a small number of basis vectors obtained offline from a set of reference speakers, and thus greatly reduces the number of free parameters to be estimated from adaptation data. These "eigenvoice" basis vectors are orthogonal to each other and guaranteed to represent the most important components of variation between the reference speakers. Experimental results for a small-vocabulary task (letter recognition) given in the paper show that the approach yields major improvements in performance for tiny amounts of adaptation data. For instance, we obtained 16% relative improvement in error rate with one letter of supervised adaptation data, and 26% relative improvement with four letters of supervised adaptation data. After a comparison of the eigenvoice approach with other speaker adaptation algorithms, the paper concludes with a discussion of future work.

Previously, we presented a radically new class of fast adaptation techniques for speech recognition, based on prior knowledge of speaker variation. To obtain this prior knowledge, one applies a dimensionality reduction technique to T vectors of dimension D derived from T speaker-dependent (SD) models. This offline step yields T basis vectors, the eigenvoices. We constrain the model for new speaker S to be located in the space spanned by the first K eigenvoices. Speaker adaptation involves estimating K eigenvoice coefficients for the new speaker; typically, K is very small compared to original dimension D. Here, we review how to find the eigenvoices, give a maximum-likelihood estimator for the new speaker's eigenvoice coefficients, and summarize mean adaptation experiments carried out on the Isolet database. We present new results which assess the impact on performance of changes in training of the SD models. Finally, we interpret the first few eigenvoices obtained.

Building on earlier work [2], we show how a set of binary decision trees grown by means of the Gelfand-Ravishankar-Delp algorithm [8] can be trained to generate an ordered list of possible pronunciations from a spelled word. Training is carried out on a database consisting of spelled words paired with their pronunciations (in a particular language). We show how phonotactic information can be learned by a second set of decision trees, which reorder the multiple pronunciations generated by the first set. The paper defines the “inclusion” metric for scoring phoneticizers that generate multiple pronunciations. Experimental results employing this metric indicate that phonotactic reordering yields a slight improvement when only the top pronunciation is retained, and a large improvement when more than one hypothesis is retained. Isolated-word recognition results which show good performance for automatically-generated pronunciations are given.

There are hidden analogies between two dissimilar research areas: face recognition and speech recognition. The standard representations for faces and voices misleadingly suggest that they have a high number of degrees of freedom. However, human faces have two eyes, a nose, and a mouth in predictable locations; such constraints ensure that possible images of faces occupy a tiny portion of the space of possible 2D images. Similarly, physical and cultural constraints on acoustic realizations of words uttered by a particular speaker imply that the true number of degrees of freedom for speaker-dependent hidden Markov models (HMMs) is quite small. Face recognition researchers have adopted representations that make explicit the underlying low dimensionality of the task, greatly improving the performance of their systems while reducing computational costs. We argue that speech researchers should use similar techniques to represent variation between speakers, and discuss applications to speaker adaptation, speaker identification and speaker verification.

Topic spotting is often performed on the output of a large vocabulary recognizer or a keyword spotter. However, this requires detailed knowledge about the vocabulary, and transcribed training data. If portability to new topics and languages is important, then a topic spotter based on phoneme recognition is preferable. A phoneme recognizer is run on training data consisting of audio files labeled by topic alone-no word transcripts are required. Phoneme sub-sequences which help to predict the topic are then extracted automatically. The work described was carried out by two teams exploring three very different approaches to phoneme-based topic spotting: the "DP-ngram", the "decision tree", and the "Euclidean" approach. Results obtained by each team on the ARM (Airborne Reconnaissance Mission) and Switchboard data sets were compared by means of receiver operating characteristic (ROC) curves. The best performance for each team was obtained via a similar type of discriminative training.

We propose multilevel semantic classification trees to combine different information sources for predicting speech events (e.g. word chains, phrases, etc.). Traditionally in speech recognition systems these information sources (acoustic evidence, language model) are calculated independently and combined via Bayes rule. The proposed approach allows one to combine sources of different types it is no longer necessary for each source to yield a probability. Moreover the tree can look at several information sources simultaneously. The approach is demonstrated for the prediction of prosodically marked phrase boundaries, combining information about the spoken word chain, word category information, prosodic parameters, and the result of a neural network predicting the boundary on the basis of acoustic-prosodic features. The recognition rates of up to 90% for the two class problem boundary vs. no boundary are already comparable to results achieved with the above mentioned Bayes rule approach that combines the acoustic classifier with a 5-gram categorical language model. This is remarkable, since so far only a small set of questions combining information from different sources have been implemented.

Recently, we described a two-step self-learning approach for grapheme-to-phoneme (G2P) conversion [1].In the first step, grapheme and phoneme strings in the training data are aligned via an iterative Viterbi procedure that may insert graphemic and phonemic nulls where required.In the second step, a Trie structure encoding pronunciation rules is generated.In this paper we describe the alignment module, and give alignment accuracies on the NETtalk database.We also compare transcription accuracies for two approaches to the second step on three databases: the NETtalk database, the CMU dictionary and the French part of the ONOMASTICA lexicon.The two transcription approaches applied in this research are a Trie approach [1] and an approach based on binary decision trees grown by means of the Gelfand-Ravishankar-Delp algorithm [2,3,4].We discuss the choice of questions for these decision trees -it may be possible to formulate questions about groups of characters (e.g., "is the next letter a vowel?") that yield better trees than those that only use questions about individual characters (e.g., "is the next letter an 'A' ?").Finally, we discuss the implications of our work for G2P conversion.

Bahl et al. (see ICASSP-91, vol.1, p.185-188, 1991) employed decision trees to specify the acoustic realization of a phone as a function of its context. By using a computationally cheap Poisson-based evaluation function, they were able to account for a much wider context than previous researchers (five preceding and five following phones). We extend this work in four ways. (1) We employ the Poisson criterion to find quickly the M best questions at a node during tree expansion, then use an HMM-based MLE criterion to make the final choice from these (and for pruning trees). (2) Bahl et al. use stopping criteria to halt the growth of a tree, which is then used for speech recognition. It is preferable to grow an over-large tree and then prune it; we apply the efficient GRD expansion-pruning algorithm of Gelfand et al. (see IEEE Trans. PAMI, vol.13, no.2, p.163-174, 1991) to the phonetic modeling problem. (3) Like Bahl et al., we allow questions about a large number of preceding and following phones. However, a given search algorithm may make some of these questions difficult to answer. In addition to the "yes" and "no" children of each question, we grow a "don't know" subtree to be used if a question is unanswerable at present. (4) We have experimented with questions based on phonetic features, as well as questions that ask about the presence of specific phones. Our approach permits an arbitrary feature schema to be read in and used in question generation.

This paper describes a system designed for use by professional translators that enables them to dictate their translation. Because the speech recognizer has access to the source text as well as the spoken translation, a statistical translation model can guide recognition. This can be done in many different ways---which is best? We discuss the experiments that led to integration of the translation model in a way that improves both speed and performance. 1 Introduction The TransTalk project attempts to integrate speech recognition and machine translation in a way that makes maximal use of their complementary strengths. Professional translators often dictate their translations first and have them typed afterwards. If they dictate to a speech recognition system instead, and if that system has access to the source language text, it can use probabilistic translation models to aid recognition. For instance, if the speech recognition system is deciding between the acoustically similar French ...

This article describes a new method for building a natural language understanding (NLU) system, in which the system's rules are learnt automatically from training data. The method has been applied to design of a speech understanding (SU) system. Designers of such systems rely increasingly on robust matchers to perform the task of extracting meaning from one or several word sequence hypotheses generated by a speech recognizer. We describe a new data structure, the semantic classification tree (SCT), that learns semantic rules from training data and can be a building block for robust matchers for NLU tasks. By reducing the need for handcoding and debugging a large number of rules, this approach facilitates rapid construction of an NLU system. In the case of an SU system, the rules learned by an SCT are highly resistant to errors by the speaker or by the speech recognizer because they depend on a small number of words in each utterance. Our work shows that semantic rules can be learned automatically from training data, yielding successful NLU for a realistic application.>

Previously (see ICASSP-93, vol.2, p.55, 1993 and Eurospeech 93, vol.2, p.1331, 1993) we described a speech understanding system called "CHANEL" with two components: a chart-based parser that analyzes semantically important word islands within an utterance; and a component based on "semantic classification trees" (SCTs) that builds the representation for the complete utterance. The construction of a natural-language understanding (NLU) system is a task that has traditionally required lavish expenditure of programmer-hours. By dividing the task in this way, we enabled many of the system's rules (those contained in the SCT component) to be learned automatically from training data, freeing human expertise to be applied where it is most effective. This paper describes recent improvements to both components of CHANEL, along with a new module that handles context-dependent utterances. The new version of CHANEL has a new use for SCTs: a special SCT decides whether a sentence is context-dependent or not.>

An approach to designing dialogue systems, as applied to a robot with speech recognition capabilities, is described. One of the goals of this research is to develop probabilistic methods for dialogue systems analogous to those used in language modeling for dictation systems. These methods will use knowledge about dialogue structure, performance, syntax, and semantics to constrain the speech recognition task.>

A model for spoken dialogue between an operator and a mobile robot in the procedural network formalism is described. The model is used to improve speech recognition by the robot and aid in the design of the robot's language generation component. Some probabilistic language models for understanding dialogue, which may be generally applicable to a variety of systems, are described.>

Speech-recognition systems must often decide between competing ways of breaking up the acoustic input into strings of words. Since the possible strings may be acoustically similar, a language model is required; given a word string, the model returns its linguistic probability. Several Markov language models are discussed. A novel kind of language model which reflects short-term patterns of word use by means of a cache component (analogous to cache memory in hardware terminology) is presented. The model also contains a 3g-gram component of the traditional type. The combined model and a pure 3g-gram model were tested on samples drawn from the Lancaster-Oslo/Bergen (LOB) corpus of English text. The relative performance of the two models is examined, and suggestions for the future improvements are made.>