Understanding generalization in deep neural networks is an active area of research. A promising avenue of exploration has been that of margin measurements: the shortest distance to the decision boundary for a given sample or its representation internal to the network. While margins have been shown to be correlated with the generalization ability of a model when measured at its hidden representations (hidden margins), no such link between large margins and generalization has been established for input margins. We show that while input margins are not generally predictive of generalization, they can be if the search space is appropriately constrained. We develop such a measure based on input margins, which we refer to as 'constrained margins'. The predictive power of this new measure is demonstrated on the 'Predicting Generalization in Deep Learning' (PGDL) dataset and contrasted with hidden representation margins. We find that constrained margins achieve highly competitive scores and outperform other margin measurements in general. This provides a novel insight on the relationship between generalization and classification margins, and highlights the importance of considering the data manifold for investigations of generalization in DNNs.

The choice of airfoil parameterisation method significantly influences the overall wing optimisation performance by affecting the flexibility and computational efficiency of the process. Ideally, one should be able to intuitively constrain airfoil shape and structural characteristics as input to the optimisation process. Current parameterisation techniques lack the flexibility to generate airfoils efficiently by specifying parsimonious shape and structural features. To address this limitation, a deep learning framework is proposed, enabling conditional airfoil generation from an airfoil’s shape and structural feature definition. Specifically, we demonstrate the application of Bidirectional Long Short Term Memory models and Bayesian Gaussian Mixture models to derive airfoil coordinates from a compact set of shape and structural characteristics that we define. The proposed framework is shown to achieve favorable airfoil performance optimisation due to improved exploration and exploitation of the design space, compared to traditional approaches. Overall, the proposed optimisation framework is able to realise a 9.04% performance improvement over an airfoil design optimised with traditional parameterisation techniques.

Traffic speed prediction using deep learning has been the topic of many studies. In this paper, we analyse \nthe performance of Graph Neural Network-based techniques during periods of traffic congestion. We first \ncompare a selection of recently proposed techniques that claim to achieve good results using the METR-LA \nand PeMS-BAY data sets. We then investigate the performance of three of these approaches – GraphWaveNet, \nSpacetime Neural Network (STNN) and Spatio-Temporal Attention Wavenet (STAWnet) – during congested \nperiods, using recurrent congestion patterns to set a threshold for general congestion through the entire traffic \nnetwork. Our results show that performance deteriorates significantly during congested time periods, which \nis concerning, as traffic speed prediction is usually of most value during times of congestion. We also found \nthat, while the above approaches perform almost equally in the absence of congestion, there are much bigger \ndifferences in performance during periods of congestion.

Spoken term detection, especially of out-of-vocabulary (OOV) keywords, benefits from the use of sub-word systems. We experiment with different language-independent approaches to sub-word unit generation, generating both syllable-like and morpheme-like units, and demonstrate how the performance of syllable-like units can be improved by artificially increasing the number of unique units. The effect of unit choice is empirically evaluated using the eight languages from the 2016 IARPA BABEL evaluation.

This paper describes the development of text-to-speech corpora \nfor four South African languages. The approach followed investigated \nthe possibility of using low-cost methods including informal \nrecording environments and untrained volunteer speakers. \nThis objective and the additional future goal of expanding \nthe corpus to increase coverage of South Africa’s 11 official \nlanguages necessitated experimenting with multi-speaker and \ncode-switched data. The process and relevant observations are \ndetailed throughout. The latest version of the corpora are available \nfor download under an open-source license and will likely \nsee further development and refinement in future. \nIndex Terms: text-to-speech corpus, under-resourced languages

We investigate the impact of recent advances in speech recognition \ntechniques for under-resourced languages. Specifically, \nwe review earlier results published on the Lwazi ASR corpus \nof South African languages, and experiment with additional \nacoustic modeling approaches. We demonstrate large gains by \napplying current state-of-the-art techniques, even if the data itself \nis neither extended nor improved. We analyze the various \nperformance improvements observed, report on comparative \nperformance per technique – across all eleven languages \nin the corpus – and discuss the implications of our findings for \nunder-resourced languages in general.

Pronunciation lexicons can range from fully graphemic (modeling \neach word using the orthography directly) to fully phonemic \n(first mapping each word to a phoneme string). Between these \ntwo options lies a continuum of modeling options. We analyze \ntechniques that can improve the accuracy of a graphemic system \nwithout requiring significant effort to design or implement. \nThe analysis is performed in the context of the IARPA Babel \nproject, which aims to develop spoken term detection systems \nfor previously unseen languages rapidly, and with minimal human \neffort. We consider techniques related to letter-to-sound \nmapping and language-independent syllabification of primarily \ngraphemic systems, and discuss results obtained for six languages: \nCebuano, Kazakh, Kurmanji Kurdish, Lithuanian, Telugu \nand Tok Pisin.

Within a multilingual automatic speech recognition (ASR) system, \nknowledge of the language of origin of unknown words \ncan improve pronunciation modelling accuracy. This is of particular \nimportance for ASR systems required to deal with codeswitched \nspeech or proper names of foreign origin. For words \nthat occur in the language model, but do not occur in the pronunciation \nlexicon, text-based language identification (T-LID) \nof a single word in isolation may be required. This is a challenging \ntask, especially for short words. We motivate for the \nimportance of accurate T-LID in speech processing systems and \nintroduce a novel way of applying Joint Sequence Models to the \nT-LID task. We obtain competitive results on a real-world 4- \nlanguage task: for our best JSM system, an F-measure of 97:2% \nis obtained, compared to a F-measure of 95:2% obtained with a \nstate-of-the-art Support Vector Machine (SVM).

In this paper, we describe the “Spoken Web Search” Task, which was held as part of the 2012 MediaEval benchmark evaluation campaign. The purpose of this task was to perform audio search with audio input in four languages, with very few resources being available. Continuing in the spirit of the 2011 SpokenWeb Search Task, which used speech from four Indian languages, the 2012 data was taken from the LWAZI corpus, to provide even more diversity and allow for a task that will allow both zero resource “pattern matching” approaches and “speech recognition” based approaches to participate. In this paper, we summarize the results from several independent systems, developed by nine teams, analyze their performance, and provide directions for future research.

Grapheme-based speech recognition systems are faster to develop but typically do not reach the same level of performance as phoneme-based systems. In this paper we introduce a technique for improving the performance of standard grapheme-based systems. We find that by handling a relatively small number of irregular words through phoneme-to-grapheme (P2G) transliteration – transforming the original orthography of irregular words to an ‘idealised’ orthography – grapheme-based accuracy can be improved. An analysis of speech recognition accuracy based on word categories shows that P2G transliteration succeeds in improving certain word categories in which grapheme-based systems typically perform poorly, and that the problematic categories can be identified prior to system development. We evaluate when category-based P2G transliteration is beneficial and discuss how the technique can be implemented in practice. Index Terms: speech recognition, P2G transliteration, phoneme-to-grapheme rules, grapheme-based ASR

Introducing pronunciation variants into a lexicon is a balancing \nact: incorporating necessary variants can improve automatic \nspeech recognition (ASR) and spoken term detection (STD) \nperformance by capturing some of the variability that occurs \nnaturally; introducing superfluous variants can lead to increased \nconfusability and a decrease in performance. We experiment \nwith two very different grapheme-to-phoneme variant prediction \ntechniques and analyze the variants generated, as well as \ntheir effect when used within fairly standard ASR and STD systems \nwith unweighted lexicons. Specifically, we compare the \nvariants generated by joint sequence models, which use probabilistic \ninformation to generate as many or as few variants as \nrequired, with a more discrete approach: the use of pseudophonemes \nwithin the default-and-refine algorithm. We evaluate \nresults using three of the 2013 Babel evaluation languages \nwith quite different variant characteristics – Tagalog, Pashto and \nTurkish – and find that there are clear trends in how the number \nand type of variants influence performance, and that the implications \nfor lexicon creation for ASR and STD are different. \nIndex Terms: pronunciation variants, speech recognition, spoken \nterm detection, grapheme-to-phoneme

In this paper, we describe the “Spoken Web Search” Task, which was held as part of the 2011 MediaEval benchmark campaign. The purpose of this task was to perform audio search with audio input in four languages, with very few resources being available in each language. The data was taken from “spoken web” material collected over mobile phone connections by IBM India. We present results from several independent systems, developed by five teams and using different approaches, compare them, and provide analysis and directions for future research.

Spoken recordings that have been transcribed for human reading \n(e.g. as captions for audiovisual material, or to provide alternative \nmodes of access to recordings) are widely available in many \nlanguages. Such recordings and transcriptions have proven to be \na valuable source of ASR data in well-resourced languages, but \nhave not been exploited to a significant extent in under-resourced \nlanguages or dialects. Techniques used to harvest such data typically \nassume the availability of a fairly accurate ASR system, \nwhich is generally not available when working with resourcescarce \nlanguages. In this work, we define a process whereby an \nASR corpus is bootstrapped using unmatched ASR models in \nconjunction with speech and approximate transcriptions sourced \nfrom the Internet. We introduce a new segmentation technique \nbased on the use of a phone-internal garbage model, and demonstrate \nhow this technique (combined with limited filtering) can \nbe used to develop a large, high-quality corpus in an underresourced \ndialect with minimal effort.

European Media Laboratory GmbH, Heidelberg, Germany Multilingual Speech Technologies, North-West University, Vanderbijlpark, South Africa

Building transcribed speech corpora for under-resourced \nlanguages plays a pivotal role in developing speech technologies \nfor such languages. We have developed an open-source \ntool for devices running the Android operating system to facilitate \nthe efficient collection of speech data for Automatic Speech \nRecognition system development. The tool was designed for \nuse in typical developing-world conditions; we present the relevant \ndesign choices and analyse the effectiveness of this tool \nby means of a case study. In particular, we introduce a novel \nsemi-real-time quality monitoring system, which increases the \nefficiency of the data collection process.

We present a novel approach to automatic speaker age classification, which combines regression and classification to achieve competitive classification accuracy on telephone speech. Support vector machine regression is used to generate finer age estimates, which are combined with the posterior probabilities of well-trained discriminative gender classifiers to predict both the age and gender of a speaker. We show that this combination performs better than direct 7-class classifiers. The regressors and classifiers are trained using longterm features such as pitch and formants, as well as short-term (frame-based) features derived from MAP adaptation of GMMs that were trained on MFCCs.

We describe a new language-independent technique for automatically identifying errors in an electronic pronunciation dictionary by analyzing the source of conflicting patterns directly. We evaluate the effectiveness of the technique in two ways: we perform a controlled experiment using artificially corrupted data (allowing us to measure precision and recall exactly); and then apply the technique to a real-world pronunciation dictionary, demonstrating its effectiveness in practice. We also introduce a new freely available pronunciation resource (the RCRL Afrikaans Pronunciation Dictionary), the largest such dictionary that currently exists. Index Terms: pronunciation dictionaries, error detection, quality verification, Default&Refine, grapheme-to-phoneme, g2p

We investigate the number of speakers and the amount of data that is required for the development of useable speakerindependent speech-recognition systems in resource-scarce languages. Our experiments employ the Lwazi corpus, which contains speech in the eleven official languages of South Africa. We find that a surprisingly small number of speakers (fewer than 50) and around 10 to 20 hours of speech per language are sufficient for the purposes of acceptable phone-based recognition. Index Terms: speech recognition, corpus design

The deployment of speech technology systems in the developing world is often hampered by the lack of appropriate linguistic resources. A suitable pronunciation dictionary is one such resource that can be difficult to obtain for lesser-resourced languages. We design a process for the development of pronunciation dictionaries in resource-scarce environments, and apply this to the development of pronunciation dictionaries for ten of the official languages of South Africa. We define the semiautomated development and verification process in detail and discuss practicalities, outcomes and lessons learnt. We analyse the accuracy of the developed dictionaries and demonstrate how the distribution of rules generated from the dictionaries provides insight into the inherent predictability of the languages studied. Index Terms: pronunciation dictionaries, dictionary verification, resource-scarce, bootstrapping, Southern Bantu languages.

Spoken dialogue systems (SDSs) have great potential for information access in the developing world. However, the realisation of that potential requires the solution of several challenging problems, including the development of sufficiently accurate speech recognisers for a diverse multitude of languages. We investigate the feasibility of developing smallvocabulary speaker-independent ASR systems designed for use in a telephone-based information system, using ten resourcescarce languages spoken in South Africa as a case study. We contrast a cross-language transfer approach (using a well-trained system from a different language) with the development of new language-specific corpora and systems, and evaluate the effectiveness of both approaches. We find that limited speech corpora (3 to 8 hours of data from around 200 speakers) are sufficient for the development of reasonably accurate recognisers: Error rates are in the range 2 % to 12 % for a tenword task, where vocabulary words are excluded from training to simulate vocabulary-independent performance. This approach is substantially more accurate than cross-language transfer, and sufficient for the development of basic spoken dialogue systems. Index Terms: speech recognition, limited vocabularies, technology for the developing world

The commercial successes of spoken dialog systems in the developed world provide encouragement for their use in the developing world, where speech could play a role in the dissemination of relevant information in local languages. We investigate the evolution of spoken dialog system research in the developed world, and show that the utility of speech is based on user factors and application factors (amongst others). After adjusting the factors for the developing world context and plotting their interactions, we offer several predictions for the field. In particular, we show that the field of spoken dialog system for the developing world is in a nascent stage and will likely take another decade to have an impact similar to that in the developed world.

We investigate the addition of a new language, for which limited resources are available, to a phonotactic language identification system. Two classes of approaches are studied: in the first class, only existing phonetic recognizers are employed, whereas an additional phonetic recognizer in the new language is created for the second class. It is found that the number of acoustic recognizers employed plays a crucial role in determining the recognition accuracy for the new language. We study different approaches to incorporating a language for which audio-only data is available (no pronunciation dictionaries or transcriptions) and find that if more than about 2 000 training utterances are available, a bootstrapped acoustic model for the new language can improve accuracy substantially. Index Terms: spoken language identification, generalization, resource scarce languages

Pronunciation lexicons often contain pronunciation variants. This can create two problems: It can be difficult to define these variants in an internally consistent way and it can also be difficult to extract generalised grapheme-to-phoneme rule sets from a lexicon containing variants. In this paper we address both these issues by creating ‘pseudo-phonemes ’ associated with sets of ‘generation restriction rules ’ to model those pronunciations that are consistently realised as two or more variants. By pre-processing and post-processing the lexicon appropriately, grapheme-to-phoneme algorithms that were not able to deal with pronunciation variants previously can now be extended to incorporate variants easily, without requiring changes to the standard algorithms. We evaluate the effectiveness of this approach using the Default&Refine rule extraction algorithm, and apply the method to both the English Oxford Advanced Learners Dictionary (OALD) and the Flemish FONILEX pronunciation lexicon. We find that the approach generalises to different languages, is able to model phonemic variation accurately and is able to identify inconsistent variants in preexisting lexicons. Index Terms: pronunciation modelling, pronunciation variants, grapheme-to-phoneme rules, pseudo-phonemes.

Bootstrapping techniques are an efficient way to develop electronic pronunciation dictionaries [1, 2], but require fast system response to be practical for medium-to-large lexicons. In addition, user errors are inevitable during this process, and it is useful if automatic means can be used to assist in the search for such errors. We describe how the Default&Refine grapheme-tophoneme rule extraction algorithm [3] can be adapted to meet both of these goals. Experimental results demonstrate the utility of these methods. 1.

Much theoretical work has been done on the tonal structure of languages in the Bantu family. However, most of these studies are not supported by physical measurements, or even a consistent model for mapping from linguistic constructs to such measurements. As a first step towards addressing this deficiency, we report on initial measurements regarding the relationship between fundamental frequency and linguistic tone in isiZulu. After choosing a suitable algorithm for pitch extraction, we have correlated a number of linguistically assigned tone values with measured values for fundamental frequency. These measurements indicate a fairly complex relationship between tone and pitch, and suggest that the commonly observed ‘falling ’ tone in isiZulu may be a context-specific realization of the high tone. 1.

Several factors affect the efficiency of bootstrapping approaches to the generation of pronunciation dictionaries. We focus on factors related to the underlying rule-extraction algorithms, and demonstrate variants of the Dynamically Expanding Context algorithm, which are beneficial for this application. In particular, we show that continuous updating of the learned rules, coupled with a new approach to grapheme-to-phoneme alignment and a sliding-window approach to choosing the context window, leads to an efficient and accurate bootstrapping mechanism. 1.

Bootstrapping techniques have significant potential for the efficient generation of linguistic resources such as electronic pronunciation dictionaries. We describe a system and an approach to bootstrapping for the development of such dictionaries, and report on experiments conducted to investigate the efficiency and effectiveness of the system, focusing on the human factors that influence the process. Encouraging results were obtained: even developers with limited linguistic experience can develop accurate pronunciation models in substantially less time than a trained pronunciation expert takes using conventional methods.