Millions of multilingual multi-Indic+English users would benefit if mobile text input eschewed manual switching between monolingual transliteration keyboards. We propose to make neural transliteration tagless to enable multilingual inference, and leverage multi-word training to build up a stronger internal language model (LM) within the associated sequence-to-sequence architecture. Coupling that approach with appropriate filtering to promote or demote multilingual candidates based on external constraints paves the way toward a single pan-Indic keyboard. Experiments comparing monolingual and multilingual typing experience show that such alternative is viable to replace separate language-specific keyboards.

To achieve effective on-device textual prediction, language modeling must account for changing user style, emerging idiosyncratic expressions, and other evolving linguistic events. But typical domain adaptation techniques are not designed to preserve domain-independent behavior, leading to a data-dependent trade-off between learning and forgetting. This paper studies the Pareto efficiency of this compromise within an adversarial adaptation framework under plausible real-world conditions. The idea is that adversarial constraints placed on the (sparse) on-device distribution prevent the adapted distribution from straying too far from the (dense) initial distribution resulting from pre-training. Cross-domain experiments confirm that such distribution-level constraints help retain generic linguistic knowledge acquired during pre-training. Prediction candidates therefore suffer less from unpredictable forgetting, while being better aligned with every user’s fine-grained personal interests.

Improving keyboard autocorrection is critical to making textual input on mobile devices faster and more accurate. End-to-end sequence-to-sequence modeling is a promising avenue, but analysis of user revisions reveals that most users are not patient enough to wait until the end of the word for autocorrection to be performed. Accordingly, this paper proposes a prefix autocorrection framework, in which error detection and correction occurs at the prefix level for a more immediate response. We also introduce a new metric to assess the quality of prefix matching, which more tightly correlates with typo reduction than conventional predictive typing measures. Experiments conducted on large touch sequence datasets suggest that prefix autocorrection is a viable approach to speed up error detection and correction, and that attendant improvements in typo recovery early in the word lead to a smoother keyboard input experience.

To be perceived as trustworthy, artificially generated text must be sufficiently congruent with the available discourse history.Pre-trained language models (LMs) operating in generative mode are capable of predicting locally coherent phrases, but those do not always reflect salient syntactic, semantic, or pragmatic facets of prior content.This paper introduces a learnable evaluation metric to assess the pragmatic pertinence of LM-generated text for a given history.Pertinence is closely aligned with qualitative human judgments of acceptability, thereby emerging as a blend of sensibleness and specificity.Experiments conducted across different domains using different learning architectures show that this approach circumvents the issue of multiple valid ground-truths, while providing a reliable quantitative ranking of generated text completion candidates in context.Pertinence scoring could thus prove useful for the detection of hallucinations.

As handwriting input becomes more prevalent, the large symbol inventory required to support Chinese handwriting recognition poses unique challenges. This paper describes how the Apple deep learning recognition system can accurately handle up to 30,000 Chinese characters while running in real-time across a range of mobile devices. To achieve acceptable accuracy, we paid particular attention to data collection conditions, representativeness of writing styles, and training regimen. We found that, with proper care, even larger inventories are within reach. Our experiments show that accuracy only degrades slowly as the inventory increases, as long as we use training data of sufficient quality and in sufficient quantity.

Continuous path keyboard input has higher inherent ambiguity than standard tapping, because the path trace may exhibit not only local overshoots/undershoots (as in tapping) but also, depending on the user, substantial mid-path excursions. Deploying a robust solution thus requires a large amount of high-quality training data, which is difficult to collect/annotate. In this work, we address this challenge by using GANs to augment our training corpus with user-realistic synthetic data. Experiments show that, even though GAN-generated data does not capture all the characteristics of real user data, it still provides a substantial boost in accuracy at a 5:1 GAN-to-real ratio. GANs therefore inject more robustness in the model through greatly increased word coverage and path diversity.

Historical handwritten documents guard an important part of human knowledge only at the reach of a few scholars and experts. Recent developments in machine learning have the potential of rendering this information accessible to a larger audience. Data-driven approaches to automatic manuscript recognition require large amounts of transcribed scans to work. To this end, we introduce a new handwritten corpus based on 400-year-old, cursive, early modern Dutch documents such as ship journals and daily logbooks. This is a 1000 page collection, segmented into lines, to facilitate fully-, weakly- and un-supervised research and with textual transcriptions on 20% of the pages. Other annotations such as handwriting slant, year of origin, complexity, and writer identity have been manually added. With over 80 writers this corpus is significantly larger and more varied than other existing historical data sets such as Spanish RODRIGO. We provide train/test splits, experimental results from an automatic transcription baseline and tools to facilitate its use in deep learning research. The manuscripts span over 150 years of significant journeys by captains and traders from the Vereenigde Oost-indische Company (VOC) such as Tasman, Brouwer and Van Neck, making this resource also valuable to historians and the paleography community.

status: Published

Recent years have seen rapid growth in the deployment of statistical methods for computational language and speech processing. The current popularity of such methods can be traced to the convergence of several factors, including the increasing amount of data now accessible, sustained advances in computing power and storage capabilities, and ongoing improvements in machine learning algorithms. The purpose of this contribution is to review the state of the art in both areas, point out the top trends in statistical modelling across a wide range of problems, and identify their most salient characteristics. The paper concludes with some prognostications regarding the likely impact on the field going forward.

Submitted to Special Issue on “Computational Approaches to Analysis of Emotion in Text” Guest editors:Diana InkpenandCarlo Strapparava Initial publication:NAACL‐HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text, June 5, 2010, Los Angeles, CA Though data‐driven in nature, emotion analysis based on latent semantic analysis still relies on some measure of expert knowledge to isolate the emotional keywords or keysets necessary to the construction of affective categories. This makes it vulnerable to any discrepancy between the ensuing taxonomy of affective states and the underlying domain of discourse. This paper proposes a more general strategy which leverages two separate semantic levels: one that encapsulates the foundations of the domain considered, and one that specifically accounts for the overall affective fabric of the language. Exposing the emergent relationship between these two levels advantageously informs the emotion classification process. Empirical evidence suggests that this approach is promising for automatic emotion analysis in text. This bodes well for its deployability in a variety of applications, such as sentiment prediction.

In recent work, we introduced Latent Perceptual Mapping (LPM) [1], a new framework for acoustic modeling suitable for template-like speech recognition. The basic idea is to leverage a reduced dimensionality description of the observations to derive acoustic prototypes that are closely aligned with perceived acoustic events. Our initial work adopted a bag-of-frames strategy to represent relevant acoustic information within speech segments. In this paper, we extend this approach by better integrating temporal information into the LPM feature extraction. Specifically, we use variable-length units to represent acoustic events at the supra-frame level, in order to benefit from finer temporal alignments when deriving the acoustic prototypes. The outcome can be viewed as a generalization of both conventional template-based approaches and recently proposed sparse representation solutions. This extension is experimentally validated on a context-independent phoneme classification task using the TIMIT corpus.

To impart a congruent emotional quality to synthetic speech, it is expedient to leverage the overall polarity of the input text. This is feasible inasmuch as speech generation complies with the outcome of sentiment analysis. We have recently introduced latent affective mapping [1]-[3], a new approach to emotion detection which exploits two separate levels of semantic information: one that encapsulates the foundations of the domain considered, and one that specifically accounts for the overall affective fabric of the language. The ensuing framework exposes the emergent relationship between these two levels in order to advantageously inform affective evaluation. This paper applies latent affective mapping to the narrower problem of sentiment analysis, in order to achieve a more robust identification of the polarity of textual data. Empirical evidence gathered on the “Affective Text” portion of the SemEval-2007 corpus [4] shows that this approach is promising for automatic sentiment prediction in text. This bodes well as a first step in ensuring emotional congruence in text-to-speech synthesis.

An essential step in the generation of expressive speech synthesis is the automatic detection and classification of emotions most likely to be present in textual input. At last Interspeech, we introduced latent affective mapping, a new emotion analysis approach which leverages two separate levels of semantic information: one that encapsulates the foundations of the domain considered, and one that specifically accounts for the overall affective fabric of the language. The ensuing framework exposes the emergent relationship between these two levels in order to advantageously inform the emotion classification process. This paper presents further validation of latent affective mapping, as well as a detailed analysis of its behavior given the attendant richer emotional description. The various mapping instantiations supported compare favorably with more conventional techniques based on expert knowledge. In particular, representative case studies point to a better approximation of the true probability distribution across the range of standard emotions.

One of the fundamental building blocks of text processing for text to-speech (TTS) synthesis is the assignment of a part-of-speech (POS) tag to each input word. POS tags are heavily relied upon for downstream natural language analysis and prosody rendering. Conventional TTS POS tagging tends to resort to detailed hand crafted rules that can accommodate TTS specificities such as pertinent prosodic features, while mainstream tagging increasingly relies on data-driven statistical models trained on large but fairly generic corpora. This paper proposes a new strategy, hybrid POS tagging, which integrates these two approaches in order to achieve higher tagging accuracy. The resulting framework combines the TTS-specific advantage of rule-based tagging with the inherent robustness of broadly-trained statistical tagging. Empirical evidence underscores the viability of this framework for improving TTS quality, e.g., in regard to phrase boundary placement and homograph selection.

An essential step in the generation of expressive speech synthesis is the automatic detection and classification of emotions most likely to be present in textual input. Though increasingly data-driven, emotion analysis still relies on critical expert knowledge in order to isolate the emotional keywords or keysets necessary to the construction of affective categories. This makes it vulnerable to any discrepancy between the ensuing taxonomy of affective states and the underlying domain of discourse. This paper proposes a more general framework, latent affective mapping, which exploits two separate levels of semantic information: the first one encapsulates the foundations of the domain considered, while the second one specifically accounts for the overall affective fabric of the language. Exposing the emergent relationship between these two levels advantageously steers the emotion classification process. Empirical evidence suggests that this approach is effective for automatic emotion analysis in text. This bodes well for its deployability toward naturally expressive speech synthesis.

Unit selection text-to-speech synthesis relies on multiple cost criteria, each encapsulating a different aspect of acoustic and prosodic context at any given concatenation point. For a particular set of criteria, the relative weighting of the resulting costs crucially affects final candidate ranking. We have recently advocated a new weighting strategy based on a data-driven framework separately optimized for each concatenation. In this approach, the cost distribution in every information stream is dynamically leveraged to locally shift weight towards those characteristics that prove most discriminative at this point. To further validate this procedure, this paper presents formal listening evidence suggesting that dynamic cost weighting indeed entails higher perceived TTS quality.

Unit selection text-to-speech synthesis relies on multiple cost criteria, each encapsulating a different aspect of acoustic and prosodic context at any given concatenation point. Constraints are normally invoked on diverse characteristics such as inter-unit discontinuity, overall pitch contour, local duration profile, etc., leading to costs often too heterogeneous for a direct quantitative comparison. In order to rank available candidate units, this complexity must be reduced to a single number, and the relative importance of each information stream becomes highly critical. Yet this influence is typically determined in an empirical manner (e.g., based on a limited amount of synthesized data), yielding global weights that are thus applied to broad classes of concatenations indiscriminately. This paper proposes an alternative approach,dynamic cost weighting, based on a data-driven framework separately optimized for each concatenation considered. Specifically, the cost distribution in every stream is dynamically leveraged on a per concatenation basis to locally shift weight towards those characteristics that offer a high discrimination between candidate units, and away from those characteristics that are intrinsically less discriminative. An illustrative case study demonstrates the potential benefits of this solution, and listening evidence suggests that it does indeed entail higher perceived TTS quality.

Unit selection text-to-speech synthesis relies on multiple cost criteria, each encapsulating a different aspect of acoustic and prosodic context at any given concatenation point. For a particular set of criteria, the relative weighting of the resulting costs crucially affects final candidate ranking. Their influence is typically determined in an empirical manner (e.g., based on a limited amount of synthesized data), yielding global weights that are thus applied to all concatenations indiscriminately. This paper proposes an alternative approach, based on a data-driven framework separately optimized for each concatenation. The cost distribution in every information stream is dynamically leveraged to locally shift weight towards those characteristics that prove most discriminative at this point. An illustrative case study underscores the potential benefits of this solution.

The level of quality that can be achieved in concatenative text-to-speech synthesis is primarily governed by the inventory of units used in unit selection. This has led to the collection of ever larger corpora in the quest for ever more natural synthetic speech. As operational considerations limit the size of the unit inventory, however, pruning is critical to removing any instances that prove either spurious or superfluous. At last ICASSP we introduced an alternative pruning strategy based on a data-driven feature extraction framework separately optimized for each unit type in the inventory. This paper presents further validation of this strategy, as well as a detailed analysis of its potential benefits for concatenative synthesis.

Part-of-speech tagging is a necessary pre-processing step for many natural language tasks. Recent statistical approaches, such as conditional random fields, rely on well chosen feature functions to ensure that important characteristics of the empirical training distribution are reflected in the trained model. In practice, however, it is not always clear how to best select these feature functions in order to obtain a suitably robust model. This paper proposes an alternative strategy based on the principle of latent analogy. For each sentence under consideration, we construct a neighborhood of globally relevant training sentences through an appropriate data-driven mapping of the input surface form. Tagging then proceeds via locally optimal sequence alignment and maximum likelihood position scoring. Empirical evidence shows that this solution is competitive with state-of-the-art Markovian techniques.

The level of quality that can be attained in concatenative text-to-speech (TTS) synthesis is primarily governed by the inventory of units used in unit selection. This has led to the collection of ever larger corpora in the quest for ever more natural synthetic speech. As operational considerations limit the size of the unit inventory, however, pruning is critical to removing any instances that prove either spurious or superfluous. This paper proposes a novel pruning strategy based on a data-driven feature extraction framework separately optimized for each unit type in the inventory. A single distinctiveness/redundancy measure can then address, in a consistent manner, the two different problems of outliers and redundant units. Detailed analysis of an illustrative case study exemplifies the typical behavior of the resulting unit pruning procedure, and listening evidence suggests that both moderate and aggressive inventory pruning can be achieved with minimal degradation in perceived TTS quality. These experiments underscore the benefits of unit-centric feature mapping for database optimization in concatenative synthesis.

The level of quality that can be achieved in concatenative text-to-speech synthesis is primarily governed by the inventory of units used in unit selection. This has led to the collection of ever larger corpora in the quest for ever more natural synthetic speech. As operational considerations limit the size of the unit inventory, however, pruning is critical to removing any instances that prove either spurious or superfluous. This paper proposes a novel pruning strategy based on a data-driven feature extraction framework separately optimized for each unit type in the inventory. A single distinctiveness/redundancy measure can then address, in a consistent manner, the (traditionally separate) problems of outliers and redundant units. Experimental results underscore the viability of this approach for both moderate and aggressive inventory pruning.

The level of quality that can be achieved by modern concatenative text-to-speech synthesis heavily depends on a judicious composition of the unit inventory used in the unit selection process. Unit boundary optimization, in particular, can make a huge difference in the users' perception of the concatenated acoustic waveform. This paper considers the iterative refinement of unit boundaries based on a data-driven feature extraction framework separately optimized for each boundary region. This guarantees a globally optimal cut point between any two matching units in the underlying inventory. The associated boundary training procedure is objectively characterized, first in terms of convergence behavior, and then by comparing the distributions in inter-unit discontinuity obtained before and after training. Experimental results underscore the viability of this approach for unit boundary optimization. Listening evidence also qualitatively exemplifies a noticeable reduction in the perception of discontinuity between concatenated acoustic units

The level of quality that can be achieved in concatenative text-to-speech synthesis depends, among other things, on a judicious chiseling of the inventory used in unit selection. Unit boundary optimization, in particular, can make a huge difference in the users' perception of the concatenated acoustic waveform. This paper considers the iterative refinement of unit boundaries based on a data-driven feature extraction framework separately optimized for each boundary region. Such unsupervised boundary training guarantees a globally optimal cut point between any two matching units in the inventory. This optimization is objectively characterized, first in terms of convergence behavior, and then by comparing the average inter-unit discontinuity obtained before and after training. Experimental results and listening evidence both underscore the viability of this approach for unit boundary optimization

The level of quality that can be achieved in concatenative text-tospeech synthesis depends, among other things, on a judicious segmentation of all units in the underlying unit selection inventory. We have recently advocated the iterative refinement of unit boundaries based on a data-driven feature extraction framework separately optimized for each boundary region [1]. This paper presents the formal proof of convergence of the iterative algorithm, as well as a detailed analysis of its potential benefits for concatenative TTS synthesis. A formal listening test, in particular, underscores the practical viability of the approach for unit boundary optimization.

The level of quality that can be achieved by modern concatenative text-to-speech synthesis heavily depends on the optimization criteria used in the unit selection process. While effective cost functions arise naturally for prosody assessment, the criteria typically selected to quantify discontinuities in the speech signal do not closely reflect users' perception of the resulting acoustic waveform. This paper introduces an alternative feature extraction paradigm, which eschews general purpose Fourier analysis in favor of a modal decomposition separately optimized for each boundary region. The ensuing transform framework preserves, by construction, those properties of the waveform which are globally relevant to each concatenation considered. In addition, it leads to a novel discontinuity measure which jointly, albeit implicitly, accounts for both interframe incoherence and discrepancies in formant frequencies/bandwidths. Experimental evaluations are conducted to characterize the behavior of this new metric, first on a contiguity prediction task, and then via a systematic listening comparison using a conventional metric as baseline. The results underscore the viability of the proposed framework in quantifying the perception of discontinuity between acoustic units.

Automatic, data-driven grapheme-to-phoneme conversion is a challenging but often necessary task. The top-down strategy implicitly followed by traditional inductive learning techniques tends to dismiss relevant contexts when they have been seen too infrequently in the training data. The bottom-up philosophy inherent in, e.g., pronunciation by analogy, allows for a markedly better handling of rarer contexts, but proves nonetheless equally dependent on local, language-dependent alignments between letters and phonemes. This paper proposes an alternative bottom-up approach, dubbed pronunciation by latent analogy, which adopts a global definition of analogy, more amenable to obviate such supervision. For each out-of-vocabulary word, a neighborhood of globally relevant pronunciations is constructed through an appropriate data-driven mapping of its graphemic form. Phoneme transcription then proceeds via locally optimal sequence alignment and maximum likelihood position scoring. This method was successfully applied to the speech synthesis of proper names with a large diversity of origin.

Spoken interaction tasks are typically approached using a formal grammar as language model. While ensuring good system performance, this imposes a rigid framework on users, by implicitly forcing them to conform to a pre-defined interaction structure. This paper introduces the concept of data-driven semantic inference, which in principle allows for any word constructs in command/query formulation. Each unconstrained word string is automatically mapped onto the intended action through a semantic classification against the set of supported actions. As a result, it is no longer necessary for users to memorize the exact syntax of every command. The underlying (latent semantic analysis) framework relies on co-occurrences between words and commands, as observed in a training corpus. A suitable extension can also handle commands that are ambiguous at the word level. The behavior of semantic inference is characterized using a desktop user interface control task involving 113 different actions. Under realistic usage conditions, this approach exhibits a 2 to 5% classification error rate. Various training scenarios of increasing scope are considered to assess the influence of coverage on performance. Sufficient semantic knowledge about the task domain is found to be captured at a level of coverage as low as 70%. This illustrates the good generalization properties of semantic inference.

Latent semantic analysis suffers from a relatively high sensitivity to both task domain and composition style. Because the traditional “folding-in” process simply populates the existing semantic vector space with current data, performance degrades when training and operating conditions differ. On the other hand, recomputing the semantic space from scratch typically precludes real-time operation. An adaptation strategy therefore makes sense as a potential compromise. This paper exploits a linear transform framework to update the semantic space as new data becomes available. Leveraging suitable Cholesky factorizations makes the update computationally efficient. Experiments with different increment sizes are conducted, and the paper discusses the comparative merits of this approach under several scenarios.

An approach is presented for the dual verification of speaker identity and verbal content in a text-dependent voice authentication system. The application considered is desktop voice login over a far-field microphone. Each speaker is allowed to select his or her own keyphrase, and enrollment is limited to four instances of the keyphrase, each 1 to 2 seconds of speech. The approach decouples the analysis of speaker and verbal content information, so as to use two light-weight components for verification: a spectral matching component based on a global representation of the entire utterance, and a temporal alignment component based on more conventional frame-level information. The resulting integration is language-independent, and experiments with deliberate imposture show an equal error rate figure of approximately 4%. The approach has been commercially deployed in the "VoicePrint Password" feature of MacOS 9.

Apple's next-generation text-to-speech (TTS) system in MacOS X uses a superpositional pitch model, comprising a relatively smooth underlying F0 contour and a separate contribution from the in uence of the phonetic segments. This paper focuses on the data-driven modelling of the underlying contour, based on electroglottographic signals obtained from a corpus of reiterant speech. F0 extraction from such signals leads to more accurate characteristic shapes, as objectively illustrated by a typically low mean absolute frequency deviation (between 2 and 3 Hz) between original and synthetic F0 contours. This in turn supports a better (both more complete and more realistic) model of F0 behavior. Experimental results illustrate the improved prosodic representation resulting from this F0 model.

The increasing availability of carefully designed and collected speech corpora opens up new possibilities for the statistical estimation of formal multivariate prosodic models. At Apple Computer, statistical prosodic modeling exploits the Victoria corpus, created to broadly support ongoing speech synthesis research and development. This corpus is composed of five constituent parts, each designed to cover a specific aspect of speech synthesis: polyphones, prosodic contexts, reiterant speech, function word sequences, and continuous speech. This paper focuses on the use of the Victoria corpus in the statistical estimation of duration and pitch models for Apple's next-generation text-to-speech system in Macintosh OS X. Duration modeling relies primarily on the subcorpus of prosodic contexts, which is instrumental to uncover empirical evidence in favor of a piecewise linear transformation in the well-known sums-of-products approach. Pitch modeling relies primarily on the subcorpus of reiterant speech, which makes possible the optimization of superpositional pitch models with more accurate underlying smooth contours. Experimental results illustrate the improved prosodic representation resulting from these new duration and pitch models.

Statistical language models used in large-vocabulary speech recognition must properly encapsulate the various constraints, both local and global, present in the language. While local constraints are readily captured through n-gram modeling, global constraints, such as long-term semantic dependencies, have been more difficult to handle within a data-driven formalism. This paper focuses on the use of latent semantic analysis, a paradigm that automatically uncovers the salient semantic relationships between words and documents in a given corpus. In this approach, (discrete) words and documents are mapped onto a (continuous) semantic vector space, in which familiar clustering techniques can be applied. This leads to the specification of a powerful framework for automatic semantic classification, as well as the derivation of several language model families with various smoothing properties. Because of their large-span nature, these language models are well suited to complement conventional n-grams. An integrative formulation is proposed for harnessing this synergy, in which the latent semantic information is used to adjust the standard n-gram probability. Such hybrid language modeling compares favorably with the corresponding n-gram baseline: experiments conducted on the Wall Street Journal domain show a reduction in average word error rate of over 20%. This paper concludes with a discussion of intrinsic tradeoffs, such as the influence of training data selection on the resulting performance.

Multispan language modeling refers to the integration of various constraints, both local and global, present in the language. It was recently proposed to capture global constraints through the use of latent semantic analysis, while taking local constraints into account via the usual n-gram approach. This has led to several families of data-driven, multispan language models for large vocabulary speech recognition. Because of the inherent complementarity in the two types of constraints, the multispan performance, as measured by perplexity, has been shown to compare favorably with the corresponding n-gram performance. The objective of this work is to characterize the behavior of such multispan modeling in actual recognition. Major implementation issues are addressed, including search integration and context scope selection. Experiments are conducted on a subset of the Wall Street Journal (WSJ) speaker-independent, 20000-word vocabulary, continuous speech task. Results show that, compared to standard n-gram, the multispan framework can lead to a reduction in average word error rate of over 20%. The paper concludes with a discussion of intrinsic multi-span tradeoffs, such as the influence of training data selection on the resulting performance.

A multi-span framework was proposed to integrate the various constraints, both local and global, that are present in the language. In this approach, local constraints are captured via n-gram language modeling, while global constraints are taken into account through the use of latent semantic analysis. The performance of the resulting multi-span language models, as measured by the perplexity, has been shown to compare favorably with the corresponding n-gram performance. This paper reports on actual speech recognition experiments, and shows that word error rate is also substantially reduced. On a subset of the Wall Street Journal speaker-independent, 20,000-word vocabulary, continuous speech task, the multi-span framework resulted in a reduction in average word error rate of up to 17%.

The "sums-of-products" approach has emerged as one of the most promising avenues to model contextual influences on phoneme duration. The associated regression is generally applied after log-transforming the durations. This paper presents empirical and theoretical evidence which suggests that this transformation is not optimal. A promising alternative solution is proposed, based on a sigmoid function. Preliminary experimental results obtained on over 50,000 phonemes in varied prosodic contexts show that this transformation reduces the unexplained deviations in the data by more than 30%. Alternatively, for a given level of performance, it halves the number of parameters required by the model.

Some Single-Microphone Speech Enhancement algorithms based on the iterative Wiener filtering Method due to Lim-Oppenheim [2] are evaluated. In the original Lim-Oppenheim algorithm, AR spectral estimation of speech is carried out using a second-order analysis, but our algorithms consider an AR estimation from a cumulant analysis. This work extends some preceding papers due to the authors [4], [5]. Third- and fourth-order cumulant-based algorithms are compared to classical second-order one. This comparison is evaluated by considering three different noisy environments. A detailed study based on a frame-by-frame analysis leads to an optimum iteration of each algorithm as a trade-off between noise reduction and distortion effects. Voiced and unvoiced sounds are separately discussed. We conclude that third-order cumulant-based algorithm offers a more valuable performance than the others.

A new framework is proposed to integrate the various constraints, both local and global, that are present in the language. Local constraints are captured via n-gram language modeling, while global constraints are taken into account through the use of latent semantic analysis. An integrative formulation is derived for the combination of these two paradigms, resulting in several families of multi-span language models for large vocabulary speech recognition. Because of the inherent complementarity in the two types of constraints, the performance of the integrated language models, as measured by the perplexity, compares favorably with the corresponding n-gram performance.

A new framework is proposed to construct multispan language models for large vocabulary speech recognition, by exploiting both local and global constraints present in the language. While statistical n-gram modeling can readily take local constraints into account, global constraints have been more difficult to handle within a data-driven formalism. In this work, they are captured via a paradigm first formulated in the context of information retrieval, called latent semantic analysis (LSA). This paradigm seeks to automatically uncover the salient semantic relationships between words and documents in a given corpus. Such discovery relies on a parsimonious vector representation of each word and each document in a suitable, common vector space. Since in this space familiar clustering techniques can be applied, it becomes possible to derive several families of large-span language models, with various smoothing properties. Because of their semantic nature, the new language models are well suited to complement conventional, more syntactically oriented n-grams, and the combination of the two paradigms naturally yields the benefit of a multispan context. An integrative formulation is proposed for this purpose, in which the latent semantic information is used to adjust the standard n-gram probability. The performance of the resulting multispan language models, as measured by perplexity, compares favorably with the corresponding n-gram performance.

A new approach is proposed for the clustering of words in a given vocabulary. The method is based on a paradigm first formulated in the context of information retrieval, called latent semantic analysis. This paradigm leads to a parsimonious vector representation of each word in a suitable vector space, where familiar clustering techniques can be applied. The distance measure selected in this space arises naturally from the problem formulation. Preliminary experiments indicate that, the clusters produced are intuitively satisfactory. Because these clusters are semantic in nature, this approach may prove useful as a complement to conventional class-based statistical language modeling techniques.

In this paper we discuss various experimental results using our continuous speech recognition system on the Wall Street Journal task. Experiments with different feature extraction methods, varying amounts and type of training data, and different vocabulary sizes are reported.

One area where on-line handwriting recognition technology is most critical is the domain of small portable platforms. Because such platforms have limited resources, it is not presently practical to consider a continuous parameterization for the hidden Markov models used in the recognition. On the other hand, discrete parameter techniques such as used in speech recognition are difficult to apply, because there is no well-understood handwriting equivalent to phonological rules. A possible solution is to extract this information directly from the data, by constructing an alphabet of sub-character, elementary handwriting units. The performance of this method is illustrated on a discrete handwriting recognition task with an alphabet of 81 characters.

Speaker adaptation typically involves customizing some existing (reference) models in order to account for the characteristics of a new speaker. This work considers the slightly different paradigm of customizing some reference data for the purpose of populating the new speaker's space, and then using the resulting (augmented) data to derive the customized models. The data augmentation technique is based on the metamorphic algorithm first proposed in Bellegarda et al. [1992], assuming that a relatively modest amount of data (100 sentences) is available from each new speaker. This contraint requires that reference speakers be selected with some care. The performance of this method is illustrated on a portion of the Wall Street Journal task.

Our goal is to perform handwritten character recognition using a bank of multilayer feedforward neural networks. This paper presents both the front-end and the back-end of such a recognition system. The front-end relies on a data pre-classification scheme based on the concept of segment. A segment can be viewed as a representative building block of handwriting. The back-end hinges on a connectionist approach. Instead of a single large network, a bank of parallel networks is developed to overcome commonly encountered difficulties such as slow training process and requirement for a large amount of training data. The recognition system has been evaluated, on tasks involving (i) discrimination between similarly shaped characters and (ii) recognition of discretely written upper-case characters.>

The performance of a large alphabet handwriting recognition system based on a probabilistic framework is critically tied to the quality of the prototype distributions that are established in the relevant feature space. To better account for handwriting variability, we describe a supervised strategy for the construction of prototype distributions which are more robust to allograph deformations. The idea is to incorporate supervision to relate the allographic models to their manifestations in the feature space. This makes for a better utilization of the available training data, while at the same time allowing for a short design time turn around. The performance of this method is illustrated on a discrete handwriting recognition task with an alphabet of 81 characters.>

The automatic recognition of online handwriting is considered from an information theoretic viewpoint. Emphasis is placed on the recognition of unconstrained handwriting, a general combination of cursively written word fragments and discretely written characters. Existing recognition algorithms, such as elastic matching, are severely challenged by the variability inherent in unconstrained handwriting. This motivates the development of a probabilistic framework suitable for the derivation of a fast statistical mixture algorithm. This algorithm exhibits about the same degree of complexity as elastic matching, while being more flexible and potentially more robust. The approach relies on a novel front-end processor that, unlike conventional character or stroke-based processing, articulates around a small elementary unit of handwriting called a frame. The algorithm is based on (1) producing feature vectors representing each frame in one (or several) feature spaces, (2) Gaussian K-means clustering in these spaces, and (3) mixture modeling, taking into account the contributions of all relevant clusters in each space. The approach is illustrated by a simple task involving an 81-character alphabet. Both writer-dependent and writer-independent recognition results are found to be competitive with their elastic matching counterparts.>

Large vocabulary speaker-dependent speech recognition systems adjust to the acoustic peculiarities of each new speaker based on some enrolment data provided by this speaker. As the amount of data required increases with the sophistication of the underlying acoustic models, the enrolment may get lengthy. To streamline it, it is therefore desirable to make use of previously acquired speech data. The authors describe a data augmentation strategy based on a piecewise linear mapping between the feature space of a new speaker and that of a reference speaker. This speaker-normalizing mapping is used to transform the previously acquired data of the reference speaker onto the space of the new speaker. The performance of the resulting procedure, dubbed the metamorphic algorithm, is illustrated on an isolated utterance speech recognition task with a vocabulary of 20000 words. Results show that the metamorphic algorithm can substantially reduce the word error rate when only a limited amount of enrolment data is available. Alternatively, it leads to a level of performance comparable to that obtained when a much greater amount of enrolment data is required from the new speaker. In addition, it can also be used for tracking spectral evolution over time, thus providing a possible means for robust speaker self-adaptation.>

A new class of hidden Markov models is proposed for the acoustic representation of words in an automatic speech recognition system. The models, built from combinations of acoustically based sub-word units called fenones, are derived automatically from one or more sample utterances of a word. Because they are more flexible than previously reported fenone-based word models, they lead to an improved capability of modeling variations in pronunciation. They are therefore particularly useful in the recognition of continuous speech. In addition, their construction is relatively simple, because it can be done using the well-known forward-backward algorithm for parameter estimation of hidden Markov models. Appropriate reestimation formulas are derived for this purpose. Experimental results obtained on a 5000-word vocabulary natural language continuous speech recognition task are presented to illustrate the enhanced power of discrimination of the new models.>

In a large vocabulary speech recognition system, it is desirable to make use of previously acquired speech data when encountering new speakers. The authors describe an adaptation strategy based on a piecewise linear mapping between the feature space of a new speaker and that of a reference speaker. This speaker-normalizing mapping is used to transform the previously acquired parameters of the reference speaker onto the space of the new speaker. This results in a robust speaker adaptation procedure which allows for a drastic reduction in the amount of training data required from the new speaker. The performance of this method is illustrated on an isolated utterance speech recognition task with a vocabulary of 20000 words.>

A technique for constructing hidden Markov models for the acoustic representation of words is described. The models, built from combinations of acoustically based subword units called fenones, are derived automatically from one or more sample utterances of words. They are more flexible than previously reported fenone-based word models and lead to an improved capability of modeling variations in pronunciation. In addition, their construction is simplified, because it can be done using the well-known forward-backward algorithm for the parameter estimation of hidden Markov models. Experimental results obtained on a 5000-word vocabulary continuous speech recognition task are presented to illustrate some of the benefits associated with the new models. Multonic baseforms resulted in a reduction of 16% in the average error rate obtained for ten speakers.>

Frequency-hop pulse train signals used in applications such as coherent multiuser of multibeam echolocation must be selected on the basis of good time-frequency characteristics in both auto- and cross-ambiguity domains. A comparative analysis is presented for several classes of such signals, all algebraically constructed via congruences on a finite field. For each class, an uniform upper bound is placed on the entire cross-ambiguity function surface, and bounds are placed on the amplitude of spurious peaks in the auto-ambiguity functions. These bounds depend on time-bandwidth product and code length exclusively, which allows for a meaningful comparison across classes. Typical aspects of time-frequency behavior are shown to illustrate the associated detection and resolution properties.>

A description is presented of the authors' current research on automatic speech recognition of continuously read sentences from a naturally-occurring corpus: office correspondence. The recognition system combines features from their current isolated-word recognition system and from their previously developed continuous-speech recognition system. It consists of an acoustic processor, an acoustic channel model, a language model, and a linguistic decoder. Some new features in the recognizer relative to the isolated-word speech recognition system include the use of a fast match to prune rapidly to a manageable number the candidates considered by the detailed match, multiple pronunciations of all function words, and modeling of interphone coarticulatory behavior. The authors recorded training and test data from a set of ten male talkers. The perplexity of the test sentences was found to be 93; none of sentences was part of the data used to generate the language model. Preliminary (speaker-dependent) recognition results on these talkers yielded an average word error rate of 11.0%.>

Frequency-hop pulse train signals, used in applications such as coherent multiuser or multibeam echo location, must be selected on the basis of good time-frequency characteristics in both auto- and cross-ambiguity domains. Toward this end, a class of frequency-hop signals is developed on the basis of an extension of the theory of quadratic congruences. An uniform upper bound is placed on the entire cross-ambiguity function surface, and bounds are placed on the amplitude of spurious peaks in the auto-ambiguity function. These bounds depend on time-bandwidth product and code length exclusively. Examples of typical auto- and cross-ambiguity functions are shown to illustrate the time-frequency properties of the signals.>

The acoustic modeling problem in automatic speech recognition is estimated with the specific goal of unifying discrete and continuous parameter approaches. The authors consider a class of very general hidden Markov models which can accommodate sequences of information-bearing acoustic feature vectors lying either in a discrete or in a continuous space. More generally, the new class allows one to represent the prototypes in an assumption-limited, yet convenient, way, as (tied) mixtures of simple multivariate densities. Speech recognition experiments, reported for a large (5000-word) vocabulary office correspondence task, demonstrate some of the benefits associated with this technique.>

A continuously adaptive approach to speech encoding is presented. In contrast with other adaptation methods, it provides reliable modeling of the transition between two phonemes, and, unlike the usual block-stationary techniques, it eliminates the need to detect these transitions. Continuously adaptive linear predictive coding takes into account the inherent nonstationarity of the speech signal by using the expected minimum rate of change of the model parameters as a constraint in the recursive estimation of these parameters. The criterion considered is a constrained-least-squares cost functional which incorporates with equal weight all instantaneous errors up to the time of observation. An appropriate algorithm is given, and simulations are presented to illustrate the basic cost-performance tradeoffs involved in the approach.>

A novel approach to time-varying signal modeling is presented to reliably identify rational models in arbitrary nonstationary environments. It is motivated by the shortcomings of both adaptation methods, which cannot handle arbitrary nonstationarities, and description techniques, which tend to use unjustifiable assumptions on the observed data. Only limited a priori knowledge about the nonstationarity, namely, the expected maximum rate of change of the model parameters, is necessary to estimate these parameters online. The criterion considered is a constrained-least-squares cost functional which incorporates with equal weight all instantaneous errors up to the time of observation. An appropriate algorithm is developed and its performance is discussed to illustrate the increased confidence in time-varying model estimation which results from the approach.>

One appropriate technique for the recursive design of linear phase filters is via a minimum mean square error estimation procedure. This paper presents a different parameterization of the problem, which enforces the linear phase structure at all stages of the procedure. The computational complexity of the resulting filter design algorithm is approximately one-half that of existing algorithms, when measured as the overall number of multiplications required. This algorithm in turn leads to a new linear phase realization, simpler than a linear phase lattice/ladder structure and exhibiting better numerical properties than a direct form implementation. The improvement in finite wordlength effects is illustrated by the example design of a lowpass filter.