The recurrent neural network language model (RNNLM) has been demonstrated to consistently reduce perplexities and au-tomatic speech recognition (ASR) word error rates across a variety of domains. In this paper we propose a pre-training method for the RNNLM, by sharing the output weights of the feed forward neural network language model (NNLM) with the RNNLM. This is accomplished by first fine-tuning the weights of the NNLM, which are then used to initialise the output weights of an RNNLM with the same number of hidden units. We have carried out text-based experiments on the Penn Tree-bank Wall Street Journal data, and ASR experiments on the TED talks data used in the International Workshop on Spoken Language Translation (IWSLT) evaluation campaigns. Across the experiments, we observe small improvements in perplexity and ASR word error rate.

Speech segmentation is the problem of finding the end points of a speech utterance for passing to an automatic speech recogni-tion (ASR) system. The quality of this segmentation can have a large impact on the accuracy of the ASR system; in this pa-per we demonstrate that it can have an even larger impact on downstream natural language processing tasks – in this case, machine translation. We develop a novel semi-Markov model which allows the segmentation of audio streams into speech ut-terances which are optimised for the desired distribution of sen-tence lengths for the target domain. We compare this with exist-ing state-of-the-art methods and show that it is able to achieve not only improved ASR performance, but also to yield signifi-cant benefits to a speech translation task. Index Terms: speech activity detection, speech segmentation, machine translation, speech recognition

This paper presents a new system for automatic transcription of lectures. The system combines a number of novel features, including deep neural network acoustic models using multi-level adaptive networks to incorporate out-of-domain information, and factored recurrent neural network language models. We demonstrate that the system achieves large improvements on the TED lecture transcription task from the 2012 IWSLT evaluation - our results are currently the best reported on this task, showing an relative WER reduction of more than 16% compared to the closest competing system from the evaluation.

This paper presents the MUST-VIS system for the MediaMixer/VideoLectures.NET Temporal Segmentation and Annotation Grand Challenge. The system allows users to visualize a lecture as a series of segments represented by keyword clouds, with relations to other similar lectures and segments. Segmentation is performed using a multi-factor algorithm which takes advantage of the audio (through automatic speech recognition and word-based segmentation) and video (through the detection of actions such as writing on the blackboard). The similarity across segments and lectures is computed using a content-based recommendation algorithm. Overall, the graph-based representation of segment similarity appears to be a promising and cost-effective approach to navigating lecture databases.

This paper describes an experiment to investigate the usability of voiceprints for customer authentication in automated telephone banking. The usability of voiceprint authentication using digits (random strings and telephone numbers) and sentences (branded and unbranded) are compared in a controlled experiment with 204 telephone banking customers. Results indicate high levels of usability and customer acceptance for voiceprint authentication in telephone banking. Customers find voiceprint authentication based on digits more usable than that based on sentences, and a majority of participants would prefer to use digits.

This paper explores the consequences of adopting an alternative strategy to that of explicitly listing all options within the main menu of a speech-driven automated telephone banking service. An existing service was augmented with an overdraft request dialogue, accessible at its main menu, which could be triggered using the keyword “overdraft”. However, the keyword was not explicitly mentioned as an option in the main menu. Instead, system-initiated proposals for an overdraft were introduced into the call flow, notifying callers that they could apply for an overdraft by saying “overdraft” at the main menu. An experiment with 114 participants was carried out to investigate the effectiveness of this strategy as a way of offering new services without increasing the length of the main menu. Results showed that a significant proportion of participants (37%) did not succeed in completing an overdraft request. The reasons for this failure are discussed.

This paper addresses the problem of temporal constraints in the Viterbi algorithm in speaker-dependent and independent tasks. The results here presented suggest that in a speaker-dependent task the introduction of temporal constraints can lead to a high improvement with additive or convolutional noise, the statistical modeling of state durations is not relevant if the max and min state duration restrictions are imposed, and truncated probability densities give better results than a metric previously proposed. Finally, word position dependent and independent temporal restrictions are compared in connected word speech recognition experiments and it is shown that the former leads to better results with the same computational load. However the duration model effect could be much less significant when the acoustic model is optimized and when the training and testing conditions are matched.

This paper presents a new approach to tone modeling for continuous Mandarin speech recognition. Mandarin tones provide rich information for speech recognition. In this paper, we treat the tone as an attribute of the final vowel part of a Mandarin syllable. Separate distributions are estimated for cepstral coefficients and pitch features respectively, and the phonetic state tied-mixture technique is exploited to achieve improved modeling. Several tying structures are investigated, and the results are compared with that without using tonal parameters. After integrating tone models, decent improvements can be achieved in large vocabulary continuous Mandarin speech recognition. Besides, this approach can be easily incorporated into the one-pass Viterbi search framework for practical implementation of Mandarin dictation system.

A weighted Viterbi algorithm (HMM) is proposed and applied in combination with spectral subtraction and cepstral mean normalization to cancel both additive and convolutional noise in speech recognition. The weighted Viterbi approach is compared and used in combination with state duration modelling. The results presented show that a proper weight on the information provided by static parameters can substantially reduce the error rate, and that the weighting procedure improves better the robustness of the Viterbi algorithm than the introduction of temporal constraints with a low computational load. Finally, it is shown that the weighted Viterbi algorithm in combination with temporal constraints leads to a high recognition accuracy at moderate SNRs without the need of an accurate noise model.

This paper reports an experiment to investigate users' preferences amongst three modes of data entry in an automated home shopping service: DTMF input on the telephone keypad, and isolated word (IW) and connected word (CW) speech input. Preferences were measured both by means of attitude questionnaires and by giving participants an explicit choice among the three versions of the service once they had experienced them all. Users' attitudes to the service with a given mode of data entry were found to vary according to their cognitive skills (verbal and spatial abilities) and according to whether they had previously experienced a different data entry mode. Overall, DTMF and CW were rated similarly, and were strongly preferred to IW. Implications of these findings for the implementation of telephone-based services are discussed.

Addresses the problem of speech recognition with signals corrupted by additive noise at moderate signal-to-noise ratio (SNR). A model for additive noise is presented and used to compute the uncertainty about the hidden clean signal so as to weight the estimation provided by spectral subtraction. Weighted dynamic time warping (DTW) and Viterbi (HMM) algorithms are tested, and the results show that weighting the information along the signal can substantially increase the performance of spectral subtraction, an easily implemented technique, even with a poor estimation for noise and without using any information about the speaker. It is also shown that the weighting procedure can reduce the error rate when cepstral mean normalization is also used to cancel the convolutional noise.

This paper addresses the problem of speech recognition with signals corrupted by additive noise at moderate SNR. A technique based on spectral subtraction and noise cancellation reliability weighting in acoustic pattern matching algorithms is studied. A model for additive noise is proposed and used to compute the variance of the hidden clean signal information and the reliability of the spectral subtraction process. The results presented show that a proper weight on the information provided by static parameters can substantially reduce the error rate.

The multi-mutation rates, multi-crossover rates and a scheme of reinitialization are applied to parallel genetic algorithm for assigning the codevector indices for noisy channels for the purpose of minimizing the distortion caused by bit errors.Experimental results based on the memoryless binary symmetric channel for any bit error demonstrate the robustness of this new approach compared with our previous work [1].The property o f m ultiple global optima is also emphasized in this paper.

The problems of ecacy estimation in noise cancelling by a neural net ) and the use of this information in weighting matching algorithms are focused.Since the eect of noise on the speech signal is variable and the backpropagation training algorithm is essentially stochastic (most common patterns have more inuence in the weights re-estimation process), it is reasonable to suppose that the LIN ecacy depends on the input and each noisy frame could be associated to a reliability coecient that attempts to measure how reliable is the result of the neural net processing.Isolated word recognition experiments have shown that reliability weighting can result in a mean error rate reduction as high as 96, 80, 58 and 36 % at SNR=12, 6, 3 and 0dB, respectively, when the noise is white Gaussian.

The authors describe the results of an experiment to study the effectiveness of using acoustic stress to improve automatic speech recognition. The CSTR speech recognition system uses hidden semi-Markov models (HSMM) with a separate lexical search component. A hybrid prosodic component has been included which determines the sentence level stress and marks the vowel of stressed syllables as stressed in the phoneme lattice. Lexical stress is marked on all content words in the lexicon. Adding stress information to the system in this way resulted in a 65% reduction in word error rate and a 45% reduction in sentence error rate, relative to a baseline system without prosody.>

The recognition performances of two front ends are compared for two continuous speech recognition tasks. First, a neural network model (NNM) front end was used, with frame labeling performed by a radial basis function network and segmentation by a Viterbi algorithm. The second front end was a discrete hidden Markov model (HMM), featuring explicit state duration probability distributions. Two experiments were performed. The first used a speaker-dependent database, with a lexicon of 571 words. Using a low-perplexity grammar, the NNM front end produced a word accuracy of 94% and a sentence accuracy of 86%. This was slightly inferior to the HMM front end, which produced word accuracies of 96% and sentence accuracies of 88%. Without a grammar, word accuracies of 58% (NNM) and 49% (HMM) were recorded. The second set of experiments used the MIT portion of the TIMIT database (415 speakers and 2072 sentences in total). Results were poor for both front ends, with the NNM producing marginally better results.>

A report is presented of comparative results for vowel classification using hidden Markov models based on linear predictive coding (LPC)-based cepstral vectors and formant features. The classification accuracy is shown to be significantly improved by using time duration constraints in formant feature space, especially for the formant mel-frequency representation and its time derivative. The highest vowel recognition accuracy is obtained by integrating the two feature spaces, multiplying the probabilities computed in the separate feature spaces. This improvement of vowel recognition is extended to the more general phoneme recognition task by use of a hierarchical feature integration method, which utilizes the vowel recognition results in formant feature space together with consonant recognition based on the LPC-based cepstral feature space.>