For speech translation to be practical and useful, speech translation systems should be portable to multiple languages without substantial modification. The authors present results of expanding the English-based JANUS speech translation system to translate from spoken German sentences to English and Japanese utterances. The authors also report the results of implementing part of the linked predictive neural network (LPNN) speech recognition module on a massively parallel machine. The JANUS approach generalizes well, with overall system performance of 97%. This surpasses English-based JANUS performance.>

The authors present a context-dependent, phoneme and function work based, hidden control neutral network (HCNN-CDF) architecture for continuous speech recognition. The system can be seen as a large vocabulary extension of the word-based HCNN system proposed by E. Levin (1990). Two main extensions towards a large vocabulary speech recognition system are presented and discussed, i.e., the context-dependent HCNN phoneme model and the context-dependent HCNN function word model. When compared to the linked predictive neural network (LPNN) system of, J. Tebelskis (1990) significant savings in resource requirements and computational load for the HCNN-CDF implementation can be achieved. In speaker-dependent recognition experiments with perplexity 111, the current versions of the LPNN and HCNN-CDF systems achieve 60% and 75% word recognition accuracies, respectively.>

A novel type of hierarchical phoneme model for speaker adaptation, based on both hidden Markov models (HMM) and learned vector quantization (LVQ) networks is presented. Low-level tied LVQ phoneme models are trained speaker-dependently and independently, yielding a pool of speaker-biased phoneme models which can be mixed into high-level speaker-adaptive phoneme models. Rapid speaker adaptation is performed by finding an optimal mixture for these models at recognition time, given only a small amount of speech data; subsequently, the models are fine-tuned to the new speaker's voice by further parameter reestimation. In preliminary experiments with a continuous speech task using 40 context-free phoneme models at task perplexity 111, the authors achieved 82% word accuracy for speaker-dependent recognition and 73% in the speaker-adaptive mode.>

The authors present a large vocabulary, continuous speech recognition system based on linked predictive neural networks (LPNNs). The system uses neural networks as predictors of speech frames, yielding distortion measures which can be used by the one-stage DTW algorithm to perform continuous speech recognition. The system currently achieves 95%, 58%, and 39% word accuracy on tasks with perplexity 7, 111, and 402, respectively, outperforming several simple HMMs that have been tested. It was also found that the accuracy and speed of the LPNN can be slightly improved by the judicious use of hidden control inputs. The strengths and weaknesses of the predictive approach are discussed.>

The authors present JANUS, a speech-to-speech translation system that utilizes diverse processing strategies including dynamic programming, stochastic techniques, connectionist learning, and traditional AI knowledge representation approaches. JANUS translates continuously spoken English utterances into Japanese and German speech utterances. The overall system performance on a corpus of conference registration conversations is 87%. Two versions of JANUS are compared: one using a LR parser (JANUS 1) and one using a connectionist parser (JANUS 2). Performance results were mixed, with JANUS 1 deriving benefit from a tighter language model and JANUS 2 benefitting from greater flexibility.>

A large-vocabulary isolated word recognition system based on linked predictive neural networks (LPNNs) is presented. In this system, neural networks are used as predictors of speech frames, enabling a pool of such networks to serve as phoneme models. Higher-level algorithms are used to organize these networks, linking them into sequences corresponding to the phonetic spellings of words, and to train and evaluate the networks for word recognition. By virtue of linking phonemic networks, the LPNN is vocabulary independent and can be applied to large-vocabulary recognition. Recognition rates of 94% for a 234-word Japanese vocabulary of acoustically similar words and 90% for a larger vocabulary of 924 words are obtained.>