In this paper, we propose a robust classification method to distinguish between normal lung sounds from healthy subjects and abnormal lung sounds containing adventitious sounds from patients b y using lung sounds contaminated with heart sounds. Heart sounds make it difficult to perform the aforementioned classification with high accuracy. To address this problem, we propose the use of stochastic models to represent the acoustic feature of heart sounds in the classification method based on the maximum likelihood approach b y using hidden Markov models (HMMs) for the calculation of a more exact acoustic likelihood of lung sounds. Our method distinguishes between periods of adventitious sounds and heart sounds by yielding the most likely acoustic segment sequence for each respiration. For the test set of lung sounds contaminated with heart sounds, our classification method achieved a higher classification rate between normal and abnormal lung sounds compared to the conventional method without heart-sound models (88.6% vs. 87.1%, respectively). The classification of healthy and patient subjects b y using the proposed method also achieved a higher classification rate of 83.0% when two samples from different auscultation points were used.

In this paper, we propose a robust classification method to distinguish between a healthy subject and a patient with pulmonary emphysema using lung sound samples recorded from multiple auscultation points. Although the symptom of pulmonary emphysema can be determined from lung sounds that frequently include abnormal (i.e., adventitious) sounds, these are not observed in every auscultation point. Furthermore, noise pollution during auscultation makes high-accuracy detection difficult. To overcome these difficulties, our proposed method took into account lung sound samples from multiple auscultation points in diagnosing a patient. After the calculation of the acoustic likelihood for each respiratory phase based on the maximum likelihood approach using hidden Markov models and a segmental bigram, patient diagnosis was carried out based on the comparison of the average likelihood of all auscultation points between a patient and a healthy subject. Our classification method significantly increased the classification performance to 90.5% (using samples from four auscultation points) from the 82.7% classification performance of the conventional method (using a sample from one auscultation point), validating the usefulness of our proposed method.

In this paper, we propose a robust classification strategy for distinguishing between a healthy subject and a patient with pulmonary emphysema on the basis of lung sounds. A symptom of pulmonary emphysema is that almost all lung sounds include some abnormal (i.e., adventitious) sounds. However, the great variety of possible adventitious sounds and noises at auscultation makes high-accuracy detection difficult. To overcome this difficulty, our strategy is to adopt a two-step classification approach based on the detection of "confident abnormal respiration." In the first step, hidden Markov models and bigram models are used for acoustic features and the occurrence of acoustic segments in each abnormal respiratory period, respectively, to calculate two kinds of stochastic likelihoods: the highest likelihood for a segment sequence to be abnormal respiration and the likelihood for normal respiration. In the second step, the patients are identified on the basis of the detection of confident abnormal respiration, which is when difference between these two likelihoods is larger than a predefined threshold. Our strategy achieved the highest classification rate of 88.7% between healthy subjects and patients among three basic classification strategies, which shows the validity of our approach.

In this paper, we have proposed a novel classification procedure for distinguishing between normal respiratory and abnormal respiratory sounds based on a maximum likelihood approach using hidden Markov models. We have assumed that each inspiratory/expiratory period consists of a time sequence of characteristic acoustic segments. The classification procedure detects the segment sequence with the highest likelihood and yields the classification result. We have proposed two elaborate acoustic modeling methods: one method is individual modeling for adventitious sound periods and for breath sound periods for the detection of abnormal respiratory sounds, and the other is a microphone-dependent modeling method for the detection of normal respiratory sounds. Classification experiments conducted using the former method revealed that this method demonstrated an increase of 19.1% in its recall rate of abnormal respiratory sounds as compared with the recall rate of a baseline method. It has also been revealed that the latter modeling method demonstrates an increase in its recall rate for the detection of not only normal respiratory sounds but also for abnormal respiratory sounds. These experimental results have confirmed the validity of our proposed classification procedure.

This paper describes a sound source detection approach based on elaborate noise-modeling techniques for audio indexing. For accurate detection, we devised two methods to generate multiple-noise models through clustering techniques. One method is based on frame-wise data similarity, and the other is based on noise source similarity. The former method employs K-means clustering and a smoothing technique to avoid inaccurate segmentation. The latter method involves noise modeling based on a tree data structure generated by the progressive merging of noise clusters. The classification experiments show that by using these proposed methods, audio sources can be detected with better accuracy than that achieved by a conventional method. When four noise models generated by the latter method were used, the noise detection performance increased by 3.9% for the periods in which the sound sources did not overlap. With regard to the experiments for an audio stream that included overlapped segments, the noise detection performance increased by 1.2% without a decrease in the speech detection performance.

This paper proposes an emotion clustering procedure for emotion detection in infants ’ cries. Our clustering procedure is performed using the results of subjective opinion tests regarding the emotions expressed in infants ’ cries. Through the procedure, we obtain a tree data structure of emotion clusters that are generated by the progressive merging of emotions. Emotion merging is carried out on the condition that the objective function concerning the ambiguity of emotions that were detected in the opinion tests is minimized. Clustering experiments are performed on the results of opinion tests completed by infants ’ mothers and baby-rearing experts. The experimental results show that the proposed clustering, which considers the evaluation rank of each emotion, is superior to the clustering that is only concerned with the detection/nondetection of each emotion. Based on the clustering results, we performed a preliminary recognition experiment on two emotion clusters. According to the recognition results, the proposed emotion cluster achieves a detection rate of 75%, which shows the effectiveness of the proposed clustering procedure. Index Terms: emotion recognition, infants ’ cry, clustering 1.

This paper describes a system that automatically generates meeting minutes by extracting a topic hierarchy from a meeting’s speech. The topic hierarchy is a tree structure whose nodes comprise a topic summary. The topic structure extraction process converts speech recognition results into a word conceptual vector sequence and divides the sequence into the topic segments (topic segmentation). It classifies the topic segments hierarchically (segment clustering). Experimental results show that for the transcription of a meeting, the proposed algorithm is useful. Experiments on the transcription of a televised debate showed that the proposed topic segmentation algorithm is superior to the conventional method using local word frequency vectors. We also discuss experiments on the speech recognition results for the televised debate.

Current automatic speech recognition (ASR) systems have to limit their vocabulary size depending on available memory size, expected processing time, and available text data for building a vocabulary and a language model. Although the vocabularies of ASR systems are designed to achieve high coverage for the expected input data, it cannot be avoided that input data includes out-of-vocabulary (OOV) words. This is called the OOV problem. We propose dynamic vocabulary expansion using a conceptual base and multi-pass speech recognition using an expanded vocabulary. Relevant words to content of input speech are extracted based on a speech recognition result obtained using a reference vocabulary. An expanded vocabulary that includes fewer OOV words is built by adding the extracted words to the reference vocabulary. The second recognition process is performed using the new vocabulary. The experimental results for broadcast news speech show our method achieves a 30% reduction in OOV rate and improves speech recognition accuracy.

This paper proposes an English speech recognition system which can recognize both non-native (i.e. Japanese) and native English speakers' pronunciation of English speech. The system uses a bilingual pronunciation lexicon in which each word has both English and Japanese phoneme transcriptions. The Japanese transcription is constructed considering typical Japanese pronunciation of English. Japanese and English acoustic models are used in recognizing both transcriptions, and the highest-likelihood word sequence obtained in combining with native English- and Japanese-pronounced words is the recognition result. Continuous speech recognition experiments show that the proposed system greatly improves Japanese-English speech recognition performance while maintaining the same performance level as that of a purely native English recognition system.

This paper proposes an English speech recognition system which can recognize both non-native (i.e. Japanese) and native English speaker's pronunciation of English speech. The system uses a bilingual pronunciation lexicon in which each word has both English and Japanese phoneme transcriptions. The Japanese transcription is constructed considering typical Japanese pronunciation of English. Japanese and English acoustic models are used in recognizing both transcriptions, and the highest-likelihood word sequence obtained in combining with native English- and Japanese-pronounced words is the recognition results. Continuous speech recognition experiments show that the proposed system greatly improves Japanese-English speech recognition performance while maintaining the same performance levels as that of a purely native English recognition system.

This paper describes an automatic content indexing system for news programs, with a special emphasis on its segmentation process. The process can successfully segment an entire news program into topic-centered news stories; the primary tool is a linguistic topic segmentation algorithm. Experiments show that the resulting speech-based segments are fairly accurate, and scene change points supplied by an external video processor can be of help in improving segmentation effectiveness.

This paper proposes two-step generation of a variable-length class-based language model that integrates local and global constraints. In the first-step, an initial class set is recursively designed using local constraints. Word elements for each class are determined using Kullback divergence and total entropy. In the second step, the word classes are recursively and words are iteratively recreated, by grouping consecutive words to generate longer units and by splitting the initial classes into finer classes. These operations in the second step are carried out selectively, taking into account local and global constraints on the basis of a minimum entropy criterion. Experiments showed that the perplexity of the proposed initial class set is superior to that of the conventional part-of-speech class, and the perplexity of the variable-word-length model consequently becomes lower. Furthermore, this two-step model generation approach greatly reduces the training time.

This paper reports on topic extraction in Japanese broadcast-news speech. We studied, using continuous speech recognition, the extraction of several topic-words from broadcast-news. A combination of multiple topic-words represents the content of the news. This is a more detailed and more flexible approach than using a single word or a single category. A topic extraction model shows the degree of relevance between each topic-word and each word in the article. For all words in an article, topic-words which have high total relevance score are extracted. We trained the topic extraction model with five years of newspapers, using the frequency of topic-words taken from headlines and words in articles. The degree of relevance between topic-words and words in articles is calculated on the basis of statistical measures, i.e., mutual information or the /spl chi//sup 2/-value. In topic extraction experiments for recognized broadcast-news speech, we extracted five topic-words from the 10-best hypotheses using a /spl chi//sup 2/-based model and found that 76.6% of them agreed with the topic-words chosen by subjects.

This paper proposes a new approach of using a two-pass strategy to effectively recognize continuous speech including unknown words. In this approach, the first pass uses context-independent phoneme HMMs to recognize registered words and phoneme-cluster HMMs to detect unknown words. In the second pass context-dependent phone models are used for precise recognition where unknown words are transcribed. In sentence recognition experiments using this unknown-word processing, phoneme cluster models that consider the Japanese syllabic construction achieved a higher word accuracy rate of 70.3%, compared with 59.2% for sentence recognition without this processing. Furthermore, the amount of processing was reduced by about half compared with a detection method using phoneme HMMs. The total system achieves a 75.2% phoneme accuracy rate including the transcription of unknown words.

This paper proposes a large vocabulary spontaneous dialogue speech recognizer using cross-word context constrained word graphs. In this method, two approximation methods "cross-word context approximation" and "lenient language score smearing" are introduced to reduce the computational cost for word graph generation. The experimental results using a "travel arrangement corpus" show that this recognition method achieves a word hypotheses reduction of 25-40% and a cpu-time reduction of 30-60% compared to without approximation, and that the use of class bigram scores as the expected language score for each lexicon tree node decreases the word error rate 25-30% compared to without approximation.

This paper proposes a novel speaker adaptation method that exibly controls state-sharing of HMMs according to the amount of adaptation data.In our scheme, acoustic modeling is combined with adaptation to eciently utilize the acoustic models sharing characteristics for adaptation.The shared-state set of HMMs is determined by using tree-structured shared-state HMMs created from the history recorded for acoustic model generation.The proposed method is applied to the parameter-tying and parameter-smoothing techniques.Experiments have been performed on a Japanese phoneme recognition test using continuous density mixture Gaussian HMMs.Using 50 adaptation phrases, a 42% reduction in the phoneme recognition error rate from the speaker-independent model was achieved.

At ATR, a next-generation speech translation system is under development towards natural trans-language communication.To cope with the various requirements to speech recognition technology for the new system, further research efforts should emphasize the robustness for large vocabulary, speaking variations often found in fast spontaneous speech and speaker variances.These are key problems to be solved not only for speech translation but also for the general use of speech recognition in real environments.In this paper, three large speech databases are designed to cope with these problems in speech recognition and the current status of data collection is reported.

This paper proposes a novel method for speaker-independent phone modeling based on the composition and clustering method (CCL) of speaker-dependent HMMs. In general, HMM phone models are trained by the Baum-Welch (B-W) algorithm. We, however, propose a speaker-independent phone modeling in which speaker-dependent (SD) HMMs are combined to form speaker-independent (SI) HMMs without parameter reestimation. Furthermore, by using this method, we investigate how different kinds of reference speakers influence the development of the SI models. The method is evaluated in Japanese phoneme and phrase recognition experiments. Results show that the performance of this method is similar to the conventional B-W algorithm's with great reduction of computational cost.

This paper proposes a time-synchronous continuous speech recognizer driven by a context-free grammar that integrates generalized LR parser based phoneme context prediction and context-dependent HMMs. In this method, a phoneme hypotheses trie is introduced for the phoneme history representation of possible LR states, and an LR state network is introduced for LR path merging. Both techniques reduce the amount of computation. The experimental results show that this new method is more efficient than the conventional LR parser driven phoneme-synchronous continuous speech recognizer.

The paper proposes a novel speech adaptation algorithm that enables adaptation even with a small amount of speech data. This is a unified algorithm of two efficient conventional speaker adaptation techniques, which are maximum a posteriori (MAP) estimation and transfer vector field smoothing (VFS). This algorithm is designed to avoid the weaknesses of both MAP and VFS. A higher phoneme recognition performance was obtained by using this algorithm than with individual methods, showing the superiority of the proposed algorithm. The phoneme recognition error rate was reduced from 22.0% to 19.1% using this algorithm for a speaker-independent model with seven adaptation phrases. Furthermore, a priori knowledge concerning speaker characteristics was obtained for this algorithm by generating an initial HMM with the speech of a selected speaker cluster based on speaker similarity. The adaptation using this initial model reduced the phoneme recognition error rate from 22.0% to 17.7%.

This paper describes a speech recognition system that uses a new stochastic language model that integrates local and global constraints. Dependencies within adjacent words are used as local constraints in the same way as in conventional word N-gram models. To capture the global constraints between non-contiguous words, the sequence of the function words and that of the content words are taken into account. Furthermore, it is shown that, assuming an independence between local- and global constraints, the number of parameters to be estimated and stored is greatly reduced. The proposed language model is incorporated into a speech recognizer based on the time-synchronous Viterbi algorithm, and compared with the word bigram model and trigram model. The experimental results show that the proposed method is able to capture linguistic constraints effectively.>

The paper proposes an unsupervised speaker adaptation method using "all-phoneme ergodic hidden Markov network" that combines allophonic (context-dependent phone) acoustic models with stochastic language constraints. Hidden Markov networks (HMnet) for allophone modeling and allophonic bigram probabilities derived from a large text database are combined to yield a single large ergodic HMM which represents arbitrary speech signals in a particular language so that the model parameters can be re-estimated using text-unknown speech samples with the Baum-Welch algorithm. Combined with the vector field smoothing (VFS) technique, unsupervised speaker adaptation can be effectively performed. This method experimentally gave fairly better performances compared with the authors' previous unsupervised adaptation method using conventional phonetic HMMs and phoneme bigram probabilities.>

We describe recent improvements in ATR's experimental speech recognition system ATREUS, which serves as a recognition font end for the speech translation system ASURA. Our next goal is spontaneous speech translation. To constrain the potentially huge search space, better prosodic control, better probabilistic language models and better acoustic models are proposed. The SSS-LR parser was modified to work with non-uniform unit type acoustic and duration models. Experimental results showed, that, for example, use of mora trigram probabilities improved the phrase error rate from 17% to 14%.>

This paper describes a phonetic typewriter and a dictation machine that utilize the underlying statistical structure of phoneme or character sequences. The approach of using syllable or character trigrams is applied to language source modeling. The language source models are obtained by calculating trigram probabilities from a large text database. These models are combined with the HMM-LR continuous speech recognition system.3,6 The phonetic typewriter is tested using 274 phrases uttered by one male speaker. The syllable source model achieves a 94.9% phoneme recognition rate with the test-set phoneme perplexity of 3.9. Without the syllable source model, the phoneme recognition rate is only 73.2%. A trigram model based on characters is also evaluated. This character source model can reduce the syllable perplexity significantly to 7.7, compared with 10.5 of the syllable source model. The character source model achieves a 78.5% character transcription rate for the 274 phrase utterances. The experimental results show that a syllable source model and a character source model are very effective for realizing a Japanese dictation machine.

The authors describe two approaches for adapting a specific syllable trigram model to a new task. One uses a small amount of text data similar to the target task, and the other uses supervised learning using the most recent input phrases. The effect of each adaptation is verified with syllable perplexity and phrase recognition. Where the syntactic knowledge was only the syllable trigram model, the perplexity was reduced from 54.5 to 18.1 for the adaptation using 100 phrases of similar text, and was reduced to 14.6 by the supervised learning. The recognition rates were also improved from 42.3% to 46.6% and 50.9%, respectively. Text similarity for speech recognition is also studied.>

The authors describe a Japanese dictation system that uses a stochastic language model based on sequences of Japanese characters. The trigram probabilities, which are obtained from a text database consisting of Kanji and Kana are used to construct a source model. A Japanese dictation system generally requires Kana-to-Kanji conversion if the system uses a phoneme based unit for the acoustic processing. However, a system that uses a Kanji-and-Kana character source model can generate an output Kanji-and-Kana sequence directly from input speech without using Kana-to-Kanji conversion. The system is tested using 274 phrases uttered by one male speaker, and achieves 58.4% phrase transcription rate. When the system uses a pronunciation dictionary and eliminates the candidates whose Kanji readings are contextually inappropriate, the phrase transcription rate increases to 63.9%. It is confirmed that a Japanese character source model is efficient for a Japanese dictation system.>

A phonetic typewriter that utilizes the underlying statistical structure of phoneme/character sequences is described. The syllable/character trigram approach to language modeling is adopted to make language source models. These are obtained by calculating trigram probabilities, using a large text database. The phonetic typewriter is tested using 279 phrases uttered by one male speaker, and the syllable source model achieves a 94.9% phoneme recognition rate with the test-set phoneme perplexity of 3.9. Without the syllable trigram, the phoneme recognition rate is only 73.2%. A trigram model based on characters is also evaluated. This model can reduce the phoneme perplexity significantly compared with that of the syllable trigram.>

A Japanese continuous-speech recognition system is described which is based on phonetic hidden Markov models (HMMs) combined with two levels of grammatical representations: an intraphase transition network grammar and an interphase dependency grammar. A joint score, combining acoustic likelihood and linguistic certainty factors derived from phonetic HMMs and two levels of grammar, is maximized to obtain the optimal recognition results of sentences. Two efficient algorithms, bidirectional network parsing and breadth-first dependency parsing, are deviced to optimize the joint score globally. The system attains a phrase recognition rate of 80.8% with the intraphase parser only and 86.8% with both the intraphase and interphase parsers, where the perplexity of the phrase syntax is 40. This result shows the effectiveness of the two-level grammar approach.>