We previously proposed a new segment quantization method called Lempel-Ziv segment quantization (LZSQ), which is a modified version of Lempel-Ziv (LZ) coding that can be applied to a continuous information source. In the present paper, LZSQ is applied to the mixed excitation linear prediction (MELP) coder, which is a standardized vocoder-type speech coder that operates at 2.4 kbit/s, in order to reduce the bit rate to below 2.4 kbit/s, while preserving the quality of the coded speech. LZSQ is applied to six coding parameters of the MELP coder to reduce the total bit rate as much as possible. As a result, the total bit rate of the modified MELP coder was reduced to 1.57 kbit/s, while the subjective quality of the modified MELP coder is equivalent to that of the previous MELP coder.

We propose an aspect-model-based reference speaker weighting. The main idea of the approach is that the adapted model is a linear combination of a set of reference speakers like reference speaker weighting (RSW) and eigenvoices. The aspect model is the mixture model of speaker-dependent (SD) models. In this paper, aspect model weighting (AMW) is proposed for finding an optimal weighting of a set of reference speakers unlike RSW and the aspect model which is a kind of cluster models is trained based on likelihood maximization with respect to the training data. The number of adaptation parameters can also be reduced using aspect model approach. For evaluation, we carried out an isolated word recognition experiment on Korean database (KLE452). The results were compared to those of conventional MAP, MLLR, RSW, and eigenvoice. Even though we use only 0.5s of adaptation data, 27.24% relative error rate reduction in comparison with speaker-independent (SI) baseline performance was achieved.

We developed a system that detects abnormal sound from sound signal observed by a surveillance microphone. Our system learns the ldquonormal soundrdquo from observation of the microphone, and then detects sounds never observed before as ldquoabnormal sounds.rdquo To this end, we developed a technique that uses multiple GMMs for modeling different levels of sound events efficiently. We also consider how to determine thresholds of GMM switching and event detection. As a result, we obtained almost same detection performance using the percentile method to the manually optimized GMMs. Besides, we exploited the segment-based feature, which gave the best result among all methods.

A new segment quantization method using the Lempel-Ziv algorithm is proposed, and it is applied to quantize line spectral frequency parameters in speech codec. The proposed segment quantizer can save four bits per frame, compared with the ITU-T G.729 speech codec (18 bits/frame), without degradation of subjective or objective speech quality

This paper describes a music information retrieval system that uses humming as the key for retrieval. Humming is an easy way for a user to input a melody. However, there are several problems with humming that degrade the retrieval of information. One problem is the human factor. Sometimes, people do not sing accurately, especially if they are inexperienced or unaccompanied. Another problem arises from signal processing. Therefore, a music information retrieval method should be sufficiently robust to surmount various humming errors and signal processing problems. A retrieval system has to extract the pitch from the user's humming. However, pitch extraction is not perfect. It often captures half or double pitches, which are harmonic frequencies of the true pitch, even if the extraction algorithms take the continuity of the pitch into account. Considering these problems, we propose a system that takes multiple pitch candidates into account. In addition to the frequencies of the pitch candidates, the confidence measures obtained from their powers are taken into consideration as well. We also propose the use of an algorithm with three dimensions that is an extension of the conventional Dynamic Programming (DP)algorithm, so that multiple pitch candidates can be treated. Moreover, in the proposed algorithm, DP paths are changed dynamically to take deltaPitches and IOIratios (inter-onset-interval) of input and reference notes into account in order to treat notes being split or unified. We carried out an evaluation experiment to compare the proposed system with a conventional system . When using three-pitch candidates with conference measure and IOI features, the top-ten retrieval accuracy was 94.1%. Thus, the proposed method gave a better retrieval performance than the conventional system.

This paper describes a method to detect smiles and laughter sounds from the video of natural dialogue. A smile is the most common facial expression observed in a dialogue. Detecting a user's smiles and laughter sounds can be useful for estimating the mental state of the user of a spoken-dialogue-based user interface. In addition, detecting laughter sound can be utilized to prevent the speech recognizer from wrongly recognizing the laughter sound as meaningful words. In this paper, a method to detect smile expression and laughter sound robustly by combining an image-based facial expression recognition method and an audio-based laughter sound recognition method. The image-based method uses a feature vector based on feature point detection from face images. The method could detect smile faces by more than 80% recall and precision rate. A method to combine a GMM-based laughter sound recognizer and the image-based method could improve the accuracy of detection of laughter sounds compared with methods that use image or sound only. As a result, more than 70% recall and precision rate of laughter sound detection was obtained from the natural conversation videos

This paper presents a new segment quantization method, which is based upon the Lempel-Ziv algorithm, and it is applied to quantize LSF segments of speech signals. The proposed quantizer is compared with other practical coders such as the G.729 and the MELP, by substituting those original LSF quantizers with the proposed segment quantizer. As a result, at the same cepstral distortion, nine bits per frame were saved compared with the scalar quantizer of the MELP (25 bits/frame), and three bits per frame were saved compared with the MA prediction VQ of the G.729 (18 bits/frame), respectively.

Speech recognition by a small robot is difficult because the robot makes noise itself. In this paper, two new methods are proposed that suppresses internal noise of the small robots. These methods are based on spectral subtraction (SS). The difference of the proposed methods from the original SS is that the proposed methods use the estimated noise spectrum dependent on the motion of the robot. One method, called MDSS, prepares the noise spectrums for all motions. Another method, called NPSS, predicts the noise spectrum from angular velocities of all joints of the robot using a neural network. From the results of the comparison between the original SS and the proposed methods, the proposed methods outperformed the conventional SS. The NPSS worked well even when the noise of the motion was unstable, while the MDSS method gave good result when the noise in one motion was stable.

Speech recognition under noisy environment is one of the hottest topic in the speech recognition research. In this paper, we propose a method to improve accuracy of spoken dialog system from a dialog strategy point of view. In the proposed method, the dialog system automatically changes its dialog strategy according to the estimated recognition accuracy in noisy environment in order to keep the performance of the system constant. In a noisy environment, the system restricts its grammar and vocabulary to improve recognition accuracy. On the other hand, the system accepts any utterance from a user in a noise-free environment. To realize this strategy, we investigated a method to avoid user’s out-of-grammar utterances through an instruction given by the system to a user. Furthermore, we developed a method to estimate recognition accuracy from features extracted from noise signal. Finally, we constructed a proposed dialog system and confirmed its effectiveness.

We have been developing a spoken dialog system. Conventional spoken dialog systems need grammar descriptions and scripts of a dialog, that are difficult to develop. The system proposed in this paper is based on semantic frames, and the system generates the recognition grammar from the frames automatically. As the system requires only a frame-based description for a task of dialog, the system can be easily applied to different kinds of tasks. Moreover, the recognition accuracy is improved by sentence weighting based on phrase class template. We evaluated the system by experiments. The system reached the goal with 2.44 user’s utterances in average.

This paper describes a dialogue-based CALL (Computer Assisted Language Learning) system. One of the major problems in CALL systems is that learners are usually assigned a passive role. Learners have no practices in composing their own utterances. The other major problem is that lots of conventional CALL systems are pronunciation exercise systems. However, pronunciation exercise is only a part of exercise needed to increase a learner’s communication skill. In this paper, we propose a dialoguebased CALL system of new concept that enables exercise of composition, grammar and conversation in addition to pronunciation.

Several CALL systems have two acoustic models to evaluate a learner’s pronunciation. In order to achieve high performance for evaluation, speaker adaptation method is introduced in CALL system. It requires adaptation data of a target language, however, a learner cannot pronounce correctly. In this paper, we proposed two types of new speaker adaptation methods for CALL system. The new methods only require learner’s utterance of the native language for adaptation. The first method is an algorithm to adapt acoustic models using bilingual’s utterances. The speakerindependent acoustic models of native and target languages are adapted to a bilingual speaker once, then they are adapted to the learner again using the learner’s speech of the native language. Phoneme recognition accuracy is about 5% higher than the baseline method. The second method is a training algorithm of an acoustic model. It can robustly train bilinguals’ model from a few bilinguals’ utterances. Phoneme recognition accuracy is about 10% higher than the baseline method.

Statistic language models obtained from a large number of training samples play an important role in speech recognition. In order to obtain higher recognition performance, we should introduce long distance correlations between words. However, traditional statistic language models such as word n-grams and ergodic HMMs are insufficient for expressing long distance correlations between words. We propose an acquisition method for a language model based on HMnet taking into consideration long distance correlations and word location.

As an input method for rapidly spreading small portable information devices, development of speaker independent speech recognition technology which can be embedded on a single DSP is now urgently requested. We have reported a speech recognition method using phoneme similarity vector as a feature vector, which is quite robust for reduction of precision of the feature parameter. We’ve also developed a recognition board with a single DSP, which works 100-word vocabulary using only the internal memory inside the DSP. [1][2] In this report, we propose a new technique which makes our recognition method more robust, where a newly introduced noise standard template together with traditional phoneme standard templates for calculating phoneme similarity vector realizes precise word-spotting. When the newly proposed noise robustness method was tested with 100 isolated word vocabulary speech of 50 subjects, recognition accuracy of 94.7% was obtained under various noisy environments.

The tree-structured speaker clustering was proposed as a highspeed speaker adaptation method. It can select the model which is most similar to a target speaker. However, this method does not consider speaker difference dependent on phoneme class. In this paper, we propose a speaker adaptation method based on speaker clustering by taking speaker difference dependent on phoneme class into account. The experimental results showed that the new method gave a better performance than the original method. Furthermore, we propose the improved method which use a tree-structure of a similar phoneme as the substitute for the phoneme which does not appear in the adaptation data. From the experimental results, the improved method gave a better performance than the method previously proposed.

N-gram model is widely applied to various pattern recognition system because it well represents local features of natural languages. In this paper, we describe a contextual postprocessing method using a trigram model of character for Japanese document recognition, and its advantage is revealed by practical experiments. The model is automatically obtained by statistical processing of training documents. The ability to reduce ambiguity is evaluated by the perplexity. In the processing, two smoothing methods are examined, and the predictive power of the deleted interpolation method is shown to be superior. For leading articles, the perplexity reduced to about 22 when using deleted interpolation. The output from OCR is processed very fast using a Viterbi algorithm. Experimental results of recognition for three kinds of documents show that the error correction rates are ranged from 75 to over 90 percent.

This paper presents the performance evaluation method on sentence recognition system which uses a finite state automaton. The relationship between word recognition score and sentence recognition score can be predicted using the number of word sequences at a short distance from the sentence. But it is not clear that how we get this number when the finite state automaton is used as linguistic information. Therefore, we propose the algorithm to calculate this number in polynomial time. Then we carry out the prediction using this method and the simulation to compare with the prediction, and it is shown that our method is usable when the quality of the word lattice is good.>

This paper describes an overview of a continuous speech recognition system composed of an acoustic processor and a linguistic processor. The system deals with 843 conceptual words and 431 functional words. We have constructed an acoustic processor using a modified learning vector quantization method (MLVQ2) for phoneme recognition. The phoneme recognition score was 85.5% for 226 sentences uttered by two male speakers. The linguistic processor is composed of a processor for spotting bunsetsu units (i.e. units similar to a “phrase” in English) and a syntactic processor. The structure of the bunsetsu unit is effectively described by a finite-state automaton, the test-set word-perplexity of which is 230. In the processor for spotting bunsetsu units, using a syntax-driven continuous-DP matching algorithm, the bunsetsu units are spotted from a recognized phoneme sequence and then a bunsetsu unit lattice is generated. In the syntactic processor, the bunsetsu unit lattice is parsed based on the dependency grammar, which is expressed as the correspondence between a FEATURE marker in a modifier-bunsetsu and a SLOT-FILLER marker in a head-bunsetsu. The recognition scores of the bunsetsu units and conceptual words were 75.2% and 88.9% respectively for 226 sentences uttered by the two male speakers.

The authors propose a phoneme recognition method based on the learning vector quantization (LVQ2) algorithm. They propose three kinds of modified training algorithms for the LVQ2 algorithm. In the recognition stage, the likelihood matrix is computed using the reference vectors and then the optimum phoneme sequence is computed from the matrix using the two-level dynamic programming (DP)-matching with duration constraints. The recognition score of phonemes in isolated spoken words was 89.1% for the test set. The phoneme recognition scores obtained by the modified LVQ2 algorithm were higher than those obtained by the original LVQ2 algorithm. The authors applied this method to a multi-speaker-dependent phoneme recognition task for continuous speech uttered Bunsetsu-by-Bunsetsu. The phoneme recognition score was 85.5% for the test speech samples in continuous speech.>

A prototype of a Japanese text dictation system has been developed. It is composed of an acoustic processor, a Bunsetsu-unit spotting processor, and a syntactic processor with semantic constraints. The acoustic processor is constructed using the modified LVQ2 method. The modified LVQ2 method achieves a high phoneme recognition performance of 86.1%. The syntax driven continuous DP matching algorithm is used for spotting Bunsetsu-units. This method greatly reduces the amount of computation and the storage capacity necessary for spotting the Bunsetsu units. Analysis of the dependency structure among the Bunsetsu-unit candidates is effectively carried out using the syntactic and semantic information.>

The perceptually based linear predictive (PLP) speech analysis method is applied to isolated word automatic speech recognition (ASR). Low dimensionality of the PLP analysis vector, which is otherwise identical in form to the standard linear predictive (LP) analysis vector, allows for computational and storage savings in ASR. We show that in speaker-dependent recognition of the alpha-numeric vocabulary, the PLP method in VQ-based ASR yields similar recognition scores as does the standard ASR system. The main focus of the paper is on cross-speaker ASR. We demonstrate in experiments with vowel centroids of two male and one female speakers that PLP speech representation is more consistent with the underlying phonetic information than the standard LP method. Conclusions from the experiments are confirmed by superior performance of the PLP method in cross-speaker isolated word recognition.

This paper describes a new automatic labeling system using speaker-dependent reference patterns for 73 phonetic units in American English. The system segments arbitrary utterances into phonetic units and automatically adapts to a new speaker using a small set of training words. The labeling of the training words begins with the words which can be easily segmented into necessary phonetic units and then reference patterns for each unit are computed by use of vector quantization clustering. Using the training reference patterns together with vocalic-consonant information, the speech input is aligned with the transcription using dynamic programming with duration constraints for each phonetic unit. More accurate phonetic boundaries are obtained using new reference patterns derived from the input speech. The system was evaluated on 15 repetitions of 104 words uttered by two males and one female. Standard deviation of differences between manually labeled and automatically obtained boundaries ranged from 21 ms to 27 ms. Most of the discrepancies occurred at the boundaries between vowels, nasals and liquids.

We describe in this paper a new phoneme recognition system using phoneme discriminant filters for speaker independent continuous speech. The segmentation of speech and the classification of phoneme are carried out simultaneously by the phoneme discriminant filters. Each of them detects the candidate of the specified phoneme in input speech independent of the other phonemes. The final recognition result is uniquely determined from the outputs of the filters through the unification process. The recognition scores are 88.1% for the training set and 76.0% for the test set uttered by the other 5 male speakers.

This paper describes the speaker-independent spoken word recognition system for a large size vocabulary. Speech is analyzed by the filter bank, from whose logarithmic spectrum the 11 features are extracted every 10 ms. Using the features the speech is first segmented and the primary phoneme recognition is carried out for every segment using the Bayes decision method. After correcting errors in segmentation and phoneme recognition, the secondary recognition of part of the consonants is carried out and the phonemic sequence is determined. The word dictionary item having maximum likelihood to the sequence is chosen as the recognition output. The 75.9% score for the phoneme recognition and the 92.4% score for the word recognition are obtained for the training samples in the 212 words uttered by 10 male and 10 female speakers. For the same words uttered by 30 male and 20 female speakers different from the above speakers, the 88.1% word recognition score is obtained.

This paper proposes a new method for the recognition of consonant based on the Perceptron model. The recognition model is composed of the sensory, feature extraction, response and lateral inhibition layers. The recognition scores of 90.4% to 98.4% are obtained for unvoiced affricates, unvoiced plosives, unvoiced and voiced fricatives.

This paper describes the newly improved spoken word recognition system. The accuracy of the word recognition is improved by implementing new features extraction method and new phoneme connecting rules in addition to the previously reported system. The new features for the phoneme recognition are the spectral local peaks and four parameters by using the least squares fit line of speech spectrum. By increasing the features for the phoneme recognition, and by using new phoneme connecting rules, the accuracies of phoneme recognition and segmentation are improved. In the last step of the system, the item of the dictionary having maximum similarity to the recognized phonemic sequence is chosen. Every item of the dictionary is written in phonemic symbols, and it is easy to change the objective words. The score of the word recognition was found to be 85.7% for 166 city names uttered by 15 male speakers.

An online automatic spoken word recognition system has been developed for the researches on the automatic recognition of speech. In this system, the spoken word is first frequency analysed with a filter bank of single tuned low selectivity filters. Three major local peaks in the spectrum and the amplitude of the speech wave are extracted every 10 ms. The frequencies of two local peaks are used for classifying the vowels, and the frequencies of three local peaks, the movements of them and the amplitude are used for classifying the semi-vowels and consonants. Input speech is thus transformed into a sequence of the notations expressing the phonemes or phoneme groups every 10 ms. The sequence is again transformed into possible phonemic strings which are called input words henceforth and are convenient for the comparison with the contents of the dictionary. The Hamming's distance between each input word and each item of the contents of the dictionary is computed where the notations of phonemes and phoneme groups are expressed by 9 bits binary vectors. The item in the dictionary nearest to one of the input word is selected as the output of the recognition system. The experiments were carried out with the utterance of five speakers from whose utterances the standard patterns for P1, P2 and Pe3 distribution had been made. The recognition score was 96% for the 20 city names involving all kinds of phonemes. The speech samples were increased to 166 city names and 82% of the utterances of three speakers were correctly recognized by adding the possible combination of phonemes to every word. Next, 13 different speakers uttered 51 city names having long distance between each other, the recognition score was found to be 94% when the speakers were permitted to repeat their utterances for three times.