Spoken Document Retrieval (SDR) and Spoken Term Detection (STD) have been two of the most intensively investigated topics in spoken document processing research according to the establishment of the SDR and STD test collections by the Text REtrieval Conference (TREC) and NIST. Because Japanese spoken document processing researchers also requires such test collections for SDR and STD, we have established a working group to develop these collections in Special Interest Group-Spoken Language Processing (SIG-SLP) of the Information Processing Society of Japan. The working group has constructed and made available a test collection for SDR, and is now constructing new test collections for STD that will be open to researchers. The present paper introduces the policies, outline, and schedule of the new test collections. Then, the new test collections are compared with the NIST STD test collections. Index Terms: spoken term detection, test collection 1.

This paper proposes a new algorithm for retrieving sound based on successive relative search. In retrieving musical sound focusing on its sound features, emotional representations are more appropriate than conventional keywords of the genre or the composer. A vector-based sound retrieval system "Sound Advisor" was built up using emotional parameters. The problem is how to find a better candidate, if the first candidate is not satisfactory. The reason is that it is difficult to quantitatively image the emotional vector space of sounds. This paper proposes a method for successively retrieving sounds using relative search regarding the found candidate as the reference base. This method can contribute to sound-effect retrieval for producing movies, and emotional communication via sound.

This paper outlines our approach for describing multivariate environmental information such as weather as it might be characterized by humans using connotations and delicate nuances. The purpose of this research is to achieve smooth human-machine spoken dialogue. The key feature of our approach is the use of a vector-based method, a widely used technique in Information Retrieval research. To date, no system concept as mentioned or the use of the vector-space in such a system has been reported. First, phrases implicitly expressing environmental information are collected to make the list of candidate phrases. Next the environmental conditions of that time are entered from the GUI. The phrases and the newly entered information are represented as vectors of component values representing such information. By computing the similarity (relatedness) between the given condition and each candidate phrase, the phrase that best represents the given condition is chosen. Two experimental systems that work in Japanese have been developed to evaluate the validity of our method. The first system selects the appropriate phrase to represent the weather implicitly with a minimum number of phrases and not by lining up measured values. The other system selects poetic phrases that most appropriately represent the environmental conditions, including weather information and emotional states. The result of our evaluation has shown that both systems successfully selected the suitable phrases for representing given environmental conditions.

This paper concerns the discourse understanding process in spoken dialogue systems. This process enables the system to understand user utterances based on the context of a dialogue. Since multiple candidates for the understanding result can be obtained for a user utterance due to the ambiguity of speech understanding, it is not appropriate to decide on a single understanding result after each user utterance. By holding multiple candidates for understanding results and resolving the ambiguity as the dialogue progresses, the discourse understanding accuracy can be improved. This paper proposes a method for resolving this ambiguity based on statistical information obtained from dialogue corpora. Unlike conventional methods that use hand-crafted rules, the proposed method enables easy design of the discourse understanding process. Experiment results have shown that a system that exploits the proposed method performs sufficiently and that holding multiple candidates for understanding results is effective.

This paper describes a method for creating an evaluation measure for discourse understanding in spoken dialogue systems. Discourse understanding means utterance understanding taking the context into account. Since the measure needs to be determined based on its correlation with the system’s performance, conventional measures, such as the concept error rate, cannot be easily applied. Using the multiple linear regression analysis, we have previously shown that the weighted sum of various metrics concerning dialogue states can be used for the evaluation of discourse understanding in a single domain. This paper reports the progress of our work: verification of our approach by additional experiments in another domain. The support vector regression method performs better than the multiple linear regression method in creating the measure, indicating non-linearity in mapping the metrics to the system’s performance. The results give strong support for our approach and hint at its suitability as a universal evaluation measure for discourse understanding. 1.

This paper proposes a new noise-robust spectral estimation method for speech recognition. The new method, called PHASOR, is characterized by inside-frame processing. The speech spectrum is estimated from a single impulse response obtained by summing multiple pitch periods in a frame with synchronizing the phase. PHASOR improves the spectral estimation accuracy and suppresses the additive noise because of the inside-frame processing. These improvement is more effective when the pitch fluctuates or changes in the frame. Speaker-dependent and speaker-independent phoneme recognition experiments demonstrate that the PHASOR greatly reduces the recognition error rate for speech data contaminated by noise. It also outperforms conventional noise reduction methods, cepstral mean normalization and spectral subtraction.

In single utterance understanding, which does not include discourse understanding, the concept error rate (CER), or the keyword error rate, has been widely used as an evaluation measure for utterance understanding. However, the CER cannot be used for evaluating systems that understand user utterances based on previous user utterances. In this paper, we propose a method for evaluating incremental utterance understanding, which involves speech recognition, language understanding and discourse processing in spoken dialogue systems, by finding a measure that correlates closely with the system’s performance based on dialogue states and their way of update. We defined dialogue performance by task completion time, and performed a multiple linear regression analysis using task completion time as the explained variable and various metrics concerning dialogue states as explaining variables. The obtained multiple regression model fits comparatively well and shows validity as an evaluation measure. 1.

This paper presents a method for deciding the timing of turn-taking in spoken dialogue systems. This method uses a decision tree learned from the corpus of dialogues between human users and systems in which desirable turn-taking behaviors are annotated by hand. It utilizes a variety of attributes, such as recognition and understanding results and prosodic information. Unlike most of the existing systems it enables spoken dialogue systems to decide the timing of turn-taking based on not only pauses but also other features, so that users can speak to the system even if they put pauses in the middle of their utterances. The result of a preliminary experiment shows that the learned decision tree outperforms the baseline strategy, which takes turn at every user pauses. 1.

This paper describes new feature parameters for detecting misunderstandings in a spoken dialogue system. Although recognition errors cannot be completely avoided with current speech recognition techniques, a spoken dialogue system could be a good human-machine interface if it could automatically detect and recover from its own misunderstandings during natural interaction between it and a user. For this purpose, we collected user responses to system con rmations with/without the system misunderstandings using the wizard of OZ method so that we could analyze the di erences in the characteristics of user responses following correct/incorrect con rmations. The experimental results demonstrate that the content and duration of the user responses are good feature parameters for detecting the system's misunderstandings.

The rejection of unknown words is important in improving the performance of speech recognition. The anti-keyword model method can reject unknown words with high accuracy in a small vocabulary and specified task. Unfortunately, it is either inconvenient or impossible to apply if words in the vocabulary change frequently. We propose a new method for task independent rejection of unknown words, where a new phoneme confidence measure is used to verify partial utterances. It is used to verify each phoneme while locating candidates. Furthermore, the whole utterance is verified by a phonetic typewriter. This method can improve the accuracy of verification in each phoneme, and improve the speed of candidate search. Tests show that the proposed method improves the recognition rate by 4% compared to the conventional algorithm at equal error rates. Furthermore, a 3% improvement is obtained by training acoustic models with the MCE algorithm.

This paper investigates a new method for creating speaker models that are robust against utterance variation in continuous distribution hidden Markov model-based speaker verification. In this method, the distribution of the session-independent features for each speaker is estimated by separately modeling the session-to-session utterance variation as two distinct variations: one session-dependent and the other session-independent. In practice, joint normalization of the session-dependent utterance variation and estimation of the parameters of speaker models is performed based on a speaker adaptive training algorithm. The resulting speaker models more accurately represent session-independent speaker characteristics, and the discriminatory capabilities of these models increases. In text-independent speaker verification experiments using data uttered by 20 speakers in 7 sessions over 16 months, we show that the proposed method achieves a 15% reduction in the error rate.

This paper investigates a method of creating robust speaker models that are not sensitive to session-dependent (SD) utterance-variation and handset-dependent (HD) distortion for hidden Markov model (HMM)-based speaker veri cation systems in a real telephone network. We recently reported a method of creating session-independent (SI) speaker-HMMs that are not sensitive to SD utterance-variation. In that method, the distortion function that transforms SI speaker-HMMs to SD speaker-HMMs is introduced, and the parameters in the function and the speaker-HMM parameters are jointly estimated using a speaker adaptive training algorithm. This paper proposes a new method that is less sensitive to SD utterance-variation and HD distortion than the previous method. This new idea focuses on di erent di culties in estimating parameters in distortion functions for SD utterance-variation and HD distortion. In text-independent veri cation experiments with recorded data from di erent handsets, the error reduction rate of the improved method compared with that of the conventional method of cepstral mean normalization is 24% when each speaker-HMM is recreated using data uttered in ve sessions.

Unsupervised speaker adaptation plays an important role in "batch dictation", the aim of which is to automatically transcribe large amounts of recorded dictation using speech recognition. In the case of unsupervised speaker adaptation which uses recognition results of target speech as the means of supervision, erroneous recognition results degrade the quality of the adapted acoustic models. This paper presents a new supervision selection method. By using this method, correction of the first candidate is judged based on the likelihood ratio of the first and the second candidates. This method eliminates erroneous recognition results and corresponding speech data from the adaptive training data. We implemented this method in the iterative unsupervised speaker adaptation procedure. It is shown that the recognition errors are drastically reduced by 50% in a practical application of batch-style speech-to-text conversion of recorded dictation of Japanese medical diagnoses compared with speaker-independent recognition.

This paper proposes a new type of acoustic model called the discrete mixture HMM (DMHMM). As large scale speech databases have been constructed for speaker-independent HMMs, continuous mixture HMMs (CMHMMs) are needed to increase the number of mixture components in order to represent complex distributions. This leads to a high computational cost for calculating the output probabilities. The DMHMM represents the feature parameter space by using the mixtures of multivariate distributions in the same way as the diagonal covariance CMHMM. Instead of using Gaussian mixtures to represent the feature distributions in each dimension, the DMHMM uses the mixtures of the discrete distributions based on scalar quantization (SQ). Since the discrete distribution has a higher degree-of-freedom in terms of representation, the DMHMM is advantageous in representing the feature distributions efficiently with fewer mixture components. In isolated word recognition experiments for telephone speech, we have found that the DMHMM outperformed the CMHMMs when those models had the same number of mixture components.

To produce near-native American English (AE) vowels, Japanese speakers must extend their five vowel system with at least six new vowels. Three experiments have been conducted to acquire both perceptual and acoustic measures about Japanese accented English (JE)vowels. Six non-back vowels of AE in several phonetic environments were recorded from four Japanese male talkers with moderate English skills. Intelligibility was assessed by a panel of six Americans as the percent of JE vowels identified as intended. The first experiment was an open-set identification task. Two vowels, /i/ and /e/, were fully intelligible (>98%) while others ranked from 81% (/e/) to 23% (/#/) intelligible. The second experiment used minimal-pair responses to assess intelligibility in terms of three acoustic properties of vowels, the spectral target, dynamic formants and duration. The results indicated that the spectral property was not communicated effectively, dynamics were partially effective, and duration was used effectively. In the third experiment a stepwise multiple linear regression analysis determined how these acoustic properties contributed to the intelligibility. For the two vowels analyzed, /w/ and //, the direction and extent of the formant movement of JE vowels in relation to the AE targets was the vowel property that contributed the most to intelligibility.

The dynamic cepstrum parameter representing a masked spectrum performed extremely well in continuous speech recognition. This paper proposes a new algorithm for optimizing the dynamic cepstrum lifter array. The masking filter is represented by a set of Gaussian-shaped lifters. The standard deviation and the gain of the Gaussians are trained in order to improve the performance of the time-frequency filter. Parameterizing the lifter shape provides robustness against unknown speech samples. Because of the parameterized lifter's small degree of freedom, it can avoid over-learning. The gradient descent optimizing algorithm is formulated for both a neural network classifier and an HMM classifier. The optimized dynamic cepstrum successfully improved the speech recognition performance for the speech spoken even in a different speaking style.>

The authors propose a new mathematical method for extracting spectral movement in a time-sequence of speech spectrum. The spectral movement is characterized by the time and frequency derivative of a time-sequence of log spectrum envelopes. Spectral movement direction, the movement toward a higher or lower frequency region, can be identified by the sign of the proposed function. A parameter which can be used for speech segmentation is derived form this function A distance measure for speech recognition is also derived as the Euclidean distance between two spectral movement patterns extracted by the proposed function. This distance is easily calculated using cepstrum coefficients. Speech recognition results using dynamic time warping template matching with this new distance measure indicate that recognition error rate can be reduced to less than half compared with the conventional Euclidean cepstrum distance measure.>

This paper proposes a new automatic rule generation approach for rule-based consonant discrimination. A rule unit used here is represented by a conditional branch. All rules are optimally generated for the discrimination of the consonant learning samples. Clear and simple rule description permits immediate evaluation as well as further improvement. This approach is effective for the generation of a large number of complicated rules, which is difficult using the conventional heuristic approach. This paper also shows that a more precise discrimination rule set can be obtained by generating rules depending on the succeeding vowel. The experimental results for 100 Japanese city names spoken by 50 speakers indicate that the consonant discrimination rate is 73.7 % employing the resulting vowel dependent rules.

This paper describes a new spoken word recognition approach based on the top-down phoneme segmentation. Fourteen phoneme recognition functions are introduced to deal with various coarticulations. This new approach has two advantages. First, a precise phoneme recognition can be achieved because of the phoneme level top-down verification. Second, only phoneme symbol sequences are required for the vocabulary knowledge source, because the coarticulation knowledge is included in phoneme level knowledge sources. Experimental recognition results for 100 city names uttered by 50 speakers indicate that the phoneme concatenations showing strong coarticulation must be segmented as a unit to achieve a high recognition rate.