Our aim was to automatically and appropriately classify Japanese psychomimes such as ‘ukiuki’ and ‘wakuwaku’. Such terms are important because they represent users’ emotions and have multiple meanings. Previous studies on psychomimes have not focused on these characteristics. Therefore, we focused on such characteristics by using two soft-clustering algorithms fuzzy c-means (FCM) and probabilistic latent semantic analysis (pLSA). We conducted experiments on psychomime classification. We report the result comparing both algorithms.

We previously proposed a method that uses grammatical features to detect inadequate utterances of doctors. However, nonverbal information such as that conveyed by gestures, facial expression, and tone of voice are also important. In this paper, we propose a method that uses eight acoustic features to detect three types of mental states (sincerity, confidence, and doubtfulness/acceptance). A Support Vector Machine (SVM) is used to learn these features. Experiments showed that the system's accuracy and recall rates respectively ranged from 0.79-0.91 and 0.80-0.94.

This paper describes a client-server speech translation platform designed for use at mobile terminals. Because terminals and servers are connected via a 3G public mobile phone networks, speech translation services are available at various places with thin client. This platform realizes hands-free communication and robustness for real use of speech translation in noisy environments. A microphone array and new noise suppression technique improves speech recognition performance, and a corpus-based approach enables wide coverage, robustness and portability to new languages and domains. The experimental result for evaluating the communicability of speakers of different languages shows that task completion rates using the speech translation system of 85% and 75% are achieved for Japanese- English and Japanese-Chinese, respectively. The system also has the ability to convey approximately one item of information per 2 utterances (one turn) on average for both Japanese-English and Japanese-Chinese in a task-oriented dialogue.

This paper investigates issues in preparing corpora for developing speech-to-speech translation (S2ST). It is impractical to create a broad-coverage parallel corpus only from dialog speech. An alternative approach is to have bilingual experts write conversational-style texts in the target domain, with translations. There is, however, a risk of losing fidelity to the actual utterances. This paper focuses on balancing a tradeoff between these two kinds of corpora through the analysis of two newly developed corpora in the travel domain: a bilingual parallel corpus with 420 K utterances and a collection of in-domain dialogs using actual S2ST systems. We found that the first corpus is effective for covering utterances in the second corpus if complimented with a small number of utterances taken from monolingual dialogs. We also found that characteristics of in-domain utterances become closer to those of the first corpus when more restrictive conditions and instructions to speakers are given. These results suggest the possibility of a bootstrap-style of development of corpora and S2ST systems, where an initial S2ST system is developed with parallel texts, and is then gradually improved with in-domain utterances collected by the system as restrictions are relaxed

The main goal of the present paper is to propose new schemes for the overall evaluation of a speech translation system. These schemes are expected to support and improve the design of the target application system, and precisely determine its performance. Experiments are conducted on the Japanese-to-English speech translation system ATR-MATRIX, which was developed at ATR Interpreting Telecommunications Research Laboratories. In the proposed schemes, the system’s translations are compared with those of a native Japanese taking the Test of English for International Communication (TOEIC), which is used as a measure of one’s speech translation capability. Subjective and automatic comparisons are made and the results are compared. A regression analysis on the subjective results shows that the speech translation capability of ATR-MATRIX matches a Japanese person scoring around 500 on the TOEIC. The automatic comparisons also show promising results.

An automatic translation quality evaluation method is proposed. In the proposed method, a parallel corpus is used to query translation answer candidates. The translation output is evaluated by measuring the similarity between the translation output and translation answer candidates with DP matching. This method evaluates a language translation subsystem of the Japanese-to-English ATR-MATRIX speech translation system developed at ATR Interpreting Telecommunications Research Laboratories. Discriminant analysis is then carried out to examine the evaluation performance of the proposed method. Experimental results show the effectiveness of the proposed method. The discriminant ratio is 83.5% for 2-class discrimination between absolutely correct and less appropriate translations classified subjectively. Also discussed are issues of the proposed method when it is applied to speech translation systems which inevitably make recognition errors.

In this study, we investigated how well acoustic events extracted from a cross-spectral temporal measure could be used to classify the manner and voicing of consonants. In particular, we developed seven measures that look at the strength and time difference between various onsets and offsets of acoustic energy. Consistent with findings by Shannon et al. (1995), our classification results show that manner and voicing information can be determined from dynamic temporal cues.

ATR-MATRIX is a multi-lingual speech-to-speech translation system designed to facilitate communications between two parties of different languages engaged in a spontaneous conversation in a travel arrangement domain. In this paper, we propose a new evaluation method for speech translation systems. Our current focus is on measuring the robustness of a language translation sub-system, with quick calculation and low cost. Therefore, we calculate the difference between the translation output from transcription texts and the translation output from input speech by a dynamic programming method. We present the first trial experiment of this method applied to our Japanese-to-English speech translation system. We also provide related discussions on such points as error analysis and the relationship between the proposed method and translation quality evaluation manually done by humans.

In this paper we present the first public Japanese speech corpus for large vocabulary continuous speech recognition (LVCSR) technology, which we have titled JNAS (Japanese Newspaper Article Sentences). We designed it to be comparable to the corpora used in the American and European LVCSR projects. The corpus contains speech recordings (60 hrs.) and their orthographic transcriptions for 306 speakers (153 males and 153 females) reading excerpts from the newspaper's articles and phonetically balanced (PB) sentences. This corpus contains utterances of about 45,000 sentences as a whole with each speaker reading about 150 sentences. JNAS is being distributed on 16 CD-ROMs.

In a spoken language system that uses the N-best interface between the continuous-speech recognition component and the natural language processing component, the role of the language processing component is to filter speech recognition candidates and to determine the semantic interpretation of the input. We propose as efficient chart parsing method of finding the most plausible sentence within the N-best candidate sentences and determining the semantic interpretation of the sentence, while avoiding the re-computation of the common substrings. We use syntactic and semantic constraints described in the unification-based framework and context-sensitive conditional probability CFG preferences to reorder the N-best candidates. In preliminary tests, our parser has been successful in reducing parsing steps by sharing common substrings and in selecting correct sentences first.>

The application of language models to continuous speech recognition, and the kinds of linguistic constraints important for improving speech recognition accuracy are discussed. First, the authors conducted an experiment to see how inter-phrase grammar contributes to the improvement of sentence speech recognition rates. The inter-phrase grammar was developed according to a dialogue database of an international conference secretarial service containing about a half-million words. It is confirmed that syntax is a very effective constraint for continuous speech recognition. Moreover, the predictive two-level LR parsing method is much better than the phrase lattice parsing method. The kinds of constraints that should, or can be, used to suppress each remaining error are discussed by referring to an example.>

A development system for application-specific integrated circuits is described, particularly from the point of view of digital signal processing (DSP). Knowledge-based techniques are adopted to examine various architectures and to improve the design quality of this system. It is confirmed that the system can easily deal with the DSP field by tuning the knowledge of rewriting the description of the input algorithm. Users can write the input specification as with ordinary programming languages, although care must be taken to specify the I/O variables and their timing. Statements for bit operations are prepared as standard functions. However, more general expressions to specify the concurrency and I/O timing should be prepared for users who would like to describe them clearly.>