In this study, we conducted multi-label annotation based on textual entailment for text data related to incident cases that occurred at an electric power company, utilizing GPT, a general-purpose LLM, without additional training. The experiment examined GPT’s zero-shot textual entailment performance and, for comparison, its one-shot textual entailment performance using prompt engineering. Furthermore, in this study, the abstract category labels for the causes of incidents used in the annotation task were also extracted zero-shot from GPT-4, and these were approved by human participants to determine the labels. The results of the experiment showed that, particularly in the one-shot approach using prompt engineering, GPT exhibited strong generalization capabilities and demonstrated promising performance, approaching the level of human annotators in certain evaluation metrics. However, it was also suggested that when dealing with highly specialized and multifaceted cases like those in this study, careful adjustments in model choice and prompt settings are required.

In this paper, we propose to use staying area data toward the estimation of serving time for customers. To classify utterances enables us to estimate conversation types between speakers. However, its performance becomes lower in real environments. We propose a method using area data with sound data to solve this problem. We also propose a method to estimate the conversation types using the decision trees. They were tested with the data recorded in a Japanese restaurant. In the experiment to classify utterances, the proposed method performed better than the method using only sound data. In the experiment to estimate the conversation types, we succeeded to recover 70% of the mis-classified conversations using both of sound and area data.

Voice Conversion (VC) is a technique to convert speech data of source speaker into ones of target speaker. VC has been investigated and statistical VC is used for various purposes. Conventional VC uses acoustic features, however, the audio-only VC has suffered from the degradation in noisy or real environments. This paper proposes an AudioVisual VC (AVVC) method using not only audio features but also visual information, i.e. lip images. Eigenlip feature is employed in our scheme as visual feature. We also propose a feature selection approach for audio-visual features. Experiments were conducted to evaluate our AVVC scheme comparing with audio-only VC, using noisy data. The results show that AVVC can improve the performance even in noisy environments, by properly selecting audio and visual parameters. It is also found that visual VC is also successful. Furthermore, it is observed that visual dynamic features are more effective than visual static information.

We have previously described an auto-regressive hidden Markov model (AR-HMM) and an accompanying parameter estimation method. The AR-HMM was obtained by combining an AR process with an HMM introduced as a non-stationary excitation model. We demonstrated that the AR-HMM can accurately estimate the characteristics of both articulatory systems and excitation signals from high-pitched speech. In this paper, we apply the AR-HMM to feature extraction from singing voices and evaluate the recognition accuracy of the AR-HMM-based approach.

This paper describes a novel speech interface function, called speech completion, that helps a user enter a word or phrase by completing (filling in the rest of) a phrase fragment uttered by the user. Although the concept of completion is widely used in text-based interfaces, there have been no reports of completion being effectively applied to speech. By using a filled pause, we enable a user to effortlessly invoke the speech-completion function which helps the user recall uncertain phrases and saves labor when the input phrase is long. When a user hesitates by lengthening a vowel (a filled pause is uttered) during a phrase, our system immediately displays completion candidates whose beginnings acoustically resemble the uttered fragment so that the user can select the correct one. In our experiments with a system that included a filled-pause detector and a speech recognizer capable of listing candidates, the effectiveness of speech completion was confirmed.

A method of speech enhancement using microphone-array signal processing based on the subspace method is proposed and evaluated. The method consists of the following two stages corresponding to the different types of noise. In the first stage, less-directional ambient noise is reduced by eliminating the noise-dominant subspace. It is realized by weighting the eigenvalues of the spatial correlation matrix. This is based on the fact that the energy of less-directional noise spreads over all eigenvalues while that of directional components is concentrated on a few dominant eigenvalues. In the second stage, the spectrum of the target source is extracted from the mixture of spectra of the multiple directional components remaining in the modified spatial correlation matrix by using a minimum variance beamformer. Finally, the proposed method is evaluated in both a simulated model environment and a real environment.

This paper describes a method for automatically detecting filled (vocalized) pauses, which are one of the hesitation phenomena that current speech recognizers typically cannot handle. The detection of these pauses is important in spontaneous speech dialogue systems because they play valuable roles, such as helping a speaker keep a conversational turn, in oral communication. Although a few speech recognition systems have processed filled pauses within subword-based connected word recognition or word-spotting frameworks, they did not detect the pauses individually and consequently could not consider their roles. In this paper we propose a method that detects filled pauses and word lengthening on the basis of small fundamental frequency transition and small spectral envelope deformation under the assumption that speakers do not change articulator parameters during filled pauses. Experimental results for a Japanese spoken dialogue corpus show that our real-time filled-pause-detection system yielded a recall rate of 84.9 % and a precision rate of 91.5%.

This paper describes a multimodal database which consists of image data of human gestures and corresponding speech data for the research on multimodal interaction systems.The purpose of this database is to provide an underlying foundation for research and development of multimodal interactive systems.Our primary concern in selecting utterances and gestures for inclusion in the database was to ascertain the kinds of expressions and gestures that artificial systems could produce and recognize.Total 25 kinds of gestures and speech were repeated four times for the recording of each subject.The speech and gestures for a total of 48 subjects were recorded, converted into files and in the first version, the files for 12 subjects were recorded on CD-ROMs.

A method for recovering the LPC spectrum from a microphone array input signal corrupted by ambient noise is proposed. This method is based on the CSS (coherent subspace) method, which is designed for DOA (direction of arrival) estimation of broadband array input signals. The noise energy is reduced in the subspace domain by the maximum likelihood method. To enhance the performance of noise reduction, elimination of the noise-dominant subspace using projection is further employed, which is effective when the SNR is low and classification of noise and signals in the subspace domain is difficult. The results of the simulation show that some small formants, which cannot be estimated by the conventional delay-and-sum beamformer, were well estimated by the proposed method.

A design of mobile robot for robust life-long navigation in office environment is proposed and evaluated. The key idea is combining probabilistic map and dialog with humans for reducing the location uncertainty. Bayesian inference with the map represented by probabilistic automata is used in order to reduce the number of queries and to evaluate the success rate of planned paths. We experimentally implemented the design using a simple Bayesian network with continuous nodes and demonstrated its effectiveness in a real environment.

The paper describes a new method for representing and identifying isolated word accent patterns. The word accent patterns are represented by concatenation of mora hidden Markov models for fundamental frequency feature sequences. The mora HMMs are trained by accent-related features automatically extracted without manual correction from the speech wave. These algorithms are evaluated using a speech sample set consisting of 10 speakers' 121 words, where word accent patterns are classified by listening. All words have 4 mora and are selected from a phonetically balanced word set. Two experiments are performed to compare the automatically extracted features with the features manually corrected in unvoiced parts of the speech wave. Little difference was found in the results obtained using the two different features, indicating that the mora HMMs using automatically extracted features are useful for representing and identifying word accent patterns.>

This paper describes the relation between vocabulary sizes and detection errors of unknown words in large vocabulary speech recognition through recognition and detection experiments.Although the relation between vocabulary sizes and recognition performances has been reported, the relation between vocabulary sizes and detection performances has not yet been studied.Especially, it has not for the cases of vocabulary sizes of over 1,000 words.Experiments were conducted using the speech material of speaker MAU's ATR word speech database.The entries of the dictionary used is 40,000 words from the Shinmeikai Japanese Language Dictionary.It is shown that when the vocabulary size increases from 1,000 words to 40,000 words, the relation between vocabulary sizes and detection errors has a similar tendency with the relation between vocabulary sizes and recognition errors.And increases of detection errors caused by increases of vocabulary sizes are shown to be small for the case of within vocabulary, compared with increases of detection errors for the case of out of vocabulary.These results should be taken into accounts in designing large vocabulary speech recognition systems including unknown word processing.

The purpose of this paper is to introduce a new method for analyzing the amino acid sequences of proteins using the hidden Markov model (HMM), which is a type of stochastic model. Secondary structures such as helix, sheet and turn are learned by HMMs, and these HMMs are applied to new sequences whose structures are unknown. The output probabilities from the HMMs are used to predict the secondary structures of the sequences. The authors tested this prediction system on approximately 100 sequences from a public database (Brookhaven PDB). Although the implementation is 'without grammar' (no rule for the appearance patterns of secondary structure) the result was reasonable.

The authors present an explicit criterion for deciding whether to divide the hidden Markov model (HMM)-based phone model into more precise (for example, context dependent) models, or not. They also discuss the cases when linear interpolations of these models are used, and present an explicit solution of the interpolation weight lambda . These criteria are obtained by evaluating the estimation errors of the distributions of these models. The authors show that the estimation errors should be evaluated by the projected covariance matrices of the estimation errors of the logarithm of the probabilities.>

A continuous speech recognition system 'niNja' (Natural language INterface in JApanese), is presented. Efficient search algorithms are proposed to get high accuracy and to reduce the required computations. First, an LR parsing algorithm with context-dependent phone models is proposed. Second, scores of the same phone models in different hypotheses at the phone-level are represented by the single score of the best hypotheses. The system is tested for the task with a 113 word vocabulary, with a word perplexity of 4.1. It produces a sentence accuracy of 97.3% for the 10 open speakers' 110 sentences and the error reduction is as much as 77% compared with using context independent phone models.>

Two methods are presented for subword clustering. The first method is an agglomerative clustering algorithm. This method is completely data-driven and finds clusters without any external guidance. The second method uses decision trees for clustering. This method uses an expert-generated list of questions about contexts and recursively selects the most appropriate question to split the allophones. Preliminary results showed that when the training set has a good coverage of the allophonic variations in the test set, both method are capable of high-performance recognition. However, under vocabulary-independent conditions, the method using tree-based allophones outperformed agglomerative clustering because of its superior generalization capability.>

The authors describe a method to represent, acquire and implement acoustic-phonetic knowledge for large vocabulary word recognition. The knowledge is represented using networks of acoustic-phonetic segments, acquired from a speech database and stored as rules which are used to generate those networks. Different standard patterns of segments are used for each VCV or CVC environment. Network-matching and segment clustering are done to implement an efficient recognition procedure. Experiments of speaker-independent isolated-word recognition were conducted with 10 male speakers' utterances. The word recognition accuracy was 99.4% for 53 city names and 96.0% for 492 words of the phonetically-balanced word-set, respectively. These results show the effectiveness of the method.>

An automatic labeling technique for known speech samples is proposed to construct a fine speech data base for investigating the acoustic-phonetic characteristics of speech. An acoustically compact descriptive unit called Demiphoneme (DPH) is introduced, and a word (or sentence) is represented by a network using DPHs which cover the acoustic variation contained in the utterances of the word (or sentence). An input speech sample is segmented and labeled to the optimal DPH sequence by the following algorithm: (a) Generating possible DPH sequences from an input phoneme sequence by rules. (b) Segmentation of the sample parameter sequence. The resultant segments (called 'SEG's) are the candidates of DPH boundaries. (c) Determining the optimal correspondence between the SEG sequence and each of the DPH sequences generated in (b). (d) Deciding the minimum error DPH sequence and corresponding SEG boundaries. The feasibility of the method is confirmed by applying it to a word set containing 53 city names.