Recently, end-to-end text-to-speech (TTS) models have achieved a remarkable performance, however, requiring a large amount of paired text and speech data for training. On the other hand, we can easily collect unpaired dozen minutes of speech recordings for a target speaker without corresponding text data. To make use of such accessible data, the proposed method leverages the recent great success of state-of-the-art end-to-end automatic speech recognition (ASR) systems and obtains corresponding transcriptions from pretrained ASR models. Although these models could only provide text output instead of intermediate linguistic features like phonemes, end-to-end TTS can be well trained with such raw text data directly. Thus, the proposed method can greatly simplify a speaker adaptation pipeline by consistently employing end-to-end ASR/TTS ecosystems. The experimental results show that our proposed method achieved comparable performance to a paired data adaptation method in terms of subjective speaker similarity and objective cepstral distance measures.

In non-stationary environments, data might change over time, leading to variations in the underlying data distributions. This phenomenon is called concept drift and it negatively impacts the performance of scene detection models due to them being trained and evaluated on data with different distributions. This paper presents a new algorithm for detecting and adapting to concept drifts based on combining the existing and new components Gaussian mixture model then merging it. The algorithm is equipped with a drift detector based on kernel density estimation enabling the algorithm to adapt to new data and generalize over old and new concepts well.

In this paper, we propose an algorithm to improve the naturalness of the reconstructed glossectomy patient's speech that is generated by voice conversion to enhance the intelligibility of speech uttered by patients with a wide glossectomy. While existing VC algorithms make it possible to improve intelligibility and naturalness, the result is still not satisfying. To solve the continuing problems, we propose to directly modify the speech waveforms using a spectrum differential. The motivation is that glossectomy patients mainly have problems in their vocal tract, not in their vocal cords. The proposed algorithm requires no source parameter extractions for speech synthesis, so there are no errors in source parameter extractions and we are able to make the best use of the original source characteristics. In terms of spectrum conversion, we evaluate with both GMM and DNN. Subjective evaluations show that our algorithm can synthesize more natural speech than the vocoder-based method. Judging from observations of the spectrogram, power in high-frequency bands of fricatives and stops is reconstructed to be similar to that of natural speech.

In this paper, we propose a system to collect human behavior in detail with higher-level tags such as attitude, accompanying-person, expenditure, tasks and so on. The system makes it possible to construct database to model higher-level human behaviors that are utilized in context-aware services. The system has two input methods, i.e., on-the-fly by a smart phone and post-processing by a PC browser. On the PC, users can interactively know where and when they were, which results in easily and accurately constructing human behavior database. To decrease numbers of operations, users' repetitions and similarity of among users are used. According to experiment results, the system successfully decreases operation time, higher-level tags are properly collected and behavior tendencies of the users are clearly captured.

As it is becoming more common for various handheld devices to be equipped with a camera, applications that use image recognition are increasing. Therefore, improving the usability of these devices by enhancing their image capture characteristics is becoming more important. We present ClearPlate - a new optical unit for image capture. It has a physically transparent viewfinder through which the user can observe the target directly without offset; the framed image is captured by the built-in camera. We also present a user study that evaluates ClearPlate performance in the 2D barcode acquisition task. Results show that ClearPlate significantly outperforms the camera-phone-based approach with regard to 2D barcode acquisition.

Television sets and video game consoles equipped with a web browser have appeared, and we are now able to browse web pages on television screens. However, existing navigation techniques are too difficult in this situation. In this paper, we propose Arrow Tag, a new link selection technique for web browsers on TV. In this technique, sequences of arrow signs called Arrow Tags are assigned to the links of the web pages, so users can select the links by pushing the four direction keys a few times, while keeping her/his gaze fixed on the television screen. User studies show that Arrow Tag significantly outperforms the conventional techniques of Focus Move and Number Tag. Moreover, most participants preferred Arrow Tag over either Focus Move or Number Tag.

This paper proposes a new text-to-speech (TTS) system that utilizes large numbers of speech segments to produce very natural and intelligible synthetic speech. There are two innovations; new multiform synthesis units and a new speech modification algorithm based on a vocoder that offers harmonics reconstruction. The multiform units make it possible to reduce acoustic discontinuities at concatenation points and unnatural sound by preparing synthesis units with various lengths and various F/sub 0/ contours. The new speech modification algorithm, on the other hand, improves the quality of prosody modified speech. This algorithm is extremely effective in synthesizing speech whose prosodic parameters are quite different from those of synthesis units. Listening tests confirm that the new synthesis units yield speech with high intelligibility and naturalness, and that the new speech modification algorithm is superior to all other conventional vocoders and waveform domain algorithms including TD-PSOLA, especially when modifying the F/sub 0/ frequency upward.

Automatic synthesis of speech from arbitrary text requires two basic operations: linguistic analysis of input text and speech waveform generation. The achieved quality of the second stage very much depends on the reliability and richness of information generated in the first stage. In this paper we discuss possibilities and problems of text analysis for multilingual speech synthesis. The language independent approach requires the separation of all the language specific information into the language specific inventory, which is composed of different lexica, various dictionaries and lists. The remaining core represents the language independent text-processing engine.

In this paper, we first introduce the use of Gaussian mixture models (GMM) for Chinese tone classification in continuous speech. Then, we explain how to integrate it with the HMM-based speech recognition system. Finally, we provide the tone classification accuracy of this probabilistic method which is tested with Chinese continuous speech database of national “863” project.

To facilitate multimedia content production for Internet distribution, this paper proposes a new framework for generating multimedia content. Advantages of the framework are minimized amount of transmitted data and easy-to-update multimedia content. Key points are (1) to produce speech messages by a text-to-speech (TTS) system and (2) to produce moving pictures by concatenating templates from a set of moving pictures. To support the proposed framework, a speech design tool and a movie assign tool are introduced. Three kinds of contents have been produced using the proposed framework and informally evaluated. Important results are: (1) the transmitted data of WebMessenger is 120 times smaller than the full movie version of the content, (2) quality of speech synthesized by TTS is not sufficient and a speech design tool is necessary, (3) WebMessenger is 50% cheaper than human speech recording, (4) speech synthesis by combining manual and automatic methods is effective.

This paper proposes a new speech modification algorithm which makes it possible to change the fundamental frequency (F/sub 0/) while preserving high quality. One novel point of the algorithm is that the spectrum envelope is transformed according to amount of F/sub 0/ modification. Based on a codebook mapping formulation, transformation rules are generated using speech data uttered in a different F/sub 0/ range. The rules have two purposes: one is transforming the spectrum envelope of the low frequency band and the other is adjusting the balance between low band power and high band power. The proposed algorithm is applied to a text-to-speech system based on waveform concatenation, and good performance is confirmed by listening tests.

The paper proposes a new algorithm to convert the speech of one speaker so that it sounds like that of another. To modify formant frequencies and spectral tilt, the algorithm provides piecewise linear conversion rules which make it possible to control spectrum parameters in detail. Because the rules are statistically generated using training data, reliability of the conversion rules are guaranteed, and the rules are automatically generated for any pair of speakers. According to the conversion rules, formant frequencies and spectral tilt are modified by a formant modification algorithm based on the minimization of spectral distortion. Evaluation by listening tests shows that the proposed algorithm makes it possible to convert speaker individuality while maintaining high quality.>

Syllable-based F/sub 0/ units (SBUs) are proposed for F/sub 0/ contour generation, along with a two-stage strategy. The two-stage strategy provides a flexible F/sub 0/ generation framework by introducing a global model and local model. The local model consists of the SBUs which make it possible to estimate the F/sub 0/ contour precisely using segmental information. Experimental results show that the proposed approach can generate a good global model (the measured multiple correlation coefficient is 0.843) and can precisely estimate average F/sub 0/ (the measured multiple correlation coefficient is 0.875). It is also confirmed that generating SBUs according to syllable positions is important in precisely estimating the F/sub 0/ contour. Listening tests show that speech synthesized with the proposed model is preferred to the output of the conventional model. It is expected that the approach will prove to be useful and powerful for synthesizing various types of speech.>

A voice conversion algorithm that uses speech segments as conversion units is proposed. Input speech is decomposed into speech segments by a speech recognition module, and the segments are replaced by speech segments uttered by another speaker. This algorithm makes it possible to convert not only the static characteristics but also the dynamic characteristics of speaker individuality. The proposed voice conversion algorithm was used with two male speakers. Spectrum distortion between target speech and the converted speech was reduced to one-third the natural spectrum distortion between the two speakers. A listening experiment showed that, in terms of speaker identification accuracy, the speech converted by segment-sized units gave a score 20% higher than the speech converted frame-by-frame.>

First, the part of spectral difference that is due to the difference in language is assessed. This is investigated using a bilingual speaker's speech data. It is found that the interlanguage (between English and Japanese) difference is smaller than the interspeaker difference. Listening tests indicate that the difference between English and Japanese is very small. Second, a model for cross-language voice conversion is described. In this approach, voice conversion is considered a mapping problem between two speakers' spectrum spaces. The spectrum spaces are represented by codebooks. From this point of view, a cross-language voice conversion model and measures for the model are proposed. The converted speech from male to female is as understandable as the unconverted speech and, moreover, it is recognized as female speech.>

The authors propose a new speech modification algorithm using short-time Fourier transform (STFT) synthesis. This algorithm is developed using the criterion that the mean-square-error signals between the STFT spectra of the estimated and the modified should be minimized. The most important and unique ideas of the algorithm are liftering that passes all cepstra except cepstra in the pitch frequency region, and phase control by a signal reconstruction algorithm and window-shift adjustment. Listening tests for uniform and nonuniform pitch modification reveal that the proposed algorithm can synthesize high-quality speech and that it is applicable to a synthesis-by-rule system.>

The authors propose a new voice conversion technique through vector quantization and spectrum mapping. The basic idea of this technique is to make mapping codebooks which represent the correspondence between different speakers' codebooks. The mapping codebooks for spectrum parameters, power values and pitch frequencies are separately generated using training utterances. This technique makes it possible to precisely control voice individuality. To evaluate the performance of this technique, hearing tests are carried out on two kinds of voice conversions. One is a conversion between male and female speakers, the other is a conversion between male speakers. In the male-to-female conversion experiment, all converted utterances are judged as female, and in the male-to-male conversion, 65% of them are identified as the target speaker.>