Generating expressive synthetic voices requires carefully designed databases that contain sufficient amount of expressive speech material. This paper investigates voice conversion and modification techniques to reduce database collection and processing efforts while maintaining acceptable quality and naturalness. In a factorial design, we study the relative contributions of voice quality and prosody as well as the amount of distortions introduced by the respective signal manipulation steps. The unit selection engine in our open source and modular text-to-speech (TTS) framework MARY is extended with voice quality transformation using either GMM-based prediction or vocal tract copy resynthesis. These algorithms are then cross-combined with various prosody copy resynthesis methods. The overall expressive speech generation process functions as a postprocessing step on TTS outputs to transform neutral synthetic speech into aggressive, cheerful, or depressed speech. Cross-combinations of voice quality and prosody transformation algorithms are compared in listening tests for perceived expressive style and quality. The results show that there is a tradeoff between identification and naturalness. Combined modeling of both voice quality and prosody leads to the best identification scores at the expense of lowest naturalness ratings. The fine detail of both voice quality and prosody, as preserved by the copy synthesis, did contribute to a better identification as compared to the approximate models.

Voice conversion enables generation of a desired speaker's voice from audio recordings of another speaker. In this paper, we focus on a music application and describe the first steps towards generating voices of music celebrities using conventional voice conversion techniques. Specifically, rap singing transformations from English to Spanish are performed using parallel training material in English. Weighted codebook mapping based voice conversion with two different alignment methods and temporal smoothing of the transformation filter are employed. The first aligner uses a HMM trained for each source recording to force-align the corresponding target recording. The second aligner employs speaker-independent HMMs trained from a large number of speakers. Additionally, a smoothing step is devised to reduce discontinuities and to improve performance. The results of subjective evaluations indicate that both aligners perform equivalently well. The proposed smoothing technique improves both similarity to target singer and quality significantly regardless of the alignment method.

This paper presents a comparison of methods for transforming voice quality in neutral synthetic speech to match cheerful, aggressive, and depressed expressive styles. Neutral speech is generated using the unit selection system in the MARY TTS platform and a large neutral database in German. The output is modified using voice conversion techniques to match the target expressive styles, the focus being on spectral envelope conversion for transforming the overall voice quality. Various improvements over the state-of-the-art weighted codebook mapping and GMM based voice conversion frameworks are employed resulting in three algorithms. Objective evaluation results show that all three methods result in comparable reduction in objective distance to target expressive TTS outputs whereas weighted frame mapping and GMM based transformations were perceived slightly better than the weighted codebook mapping outputs in generating the target expressive style in a listening test.

Synthesizing desired emotions using concatenative algorithms relies on collection of large databases. This paper focuses on the development and assessment of a simple algorithm to interpolate the intended vocal effort in existing databases in order to create new databases with intermediate levels of vocal effort. Three diphone databases in German with soft, modal, and loud voice qualities are processed with a spectral interpolation algorithm. A listening test is performed to evaluate the intended vocal effort in the original databases as well as the interpolated ones. The results show that the interpolation algorithm can create the intended intermediate levels of vocal effort given the original databases independent of the language background of the subjects. 1.

This study and transformation of the vocal tract spectrum and the pitch contour. The first method (selective pre-emphasis) relies on band-pass filtering to perform vocal tract transformation. The second method (segmental pitch contour model) focuses on a more detailed modeling of pitch contours. Both methods are utilized in the design of a voice conversion algorithm based on codebook mapping. We compare them with existing vocal tract and pitch contour transformation methods and acoustic feature transplantations in subjective tests. The performance of the selective pre-emphasis based method is similar to the methods used in our previous work at higher sampling rates with a lower prediction order. The results also indicate that the segmental pitch contour model improves voice conversion performance. 1 proposes two new methods for detailed modeling 1.

Perception of speaker identity is an important characteristic of the human auditory system. This paper describes a subjective test for the investigation of the relevance of four acoustic features in this process: vocal tract, pitch, duration, and energy. PSOLA based methods provide the framework for the transplantations of these acoustic features between two speakers. The test database consists of different combinations of transplantation outputs obtained from a database of 8 speakers. Subjective decisions on speaker similarity indicate that the vocal tract is the most relevant feature for single feature transplantations. Pitch and duration possess similar significance whereas the energy is the least important acoustic feature. Vocal tract + pitch + duration transplantation results in the highest similarity to the target speaker. Vocal tract + pitch, vocal tract + duration + energy and vocal tract + duration transplantations also yield convincing results in transformation of the perceived speaker identity.

In this study, we describe and evaluate a subjective test to determine the importance of subbands in perceiving the speaker identity. Pairs of sound files have been generated where the first sound file is the full-band version of a word and the second file is a subband of the same word in the first part and a subband of a different word in the second part of the experimental work. Words in each pair are uttered by either the same speaker or different speakers. S ubjects were asked to score each pair of sound files they listened to according to the similarity of the speakers uttering each word. Also a consistency score is calculated for each subject to render the statistics more reliable. The experiment demonstrates that the second subband (1034 Hz-1895 Hz range) is the most important subband for speaker identification.

A new voice conversion method that improves the quality of the voice conversion output at higher sampling rates is proposed. Speaker Transformation Algorithm Using Segmental Codebooks (STASC) is modified to process source and target speech spectra in different subbands. The new method ensures better conversion at sampling rates above 16KHz. Discrete Wavelet Transform (DWT) is employed for subband decomposition to estimate the speech spectrum better with higher resolution. Faster voice conversion is achieved since the computational complexity decreases at a lower sampling rate. A Voice Conversion System (VCS) is implemented using the proposed algorithm with necessary tools. The performance of the proposed method is demonstrated by both subjective listening tests and applications to film dubbing and looping. In ABX listening tests, the listeners preferred the subband based output by 92.1% as compared to the full-band based output.

A classification system that aims to recognize the presence of sounds from different sources is described. The type of audio signals considered are speech, music, noise and silence. Appropriate subband processing is applied for the characterization of each sound source. The algorithm operates in four steps to classify the contents of a given audio signal. The acoustical parameters and statistical measures to be used in the classification process are obtained via an off-line training procedure. In the silence and onset detection stages, we aim to label the starting and finishing instants of the acoustical events present in the audio signal. Acoustical parameters of the given signal are extracted, analysis of variance and classification using the LBG algorithm is carried out by generating codebooks of acoustical vectors. Experimental work is carded out on a database that contains mixtures of human speech, music, noise and silence. The experiments demonstrate that the system achieves 88% classification success on the average when sounds from different sources are presented non-simultaneously.