Sound morphing is a transformation that gradually blurs the distinction between the source and target sounds. For musical instrument sounds, the morph must operate across timbre dimensions to create the auditory illusion of hybrid musical instruments. The ultimate goal of sound morphing is to perform perceptually linear transitions, which requires an appropriate model to represent the sounds being morphed and an interpolation function to obtain intermediate sounds. Typically, morphing techniques directly interpolate the parameters of the sound model without considering the perceptual impact or evaluating the results. Perceptual evaluations are cumbersome and not always conclusive. In this work, we seek parameters of a sound model that favor linear variation of perceptually motivated temporal and spectral features used to guide the morph towards more perceptually linear results. The requirement of linear variation of feature values gives rise to objective evaluation criteria for sound morphing. We investigate several spectral envelope morphing techniques to determine which spectral representation renders the most linear transformation in the spectral shape feature domain. We found that interpolation of line spectral frequencies gives the most linear spectral envelope morphs. Analogously, we study temporal envelope morphing techniques and we concluded that interpolation of cepstral coefficients results in the most linear temporal envelope morph.

We present an algorithm for sound analysis and re-synthesis with local automatic adaptation of time-frequency resolution. The reconstruction formula we propose is highly efficient, and gives a good approximation of the original signal from analyses with different time-varying resolutions within complementary frequency bands: this is a typical case where perfect reconstruction cannot in general be achieved with fast algorithms, which provides an error to be minimized. We provide a theoretical upper bound for the reconstruction error of our method, and an example of automatic adaptive analysis and re-synthesis of a music sound.

The model used to represent musical instrument sounds plays a crucial role in the quality of sound transformations. Ideally, the representation should be compact and accurate, while its parameters should give flexibility to independently manipulate perceptually related features of the sounds. This work describes a source-filter model for musical instrument sounds based on the sinusoidal plus residual decomposition. The sinusoidal component is modeled as sinusoidal partial tracks (source) and a time-varying spectral envelope (filter), and the residual is represented as white noise (source) shaped by a time-varying spectral envelope (filter). This article presents estimation and representation techniques that give totally independent and intuitive control of the spectral envelope model and the frequencies of the partials to perform perceptually related sound transformations. The result of a listening test confirmed that, in general, the sounds resynthesized from the source-filter model are perceptually similar to the original recordings.

The present article investigates into the use of the LF glottal pulse model for singing synthesis and transformation. A recent estimator of the LF-glottal pulse shape parameter (rd) is used to analyze a small collection of professional singing examples and the results are discussed in the context of recent findings relating the rd shape parameter to other speech signal parameters (intensity and vibrato). We propose a rd shape parameter model for vibrato rendering and present an algorithm that allows modifying the glottal pulse shape parameter of a given speech signal and is used to enhance the vibrato generation in a speech to singing transformation system.

The goal of sound morphing by feature interpolation is to obtain sounds whose values of features are intermediate between those of the source and target sounds. In order to do this, we should be able to resynthesize sounds that present a set of predefined feature values, a notoriously difficult problem. In this work, we present morphing techniques to obtain hybrid musical instrument sounds whose feature values correspond as close as possible to the ideal interpolated values. When the features capture perceptually relevant information, the morphed sound whose features are interpolated is perceptually intermediate. The features we use are acoustic correlates of salient timbre dimensions derived from perceptual studies, such that sounds whose feature values are intermediate between two would be placed between them in the underlying timbre space. We measure the perceptual impact of the morphed sounds directly by the feature values, using them as an objective measure with which to evaluate the results. Thus we consider that the morphed sounds change perceptually linearly when the corresponding feature values vary linearly.

The amplitude modulations of musical instrument sounds and speech are important perceptual cues. Accurate estimation of the amplitude, or equivalently energy, envelope of a time-domain signal (waveform) is not a trivial task, though. Ideally, the amplitude envelope should outline the waveform connecting the main peaks and avoiding over fitting. In this work we propose a method to obtain a smooth function that approximately matches the main peaks of the waveform using true envelope estimation, dubbed true amplitude envelope. True envelope is a cepstral smoothing technique that has been shown to outperform traditional envelope estimation techniques both in accuracy of estimation and ease of order selection. True amplitude envelope gives a reliable estimation that follows closely sudden variations in amplitude and avoids ripples in more stable regions with near optimal order selection depending on the fundamental frequency of the signal.

In voice analysis, the parameters estimation of a glottal model, an analytic description of the deterministic component of the glottal source, is a challenging question to assess voice quality in clinical use or to model voice production for speech transformation and synthesis using a priori constraints. In this paper, we first describe the Function of Phase-Distortion (FPD) which allows to characterize the shape of the periodic pulses of the glottal source independently of other features of the glottal source. Then, using the FPD, we de scribe two methods to estimate a shape parameter of the Liljencrants-Fant glottal model. By comparison with state of the art methods using Electro-Giotto-Graphic signals, we show that the one of these method outperform the compared methods.

The transformation of the voiced segments of a speech recording has many applications such as expressivity synthesis or voice conversion. This paper addresses the pitch transposition and the modification of breathiness by means of an analytic description of the deterministic component of the voice source, a glottal model. Whereas this model is dedicated to voice production, most of the current methods can be applied to any pseudo-periodic signals. Using the described method, the synthesized voice is thus expected to better preserve some naturalness compared to a more generic method. Using preference tests, it is shown that this method is preferred for important pitch transposition (e.g. one octave) compared to two state of the art methods. Additionally, it is shown that the breathiness of two male utterances can be controlled.

Spectral voice conversion is usually performed using a single model selected in order to represent a tradeoff between goodness of fit and complexity. Recently, we proposed a new method for spectral voice conversion, called Dynamic Model Selection (DMS), in which we assumed that the model topology may change over time, depending on the source acoustic features. In this method a set of models with increasing complexity is considered during the conversion of a source speech signal into a target speech signal. During the conversion, the best model is dynamically selected among the models in the set, according to the acoustical features of each source frame. In this paper, we present an objective evaluation demonstrating that this new method improves the conversion by reducing the transformation error compared to methods based on an single model.

Change detection within an audio stream is an important task in several domains, such as classification and segmentation of a sound or of a music piece, as well as indexing of broadcast news or surveillance applications. In this paper we propose two novel methods for spectral change detection without any assumption about the input sound: they are both based on the evaluation of information measures applied to a time-frequency representation of the signal, and in particular to the spectrogram. The class of measures we consider, the Rényi entropies, are obtained by extending the Shannon entropy definition: a biasing of the spectrogram coefficients is realized through the dependence of such measures on a parameter, which allows refined results compared to those obtained with standard divergences. These methods provide a low computational cost and are well-suited as a support for higher level analysis, segmentation and classification algorithms.

One problem in concatenative speech synthesis is how to incorporate prosodic factors in the unit selection. Imposing a predicted prosodic target is error-prone and does not benefit from the prosodic variability of the database. In this paper, we assume that several prosodic contours exist in the database for a same symbolic entry. This variability is represented by probabilistic models of the prosodic contours and the optimal sequence of units is searched by maximizing a joint likelihood at both segmental and prosodic levels. A generalized Viterbi algorithm is used to take into account the long-term dependencies introduced by the prosodic models. This method has been implemented in a unit selection synthesizer using an expressive speech database and a subjective experiment shows an improvement of the speech naturalness compared to a conventional unit-selection method.

In this paper, a method for prosodic break modelling based on segmental-HMMs and Dempster-Shafer fusion for speech synthesis is presented, and the relative importance of linguistic and metric constraints in prosodic break modelling is assessed 1 . A context-dependent segmental-HMM is used to explicitly model the linguistic and the metric constraints. Dempster-Shafer fusion is used to balance the relative importance of the linguistic and the metric constraints into the segmental-HMM. A linguistic processing chain based on surface and deep syntactic parsing is additionally used to extract linguistic informations of different nature. An objective evaluation proved evidence that the optimal combination of the linguistic and the metric constraints significantly outperforms both the conventional HMM (linguistic information only) and segmental-HMM (equal balance of linguistic and metric constraints), and confirmed that the linguistic constraint is prior to the metric. Index Terms: speech prosody, prosodic break, segmentalHMM, Dempster-Shafer fusion.

This paper assesses the ability of a HMM-based speech synthesis systems to model the speech characteristics of various speaking styles 1 .A discrete/continuous HMM is presented to model the symbolic and acoustic speech characteristics of a speaking style.The proposed model is used to model the average characteristics of a speaking style that is shared among various speakers, depending on specific situations of speech communication.The evaluation consists of an identification experiment of 4 speaking styles based on delexicalized speech, and compared to a similar experiment on natural speech.The comparison is discussed and reveals that discrete/continuous HMM consistently models the speech characteristics of a speaking style.

In this paper, a unified trajectory model based on the stylization and the modelling of f0 variations simultaneously over various temporal domains is proposed 1 . The syllable is used as the minimal temporal domain for the description of speech prosody, and short-term and long-termf0 variations are stylized and modelled simultaneously over various temporal domains. During the training, a context-dependent model is estimated according to the joint stylized f0 contours over the syllable and a set of long-term temporal domains. During the synthesis, f0 variations are determined using the long-term variations as trajectory constraints. In a subjective evaluation in speech synthesis, the stylization and trajectory modelling of short and long term speech prosody variations is shown to consistently model speech prosody and to outperform the conventional short-term modelling.

Intonation is one of the most important factors of speech expressivity. This paper presents a conversion method for the F0 contours. The F0 segments are represented with discrete cosine transform (DCT) coefficients at the syllable level. Multi-level dynamic features are added to model the temporal correlation between syllables and to constrain the F0 contour at the phrase level. Gaussian mixture models (GMM) are used to map the prosodic features between neutral and expressive speech, and the converted F0 contour is generated under the dynamic features constraints. Experimental evaluation using a database of acted emotional speech shows the effectiveness of the proposed F0 model and conversion method.

In glottal source analysis, the phase minimization criterion has already been proposed to detect excitation instants. As shown in this paper, this criterion can also be used to estimate the shape parameter of a glottal model (ex. Liljencrants-Fant model) and not only its time position. Additionally, we show that the shape parameter can be estimated independently of the glottal model position. The reliability of the proposed methods is evaluated with synthetic signals and compared to that of the IAIF and minimum/maximum-phase decomposition methods. The results of the methods are evaluated according to the influence of the fundamental frequency and noise. The estimation of a glottal model is useful for the separation of the glottal source and the vocal-tract filter and therefore can be applied in voice transformation, synthesis, and also in clinical context or for the study of the voice production.

A new method is proposed to jointly estimate the shape parameter of a glottal model and its time position in a voiced segment. We show that, the idea of phase flatness (or phase minimization) used in the most robust Glottal Closure Instant detection methods can be generalized to estimate the shape of the glottal model. In this paper we evaluate the proposed method using synthetic signals. The reliability related to fundamental frequency and noise is evaluated. The estimation of the glottal source is useful for voice analysis (ex. separation of glottal source and vocal-tract filter), voice transformation and synthesis.

This paper introduces a HMM-based speech synthesis system which uses a new method for the Separation of Vocal-tract and Liljencrants-Fant model plus Noise (SVLN). The glottal source is separated into two components: a deterministic glottal waveform Liljencrants-Fant model and a modulated Gaussian noise. This glottal source is first estimated and then used in the vocal-tract estimation procedure. Then, the parameters of the source and the vocal-tract are included into HMM contextual models of phonems. SVLN is promising for voice transformation in synthesis of expressive speech since it allows an independent control of vocal-tract and glottal-source properties. The synthesis results are finally discussed and subjectively evaluated.

cote interne IRCAM: Lanchantin10b

Speech can be divided into discourse genres based on the contextual environment it occurs in (e.g.political speech, sport commentary speech, etc.).The present study investigated whether listeners can distinguish between speech from different discourse genres on the basis of acoustic prosodic cues only 1 .In a perception experiment with delexicalized speech 70 listeners with varying experience in French (native speakers, nonnative speakers, and non-speakers) were asked to identify four different types of discourse genres (church service, political, journal, and sport commentary).Results revealed a fair identification ability with a significant increase in performance with increasing experience in French.Identification confusion was used to cluster discourse genres according to their perceptual similarity.

This paper presents a study on the use of deep syntactical features to improve prosody modeling. A French linguistic processing chain based on linguistic preprocessing, morpho- syntactical labeling, and deep syntactical parsing is used in order to extract syntactical features from an input text. These features are used to define more or less high-level syntactical feature sets. Such feature sets are compared on the basis of a HMM-based prosodic structure model. High-level syntactical features are shown to significantly improve the performance of the model (up to 21% error reduction combined with 19% BIC reduction).

This paper presents a frame-based system for estimating multiple fundamental frequencies (F0s) of polyphonic music signals based on the short-time Fourier transform (STFT) representation. To estimate the number of sources along with their F0s, it is proposed to estimate the noise level beforehand and then jointly evaluate all the possible combinations among pre-selected F0 candidates. Given a set of F0 hypotheses, their hypothetical partial sequences are derived, taking into account where partial overlap may occur. A score function is used to select the plausible sets of F0 hypotheses. To infer the best combination, hypothetical sources are progressively combined and iteratively verified. A hypothetical source is considered valid if it either explains more energy than the noise, or improves significantly the envelope smoothness once the overlapping partials are treated. The proposed system has been submitted to Music Information Retrieval Evaluation eXchange (MIREX) 2007 and 2008 contests where the accuracy has been evaluated with respect to the number of sources inferred and the precision of the F0s estimated. The encouraging results demonstrate its competitive performance among the state-of-the-art methods.

In this work, accurate spectral envelope estimation is applied to voice conversion in order to achieve high-quality timbre conversion. True-envelope based estimators allow model order selection leading to an adaptation of the spectral features to the characteristics of the speaker. Optimal residual signals can also be computed following a local adaptation of the model order in terms of the F0. A new perceptual criteria is proposed to measure the impact of the spectral conversion error. The proposed envelope models show improved spectral conversion performance as well as increased converted-speech quality when compared to linear prediction.

International audience

Compilation of musical instrument sample databases requires careful elimination of badly recorded samples and validation of sample classification into correct categories. This paper introduces algorithms for automatic removal of bad instrument samples using automatic musical instrument recognition and outlier detection techniques. Best evaluation results on a methodically contaminated sound database are achieved using the introduced MCIQR method, which removes 70.1% "bad" samples with 0.9% false-alarm rate and 90.4% with 8.8% false-alarm rate.

In this paper we present the implementation of an online HMM decoding process. The algorithm derives an online version of the Viterbi algorithm on successive variable length windows, iteratively storing portions of the optimal state path. We explicit the relation between the hidden layer's topology and the applicability and performance prediction of the algorithm. We evaluate the validity and performance of the algorithm on a phone recognition task on a database of continuous speech from a native French speaker. We specifically study the latency-accuracy performance of the algorithm.

Identification of prosodic phenomena is of first importance in prosodic analysis and modeling. In this paper, we introduce a new method for automatic prosodic phenomena labelling. The authors set their approach of prosodic phenomena in the framework of prominence. The proposed method for automatic prominence labelling is based on well-known machine learning techniques in a three step procedure: (i) a feature extraction step in which we propose a framework for systematic and multi-level speech acoustic feature extraction, (ii) a feature selection step for identifying the more relevant prominence acoustic correlates, and (iii) a modelling step in which a gaussian mixture model is used for predicting prominence. This model shows robust performance on read speech (84%).

In this work we consider the problem of spectral envelope estimation using spectra with perceptually warped frequency axis. The goal of this work is the reduction of the order of the spectral envelope model which will facilitate the use of these envelopes for training of voice conversion systems. We adapt the true-envelope estimator to Mel-frequency representations and adapt a recently proposed cepstral model order selection criterion taking into account the distortion of the frequency axis. We evaluate the modified order selection procedure using a perceptual framework for the evaluation of envelope estimation errors. The experimental evaluation carried out with real speech confirms our modifications. The results demonstrate that the Mel frequency based true envelope estimator achieves superior envelope estimation with significantly reduced model order.

ISBN: 978-1-61567-378-0 Special Session: Prosody of Spontaneous Speech I, Wednesday 24th September 2008.

We present a study into all-pole spectral envelope estimation for the case of harmonic signals. We address the problem of the selection of the model order and propose to make use of the fact that the spectral envelope is sampled by means of the harmonic structure to derive a reasonable choice for an appropriate model order. The experimental investigation uses synthetic ARMA featured signals with varying fundamental frequency and differing model structure to evaluate the performance of the selected all-pole models. The experimental results confirm the relation between optimal model order and the fundamental frequency.

Dynamic Time Warping (DTW) finds the best global match or alignment between two sequences by ‘time’ warping them optimally. Since the algorithm uses the whole sequences, it can be very demanding in calculation cost and memory. In particular, this limits the size of the sequences which can be aligned. In this paper a novel algorithm is presented, Short-Time Dynamic Time Warping (STDTW), which requires much less memory because optimization is done iteratively on portions of the sequences. The very remarkable property of the algorithm is that under some weak hypothesis, it produces the same globally optimal solution as the classical DTW. As an example, STDTW is applied to Score to Audio Alignment which links events in a musical score and points on the audio performance time axis.

n this work we address the problem of all pole spectral envelope estimation for speech signals. The currently widely used all pole spectral envelope model suffers from well-known systematic errors and more severely from model order mismatch. We will propose a procedure to first establish a band limited interpolation of the observed spectrum using a recently rediscovered true envelope estimator and then using the band limited envelope to derive an all pole envelope model named TE-LPC . The band-limited envelope that is used to derive the all pole envelope model reduces the problem of the unknown all pole model order. For the experimental investigation we propose a new perceptually motivated residual spectral peak flatness measure. The experimental results demonstrate that the proposed method significantly increases the spectral flatness for the perceptually especially important low order harmonics of voiced utterances

In this paper, we analyze some methodological and practical grounds affecting a correct estimation of the instantaneous amplitude (IA) and instantaneous frequency (IF) of the partials of vocal vibrato signals. This particular class of acoustical signals can be described as a set of harmonics, or partials, whose amplitude and frequency vary with time following a particular pattern. The interest in the correct measurement of these two magnitudes stems from the fact that they are widely used to extract more information about the production mechanisms of this vocal effect, and even the singing voice. Focusing on the estimation of these magnitudes from the signal processing point of view, the estimation of the IA and the IF of a partial represents a practical example for the more general problem of the AM-FM decomposition of a bandpass signal, which has received considerable attention by the Time-Frequency community. With this in mind, this work will investigate the most appropriate AM-FM decomposition of the partials, and how it is affected by the reverberation present in recordings. A mathematical description of the effect of reverberation on the measurement will be given. It will be shown that the measurement of the IA and the IF of the partials will be corrupted by reverberation regardless of the employed estimation method

The article is concerned with the estimation of fundamental frequencies, or F0s, in polyphonic music. We propose a new method for jointly evaluating multiple F0 hypotheses based on three physical principles, harmonicity, spectral smoothness and synchronous amplitude evolution, within a single source. Based on the generative quasiharmonic model, a set of hypothetical partial sequences is derived and an optimal assignment of the observed peaks to the hypothetical sources and noise is performed. The hypothetical partial sequences are then evaluated by a score function which formulates the guiding principles in a mathematical manner. The algorithm has been tested on a large collection of artificially mixed polyphonic samples and the results show the competitive performance of the proposed method.

We investigate the relationship between the instantaneous amplitude (IA) and the instantaneous frequency (IF) of vibrato signals in the singing voice. It is shown that this relationship is of great value in obtaining information about the vocal tract model. However, to make this analysis possible, it is necessary to cope with two basic limitations: reverberation in recordings, which shows up as multiharmonicity in each partial, and the phase effect of vocal tract formants which distort the instantaneous frequency of some partials.

Modeling expressive synthesis requires one to deal with a wide range of sound's behaviors that include spectral dynamics which are idiomatic to the instrument or characteristics of the playing technique. In the paper we analyze short time evolution of power-spectral envelopes, adaptively changing the analysis parameters so as to variably include filter envelopes, and the pitched excitation parts. Typical sequences of sound features are obtained from quantized sequences of parameters. By applying string matching techniques, new sequences of features are created and new sounds that have dynamically changing temporal behavior are re-synthesized.

In order to estimate the fundamental frequency (fO) of pseudoperiodical sounds with a wide band of possible fO, a theoretical model based on a maximum likelihood for fO is proposed. The model is simplified to make it fast enough for extensive tests. The resulting algorithm is tested on musical speech sounds. As a musical application, an instrument follower based on the algorithm and operating in real time is implemented.>

We describe a new method for very high quality speech synthesis, and the procedure used for the automatic acquisition of the rules. Our method retains positive aspects of both diphone LPC synthesis and calculation of formant trajectories. For each sequence /VC#/ or /#CV/, we apply a very precise, pitch synchronous, LP analysis, we represent each spectrum in terms of the parameters for a parallel formant syntheziser, and we extract formant trajectories from these data. At the synthesis stage, for any /V1CcV2/ sequence, trajectories are calculated by rule using the stored trajectories of /V1Cc#/ and /#CcV2/. The signal is computed according to the calculated trajectories with a high quality parallel formant syntheziser. We have synthesized singing voices, but the same procedure can be applied to speech.

The rules for vocalic sound transitions usually consider a limited frequency range for each formant. Current acoustic knowledge of speech production allows us to give a different interpretation of vocalic sound transitions. This interpretation would explain the existence of the so-called "spurious" formants, the observed "asynchrony" between transitions, and the effects of coarticulation across consonants. We show as an example that the second formant of /w/ is transformed through the transition into the fourth formant of /i/ in the sequence /wi/; a result implied in K.N. STEVENS theoretical model (1). Implications for automatic speech pattern discrimination and synthesis are discussed.

The SARA-JUNIOR text-to-speech synthesis uses new developments in the techniques pioneered in the previous SARA software. Acoustic parameters (i.e. pitch period duration, formant frequencies and formant amplitudes) are calculated by rules for each pitch period, according to prosodic features and given phonetic string. The signal is calculated as a summation of formant time-domain functions derived from the previously determined acoustic parameters. These methods are implemented with a general-purpose software system dedicated to fast generation, editing and merging of streams of samples to produce complex audio signals. This system facilitates user control of critical events in time such as the onset of sound components, beginning of envelopes etc. Synthesis commands can be stated as LL(1) rewriting rules.