Voice production models provide valuable information about the human voice generation. However, providing them with expressiveness remains a challenge. This work proposes a methodology to modify vocal effort (VE) in the numerical simulation of vowels using a glottal source Liljencrants-Fant (LF) model and a one-dimensional acoustic model based on the finite element method. Vowels recorded with high, mid, and low VE are inverse-filtered to obtain a glottal source signal, used to estimate the LF model Rd parameter. A tuning algorithm adjusts the vocal tract geometry to match the formants of the analysed vowel. Preliminary results for the vowel /a/ are presented. Objective analyses indicate the relevance of both glottal source and vocal tract changes in reproducing VE. They are also perceptually relevant for low VE, while the glottal source predominates in high VE. Perceptual assessment validates the methodology can convey different levels of VE, particularly low and medium.

A significant aging of world population is foreseen for the next decades. Thus, developing technologies to empower the independency and assist the elderly are becoming of great interest. In this framework, the IMMED project investigates tele-monitoring technologies to support doctors in the diagnostic and follow-up of dementia illnesses such as Alzheimer. Specifically, water sounds are very useful to track and identify abnormal behaviors form everyday activities (e.g. hygiene, household, cooking, etc.). In this work, we propose a double-stage system to detect this type of sound events. In the first stage, the audio stream is segmented with a simple but effective algorithm based on the Spectral Cover feature. The second stage improves the system precision by classifying the segmented streams into water/non-water sound events using Gammatone Cepstral Coefficients and Support Vector Machines. Experimental results reveal the potential of the combined system, yielding a F-measure higher than 80%.

Current research to improve state of the art Text-To-Speech (TTS) synthesis studies both the processing of input text and the ability to render natural expressive speech. Focusing on the former as a front-end task in the production of synthetic speech, this article investigates the proper adaptation of a Sentiment Analysis procedure (positive/neutral/negative) that can then be used as an input feature for expressive speech synthesis. To this end, we evaluate different combinations of textual features and classifiers to determine the most appropriate adaptation procedure. The effectiveness of this scheme for Sentiment Analysis is evaluated using the Semeval 2007 dataset and a Twitter corpus, for their affective nature and their granularity at the sentence level, which is appropriate for an expressive TTS scenario. The experiments conducted validate the proposed procedure with respect to the state of the art for Sentiment Analysis.

In the context of non-speech audio recognition and classification for multimedia applications, it becomes essential to have a set of features able to accurately represent and discriminate among audio signals. Mel frequency cepstral coefficients (MFCC) have become a de facto standard for audio parameterization. Taking as a basis the MFCC computation scheme, the Gammatone cepstral coefficients (GTCCs) are a biologically inspired modification employing Gammatone filters with equivalent rectangular bandwidth bands. In this letter, the GTCCs, which have been previously employed in the field of speech research, are adapted for non-speech audio classification purposes. Their performance is evaluated on two audio corpora of 4 h each (general sounds and audio scenes), following two cross-validation schemes and four machine learning methods. According to the results, classification accuracies are significantly higher when employing GTCC rather than other state-of-the-art audio features. As a detailed analysis shows, with a similar computational cost, the GTCC are more effective than MFCC in representing the spectral characteristics of non-speech audio signals, especially at low frequencies.

Text-to-Speech (TTS) synthesis systems produce speech from an input text. Corpus based or unit selection TTS (US-TTS) are based on retrieving the best set of speech units from a large labelled speech database. To that effect, the unit selection is guided by dynamic programming and a weighted cost function. Several weight tuning approaches have been defined so as to integrate human preferences in the unit selection process, but with no great success beyond expert-based hand tuning. However, active interactive genetic algorithms (aiGAs) have showed promising results working on the unit selection text-to-speech (US-TTS) weight tuning problem in previous works. aiGAs are an evolution of classic interactive genetic algorithms (IGAs) in terms of reducing fatigue, ambiguity and frustration in user's evaluations. This paper presents a step further in the application of aiGAs to this problem by defining new indicators of the perceptually-based evolutionary process to obtain more reliable weights. The experiments have been conducted on one hour Spanish speech database and using an acoustic plus linguistic cost function.

The affective communication channel plays a key role in multimodal human-computer interaction. In this context, the generation of realistic talking-heads expressing emotions both in appearance and speech is of great interest. The synthetic speech of talking-heads is generally obtained from a text-to-speech (TTS) synthesizer. One of the dominant techniques for achieving high-quality synthetic speech is unit-selection TTS (US-TTS) synthesis. Affective US-TTS systems are driven by affective annotated speech databases. Since affective speech involves higher acoustic variability than neutral speech, achieving trustworthy speech labeling is a more challenging task. To that effect, this paper introduces a methodology for achieving reliable pitch marking on affective speech. The proposal adjusts the pitch marks at the signal peaks or valleys after applying a three-stage restricted dynamic programming algorithm. The methodology can be applied as a post-processing of any pitch determination and pitch marking algorithm (with any local criterion for locating pitch marks), or their merging. The experiments show that the proposed methodology significantly improves the results of the input state-of-the-art markers on affective speech.

This paper presents a text classifier for automatically taggingthe sentiment of input text according to the emotion that is beingconveyed. This system has a pipelined framework composedof Natural Language Processing modules for feature extractionand a hard binary classifier for decision making between posi-tive and negative categories. To do so, the Semeval 2007 datasetcomposed of sentences emotionally annotated is used for train-ing purposes after being mapped into a model of affect. Theresulting scheme stands a first step towards a complete emotionclassifier for a future automatic expressive text-to-speech syn-thesizer.

This paper is a contribution to the recent advancements in the development of high-quality next generation text-to-speech (TTS) synthesis systems. Two of the hottest research topics in this area are oriented towards the improvement of speech expressiveness and flexibility of synthesis. In this context, this paper presents a new TTS strategy called multidomain TTS (MD-TTS) for synthesizing among different domains. Although the multidomain philosophy has been widely applied in spoken language systems, few research efforts have been conducted to extend it to the TTS field. To do so, several proposals are described in this paper. First, a text classifier (TC) is included in the classic TTS architecture in order to automatically conduct the selection of the most appropriate domain for synthesizing the input text. In contrast to classic topic text classification tasks, the MD-TTS TC should not only consider the contents of text but also its structure. To this end, this paper introduces a new text modeling scheme based on an associative relational network, which represents texts as a directional weighted word-based graph. The conducted experiments validate the proposal in terms of both objective (TC efficiency) and subjective (perceived synthetic speech quality) evaluation criteria.

This paper presents the use of analogical learning, in particular case-based reasoning, for the automatic generation of prosody from text, which is automatically tagged with prosodic features. This is a corpus-based method for quantitative modelling of prosody to be used in a Spanish text to speech system. The main objective is the development of a method for predicting the three main prosodic parameters: the fundamental frequency (F0) contour, the segmental duration and energy. Both objective and subjective experiments have been conducted in order to evaluate the accuracy of our proposal.

Consensus clustering is the task of deriving a single labeling by applying a consensus function on a cluster ensemble. This work introduces BordaConsensus, a new consensus function for soft cluster ensembles based on the Borda voting scheme. In contrast to classic, hard consensus functions that operate on labelings, our proposal considers cluster membership information, thus being able to tackle multiclass clustering problems. Initial small scale experiments reveal that, compared to state-of-the-art consensus functions, BordaConsensus constitutes a good performance vs. complexity trade-off.

This paper builds introduces visual-analytic techniques to aggregate, summarize, and visualize the information generated during interactive evolutionary processes. Special visualizations of the user-provided partial ordering of solutions, the synthetic fitness surrogates induced, and the model of user preferences were prepared. The proposed visual-analytic techniques point out potential pitfalls, strengths, and possible improvements in a non-trivial case study where the hierarchical tournament selection scheme of an active interactive genetic algorithm is replaced by an incremental selection scheme. Visual analytics provided an intuitive reasoning environment that unveiled important properties that greatly affect the performance of active interactive genetic algorithms that could not have been easily reveled otherwise.

The quality of corpus-based text-to-speech systems depends on the accuracy of the unit selection process, which in turn relies on the cost function definition. This function should map the user perceptual preference when selecting synthesis units, which is a very difficult task. This paper continues our previous work on fusing the human judgements with the cost function by means of interactive weight tuning. The application of active interactive genetics algorithms mitigates user fatigue by improving user consistency. As a result, the obtained weights generate more natural synthetic speech when compared to previous objective and subjective proposals

In this paper, a generic pitch marks filtering algorithm (PMFA) is introduced in order to achieve reliable and smooth pitch marks\nfrom any input pitch tracking or marking algorithm. The proposed PMFA is a simple yet effective filtering process based on restricted dynamic programming, but very helpful for minimizing human intervention when creating large speech corpora. Moreover, this work introduces a novel pitch marking evaluation measure for directly comparing pitch marking algorithms with different location criteria. The experiments demonstrate that the proposed PFMA improves the results of the input state-of-the-art pitch tracking and marking algorithms dramatically.

This paper describes a multi-domain text-to-speech (MD-TTS)synthesis strategy for generating speech among different domainsand so increasing the flexibility of high quality TTS systems. Tothat effect, the MD-TTS introduces a flexible TTS architecture thatincludes an automatic domain classification module, which allowsMD-TTS systems to be implemented by different synthesis strate-gies and speech corpus typologies. In this work, the performanceof a corpus-based MD-TTS system is subjectively validated bymeans of several perceptual tests

The performance of document clustering systems depends on employing optimal text representations, which are not only difficult to determine beforehand, but also may vary from one clustering problem to another. As a first step towards building robust document clusterers, a strategy based on feature diversity and cluster ensembles is presented in this work. Experiments conducted on a binary clustering problem show that our method is robust to near-optimal model order selection and able to detect constructive interactions between different document representations in the test bed.

A restricted domain text-to-speech system oriented to a weatherforecast application is presented. This TTS system is embed-ded in a multimedia interactive service accessible from differ-ent media, such as TV, Internet and mobile devices. The re-quirements of this application give rise to several particularitiesin the design and implementation of the TTS system, which arediscussed throughout this paper. Several tests have been con-ducted to analyze the TTS system in terms of performance andspeech quality.

In the framework of multi-domain text-to-speech synthesis, it is essential (i) to design a hierarchically structured database for allowing several domains in the same speech corpus and (ii) to include a text classification module that, at run time, assigns the input sentences to a domain or set of domains from the database. We present a hierarchical text classifier based on independent component analysis (ICA), which is capable of (i) organizing the contents of the corpus in a hierarchical manner and (ii) classifying the texts to be synthesized according to the learned structure. The document organization and classification performance of our ICA-based hierarchical classifier are evaluated in several encouraging experiments conducted on a journalistic-style text corpus for speech synthesis in Catalan.

The quality of corpus based text-to-speech systems depends on the accuracy of the unit selection process, which relies on the values of the weights of the cost function. This paper is focused on defining a new framework for the tuning of these weights. We propose a technique for taking into account the subjective perception of speech in the selection process by means of Interactive Genetic Algorithms. Moreover, we introduce a CARTbased method for unit clustering. Both techniques are applied to weight tuning based on diphone pairs. The conducted experiments analyze the feasibility of both proposals separately.

This paper presents a new method named text to visual synthesis with appearance models (TEVISAM) for generating videorealistic talking heads. In a first step, the system learns a person-specific facial appearance model (PSFAM) automatically. PSFAM allows modeling all facial components (e.g. eyes, mouth, etc) independently and it will be used to animate the face from the input text dynamically. As reported by other researches, one of the key aspects in visual synthesis is the coarticulation effect. To solve such a problem, we introduce a new interpolation method in the high dimensional space of appearance allowing to create photorealistic and videorealistic avatars. In this work, preliminary experiments synthesizing virtual avatars from text are reported. Summarizing, in this paper we introduce three novelties: first, we make use of color PSFAM to animate virtual avatars; second, we introduce a nonlinear high dimensional interpolation to achieve videorealistic animations; finally, this method allows to generate new expressions modeling the different facial elements.

Unit selection text-to-speech (TTS) conversion is an ongoing research issue for the speech synthesis community. This paper is focused on tuning the weights involved in the target and the concatenations cost metrics. We propose a method for automatically tuning these weights simultaneously by means of diphone pairs. This method is based on techniques provided by the evolutionary computation community, taking advantage of their robustness in noisy domains. The experiments and their analysis show its good performance in this problem, overcoming some constrains assumed by previous works and being an interesting framework for further investigations.

In this paper we present a speech synthesis method for diphonebased text-to-speech systems. Its main goal is to achieve\nprosodic modifications that result in more natural-sounding synthetic speech. This improvement is especially useful for emotional speech synthesis, which requires high-quality prosodic modification. We present a hybrid method based on TD-PSOLA and the harmonic plus noise model, which incorporates a novel method to jointly modify pitch and time-scale. Preliminary results show an improvement in the synthetic speech quality when high pitch modification is required.

This paper describes a 2D realistic talking face. The facial appearance model is constructed with a parameterised 2D sample based model. This representation supports moderated head movements, facial gestures and emotional expressions. Two main contributions for talking heads applications are proposed. First, the image of the lips is synthesized by means of shape and texture information. Secondly, a nearly automated training process makes the talking face personalization easier, due to the use of mouth tracking. Additionally, lips are synchronized in real time with speech that is generated using a SAPI compliant text-to-speech engine.