We propose a system that is capable of improving intelligibility of speech by those with hearing impairment. We have found the speech often had problems in the intonation, the duration of the unvoiced consonants, and the tone of the voiced phonemes. The system systematically compensates problematic components of the speech using the counterparts in normal speech. It corrects the intonation using TD-PSOLA and elongates consonants by repeating the original waveform using inverting technique for making the result continuously connected until the duration reaches a threshold. Experimental results show that the proposed method successfully improves the intelligibility of the speech from 28% to 35% by making phonemes more articulated, and making double consonants, ‘n’ of syllabic consonant, and unvoiced consonants perceived clearer.

This paper proposes a method of automated individualization of eye region model. The eye region model has been proposed in past research that parameterizes both the structure and the motion of the eye region. Without any prior knowledge, one can never determine a given appearance of eye region to be either neutral to any expression, i.e., the inherent structure of the eye region, or the result of motion by a facial expression. The past method manually individualized the model with respect to the structure parameters in the initial frame and tracks the motion parameters automatically across the rest of the image sequence, assuming the initial frame contains only neutral faces. Under the same assumption, we automatically determine the structure parameters for the given eye region image. We train active appearance models (AAMs) for parameterizing the variance of individuality. The system projects a given eye region image onto the low dimensional subspace spanned by the AAM and retrieves the structure parameters of the nearest training sample and initializes the eye region model using them. The AAMs are trained in the subregions, i.e., the upper eyelid region, the palpebral fissure (the eye aperture) region, and the lower eyelid region, respectively. It enables each AAM to effectively represent fine structures. Experimental results show the proposed method gives as nice initialization as manual labor and allows comparative tracking results for a comprehensive set of eye motions.

We propose a method for synthesizing a "speaker mixture voice" that has both of two speakers' individualities. We define the "speaker mixture voice" as such that 50 percent of the subjects who listen to the voice would identify either speaker A or speaker B in ABX listening test that instructs them to identify the speaker. To synthesize the speaker mixture voice, we parameterize the spectrum envelope with respect to peaks and valleys, find the correspondence between two spectrum envelopes from independent speakers using DP matching, morph one to the other, and generate waveforms using TD-PSOLA. Listening experiments showed 60 percent out of 56 synthesized voices were recognized as the speaker mixture voices. The primary application of the proposed method would be individualization of the characters in online games where multiple users play a single character as separate individuals at the same time.

An efficient speech synthesis method that uses subspace constraint in prosody is proposed. Conventional unit selection methods concatenate speech segments stored in database, that require enormous number of waveforms in synthesizing various emotional expressions with arbitrary texts. The proposed method employs principal component analysis to reduce the dimensionality of prosodic components, that also allows us to generate new speech that are similar to training samples. The subspace constraint assures that the prosody of the synthesized speech including F0, power, and speech length hold their correlative relation that training samples of emotional speech have. We assume that the combination of the number of syllables and the accent type determines the correlative dynamics of prosody, for each of which we individually construct the subspace. The subspace is then linearly related to emotions by multiple regression analysis that are obtained by subjective evaluation for the training samples. Experimental results demonstrated that only 4 dimensions were sufficient for representing the prosodic changes due to emotion at over 90% of the total variance. Synthesized emotion were successfully recognized by the listeners of the synthesized speech, especially for "anger", "surprise", "disgust", 'sorrow", "boredom", "depression", and "joy"

We propose a system that is capable of detailed analysis of eye region images in terms of the position of the iris, degree of eyelid opening, and the shape, complexity, and texture of the eyelids. The system uses a generative eye region model that parameterizes the fine structure and motion of an eye. The structure parameters represent structural individuality of the eye, including the size and color of the iris, the width, boldness, and complexity of the eyelids, the width of the bulge below the eye, and the width of the illumination reflection on the bulge. The motion parameters represent movement of the eye, including the up-down position of the upper and lower eyelids and the 2D position of the iris. The system first registers the eye model to the input in a particular frame and individualizes it by adjusting the structure parameters. The system then tracks motion of the eye by estimating the motion parameters across the entire image sequence. Combined with image stabilization to compensate for appearance changes due to head motion, the system achieves accurate registration and motion recovery of eyes.

We propose a linear model of the relationship between the physical changes in speech and perceived emotional concepts. We make use of orthogonal bases in spite of the emotional words and the physical parameters themselves in order to avoid dependence on the method of selecting words and parameters. Furthermore we regard the emotions that listeners perceive from speech as the standard or emotional concepts because the emotions that speakers intended rely on their personality and temporary psychological state. Evaluation for relative information indicates that the proposed linear model is representable for the relationship between the physical quantities and psychological quantities in speech.