International audience

International audience

As part of a project on indexing ethno-musicological audio recordings, segmentation in singer turns automatically appeared to be essential. In this article, we present the problem of segmentation in singer turns of musical recordings and our first experiments in this direction by exploring a method based on the Bayesian Information Criterion (BIC), which are used in numerous works in audio segmentation, to detect singer turns. The BIC penalty coefficient was shown to vary when determining its value to achieve the best performance for each recording. In order to avoid the decision about which single value is best for all the documents, we propose to combine several segmentations obtained with different values of this parameter. This method consists of taking a posteriori decisions on which segment boundaries are to be kept. A gain of 7.1% in terms of F-measure was obtained compared to a standard coefficient.

This article describes an audio signal processing algorithm to detect water sounds, built in the context of a larger system aiming to monitor daily activities of elderly people. While previous proposals for water sound recognition relied on classical machine learning and generic audio features to characterize water sounds as a flow texture, we describe here a recognition system based on a physical model of air bubble acoustics. This system is able to recognize a wide variety of water sounds and does not require training. It is validated on a home environmental sound corpus with a classification task, in which all water sounds are correctly detected. In a free detection task on a real life recording, it outperformed the classical systems and obtained 70% of F-measure.

In the context of the acoustic-to-articulatory inversion, various unsupervised HMM based feature-mapping methods are assessed and compared. In a previous study we introduced an unsupervised HMM as an alternative model to the phone-HMM. We propose here to evaluate this approach using different inversion methods, in order to assess the behavior of our model and its compatibility with the most efficient inversion algorithms available. The best configuration leads to similar root mean square error (up to 1.44 mm) than phoneme-based HMM.

On this paper we present a new approach for the localisation of superposed speech areas. The system is based on the frequency tracking of speech segments following the evolution of the main amplitude frequencies and uses no learning of acoustic or prosodic models. The set of trackings of the frequencies are then grouped together using a distance based on the harmonicity, each group being the production of a single speaker. The co-occurrence of different harmonic groups is then used as a consequence of the presence of multiple speakers. Our method has been evaluated on the data of the French ANR evaluation campaign ETAPE, showing the usability of this approach.

In the context of music indexation, it would be useful to have a precise information about the number of sources performing; a source is a solo voice or an isolated instrument which produces a single note at any time. This correspondence discusses the automatic distinction between monophonic music excerpts, where only one source is present, and polyphonic ones. Our method is based on the analysis of a “confidence indicator,” which gives the confidence (in fact its inverse) on the current estimated fundamental frequency (pitch). In a monophony, the confidence indicator is low. In a polyphony, the confidence indicator is higher and varies more. This leads us to compute the short term mean and variance of this indicator, take this 2-D vector as the observation vector and model its conditional distribution with Weibull bivariate models. This probability density function is characterized by five parameters. A method to perform their estimation is developed (in theory and practice). The decision is taken considering the maximum likelihood, computed over one second. The best configuration gives a global error rate of 6.3%, performed on a balanced corpus (18 minutes in total).

When listening to foreign radio or TV programs we are able to pick up some information from the way people are interacting with each others and easily identify the most dominant speaker or the person who is interviewed. Our work relies on the existence of clues about speaker roles in acoustic and prosodic low-level features extracted from audio files and from speaker segmentations. In this paper we describe an original language-independent method which achieves the recognition of 5 roles (Anchor, Journalist, Other, Punctual Journalist, Punctual Other) with an accuracy of 85% on a 13-hour corpus composed of 46 documents among which can be found different radio shows. A feature selection method is exploited in order to highlight the most relevant features for every speaker role. Index Terms: speaker role recognition, temporal and prosodic features, speaker segmentations, feature selection.

International audience

In this paper, we describe a new application for multimedia indexing, using a system that monitors the instrumental activities of daily living to assess the cognitive decline caused by dementia. The system is composed of a wearable camera device designed to capture audio and video data of the instrumental activities of a patient, which is leveraged with multimedia indexing techniques in order to allow medical specialists to analyze several hour long observation shots efficiently.

In this paper, we describe a new method for speaker segmentation and clustering of an audio document. For the segmentation phase, we combine the generalized likelihood ratio (GLR) and the Bayesian information criterion (BIC) in a way that avoids most of the parameters tuning. For the clustering phase, we use an existing approach that utilizes the eigen vector space model (EVSM) with a bottom-up hierarchical grouping but we make some improvements by introducing prosodic information. Evaluation is done on the audio database of the ESTER evaluation campaign for the rich transcription of French Broadcast news. Results show that our method which operates without any a priori knowledge about speakers is suitable for speaker diarization as it outperforms the traditional ones with an overall diarization error rate (DER) of 16.72%.

In speaker recognition, several recent studies attempt to integrate prior knowledge in order to make better distinction between speakers. This paper studies the relative speaker discriminative power of speech transient and steady zones. An automatic segmentation is used to localize these two types of zones. Experiments are carried out with the KIST 2003 speaker evaluation database. They show that transient frames, in the neighborhood of segment boundaries, are more speaker discriminative than the middle frames of long segments which correspond to the steady parts of phones. In addition, this study leads to a higher efficiency system (than the baseline one) by processing roughly three times less data during the scoring stage.

In the field of automatic language identification, several, mostly-empirical, arithmetic fusion operations are currently done to make a consensus decision from a set of acoustics-based identification systems whose estimated performance is taken into account by means of weighting techniques. The paper presents how to apply the discriminant factor analysis method formally to compute and use weighting performance confidence indexes at the expert and class levels. Moreover, the observation level is also explored. These confidence indexes allow us not only to qualify the identification decision with additional insight by means of a certainty degree, but also to provide acoustics-based identification systems with powerful uncertainty-based inference techniques where the systems' a priori performance knowledge is a key heuristic-like element to improve language identification capabilities.

The paper addresses the soundtrack indexing of multimedia documents. Our purpose is to detect and locate one or many jingles to structure the audio dataflow in program broadcasts (reports). Each jingle is commonly represented by a sequence of spectral vectors, considered as its "signature". Potential candidates are extracted from the data flow by computing a Euclidean distance. They are validated with heuristic rules. The system evaluation is performed on TV and radio corpora (more than 10 hours, 3 TV channels and 3 radio channels). First results show that the system is efficient: among 132 jingles to recognize, we have detected 130 with our reference jingle table of 32 different key sounds.

In all the efficient speaker recognition systems, the decision score is based on the average of the likelihood ratio computed on each frame of the sentence. Except for the non speech frames which are rejected, each one has the same weight in this summation. This paper deals with the study of the speaker relevance of each frame. An automatic segmentation provides quasi stationary segments of variable length; a weight is allocated to each frame in function of its segment position and a weighted mean of the likelihood ratio is then computed. Experiments are performed with NIST 2003 speaker evaluation database. They show that the frames near segment frontiers, that is to say the transient ones, are more speaker relevant than the middle frames of long segments which correspond to the steady parts of the phones.

We present and merge two speech/music classification approaches that we have developed. The first one is a differentiated modeling approach based on a spectral analysis, which is implemented using GMM (Gaussian mixture model). The other one is based on three original features: entropy modulation, stationary segment duration and number of segments. They are merged with the classical 4 Hertz modulation energy. Our classification system is a fusion of the two approaches. It is divided in two classifications (speech/non-speech and music/non-music) and provides 94% of accuracy for speech detection and 90% for music detection, with one second of input signal. Beside the spectral information and GMM, classically used in speech/music discrimination, simple parameters bring complementary and efficient information.

International audience

In this paper, we present and merge two speech / music classification approaches of that we have developed. The first one is a differentiated modeling approach based on a spectral analysis, which is implemented with GMM. The other one is based on three original features: entropy modulation, stationary segment duration and number of segments. They are merged with the classical 4 Hertz modulation energy. Our classification system is a fusion of the two approaches. It is divided in two classifications (speech/non-speech and music/non-music) and provides 94 % of accuracy for speech detection and 90 % for music detection, with one second of input signal. Beside the spectral information and GMM, classically used in speech / music discrimination, simple parameters bring complementary and efficient information.

This paper deals with an approach to automatic language identification using only prosodic modeling. The actual approach for language identification focuses mainly on phonotactics because it gives the best results. We propose here to evaluate the relevance of prosodic information for language identification with read studio recording and spontaneous telephone speech. For read speech, experiments were performed on the five languages of the MULTEXT database [E. Campoine and J. Veronis, Nov. 1998]. On the MULTEXT corpus, our prosodic system achieved an identification rate of 79% on the five languages discrimination task. For spontaneous speech, experiments are made on the ten languages of the OGI multilingual telephone speech corpus [Y. K. Muthusamy et al., October 1992]. On the OGI MLTS corpus, the results are given for languages pair discrimination tasks, and are compared with results from [F. Cummins et al., 1999]. As a conclusion, if our prosodic system achieves good performance on read speech, it might not take into account the complexity of spontaneous speech prosody.

This paper deals with an approach to Automatic Language Identification based on rhythmic modeling and vowel system modeling. Experiments are performed on read speech for 5 European languages. They show that rhythm and stress may be automatically extracted and are relevant in language identification: using cross-validation, 78% of correct identification is reached with 21 s. utterances. The Vowel System Modeling, tested in the same conditions (cross-validation), is efficient and results in a 70% of correct identification for the 21 s. utterances. Last. merging the two models slightly improves the results.

International audience

Last year, SpLC - an ISCA Special Interest Group (SIG) centered around Speaker and Language Characterization was born. The aims of this paper are to present the SpLC SIG, its objectives, and the work done during the first year.

This paper presents an unsupervised approach to automatic language identification (ALI) based on vowel system modeling. Each language vowel system is modeled by a Gaussian mixture model (GMM) trained with automatically detected vowels. Since this detection is unsupervised and language independent, no labeled data are required. GMMs are initialized using an efficient data-driven variant of the LBG algorithm: the LBG-Rissanen (1983) algorithm. With 5 languages from the OGI MLTS corpus and in a close set identification task, we reach 79% of correct identification using only the vowel segments detected in 45 second duration utterances for the male speakers.

This paper presents two unsupervised approaches to Automatic Language Identification (ALI) based on a segmental preprocessing. In the Global Segmental Model approach, the language system is modeled by a Gaussian Mixture Model (GMM) trained with auto-matically detected segments. In the Phonetic Differenti-ated Model approach, an unsupervised detection vowel/non vowel is performed and the language model is defined with two GMMs, one to model the vowel segments and a second one to model the others seg-ments. For each approach, no labeled data are required. GMMs are initialized using an efficient data-driven variant of the LBG algorithm: the LBG-Rissanen algo-rithm. With 5 languages from the OGI MLTS corpus and in a closed set identification task, we reach 85 % of correct identification with each system using 45 second dura-tion utterances for the male speakers. We increase this performance (91%) when we merge the two systems.

Our work deals with the classical problem of merging heterogenous and asynchronous parameters. It is well known that lip reading improves the speech recognition score, especially in noise conditions; so we study more precisely the modeling of the acoustic and labial parameters to propose two automatic speech recognition systems: a direct identification is performed by using a classical HMM approach, no correlation between visual and acoustic parameters is assumed; and two correlated models, a master HMM and a slave HMM, process respectively the labial observations and the acoustic ones. To assess each approach, we use a segmental pre-processing method. Our task is the recognition of spelled French letters, in clear and noisy (cocktail party) environments. Whatever the approach and conditions, the introduction of labial features improves the performance, but the difference between the two models is not enough sufficient to provide any priority.

This paper presents our work on vowel system detection as part of a project of automatic language identification using phonological typologies. We have developed a vowel detection algorithm based on spectral analysis of the acoustic signal and requiring no learning stage. It has been tested with two telephone speech corpora: with a French corpus provided by the CNET, 7.4% of detections are false while about 25% of the vowels present in the signal are not found; and experiments with 5 languages of the OGI-TS corpus result in 88.1 % of correct detection and about 15 % of non-detection. We also present the vector quantization (VQ) LBG-Rissanen (1980, 1983) algorithm that we use for vowel system modeling. Preliminary experiments are reported.

This paper presents recent work on continuous speech labelling. We propose an original automatic labelling system where elementary phone models take a segmental analysis and the phone duration into account. These models are initialized by a short speaker-independent training stage in order to constitute a model database. From the standard phonetic transcription, phonological rules are gathered to process the various pronunciations. For each new corpus or speaker, a new quick unsupervised adaptation stage is performed to re-estimate the models, and then follows the correct labelling. We assess this system by labelling a difficult corpus (sequences of connected spelled letter) and sentences of one speaker of the BREF80 corpus. These results are quite promising, in the two experiments less than 9% of phonetic boundaries are incorrectly located.

Many researchers have seen in the articulation an intermediate level of representation. In the gestural phonetic theory, units are articulatory gestures. In order to assess this theory with observed parameters, we have defined a robust labelling system (AMULET) of the multi sensor ACCOR speech database. Main articulatory gestures searched are Voice Onset and Voice Termination on both acoustic and laryngographic signals. We present here two efficient Voiced/Unvoiced/Silence detectors for the acoustic signals and a third one for the laryngographic signal.

Speech production is a complex process relying on coordinated gestures, but the acoustic signal does not depict its underlaying organization. Accepting that articulatory gestures are directly recognized through the coarticulation process, our proposal is to investigate the correlations between acoustic and articulatory information and to assess gestural phonetic theory. We present the framework for this investigation, the automatic labelling of the multi-sensor speech database ACCOR on French and English sentences.

International audience

A major deficiency of a standard HMM is that both the spectral and the prosodic features are uniformly processed. To more efficiently combine the prosodic cues together with the acoustic ones, a two level HMM which separates the spectral and suprasegmental representations is defined. Namely, the incorporation of global sound durations is explored. More, to take into account the effects of speaking rate on the phonetic unit durational parameters, two durational models are proposed. The ways those models are integrated in the recognition processing are described. Experiments on a French number database show that such an explicit introduction of prosodic parameters reduces recognition errors rates by 20%.>

International audience

A statistical approach of the automatic segmentation of the speech signal is discussed. The purpose is to detect acoustic events which reveal articulatory changes as voice or frication onset and termination, closure, release... and formantic variations. The main idea is to model the signal by a statistical model (AR, ARMA) and to use test statistics (generalized likelihood, statistics of cumulative sum type) to detect sequentially abrupt changes in the parameters of the model. In the three segmentations which are presented here, the identification and testing procedures are sequential and monitored after every sample to obtain a better precision of change time estimations. The results obtained by each one are similar and speaker-independent. The detected acoustic events define interesting infra-phonemic units.