Diarization consists in the segmentation of speech signals and the clustering of homogeneous speaker segments. State-of-the-art systems typically operate upon speaker embeddings, such as i-vectors or neural x-vectors, extracted from mel cepstral coefficients (MFCCs) or spectrograms. The recent SincNet architecture extracts x-vectors directly from raw speech signals. The work reported in this paper compares the performance of different embeddings extracted from MFCCs or the raw signal for speaker diarization and broadcast media treated with compression and sub-sampling, operations which typically degrade performance. Experiments are performed with the new ALLIES database that was designed to complement existing, publicly available French corpora of broadcast radio and TV shows. Results show that, in adverse conditions, with compression and sampling mismatch, SincNet x-vectors outperform i-vectors and x-vectors by relative DERs of 43% and 73% respectively. Additionally we found that SincNet x-vectors are not the absolute best embeddings but are more robust to data mismatch than others.

France has a long tradition of preserving its archives as well as its cultural heritage, as demonstrated by the "legal deposit". Established in the Renaissance for printed documents, the legal deposit aims to allow the collection and consultation of various kinds of documents. INA, the French National Audiovisual Institute [1], is in charge of this task for France's radio and television, as well as French media on the web. INA's mission is to make the most of its collections: commercially by selling programs, and academically by making these collections available to researchers working on humanities and social sciences.

This paper presents an approach based on acoustic analysis to describe gender equality in French audiovisual streams, through the estimation of male and female speaking time. Gender detection systems based on Gaussian Mixture Models, i-vectors and Convolutional Neural Networks (CNN) were trained using an internal database of 2,284 French speakers and evaluated using REPERE challenge corpus. The CNN system obtained the best performance with a frame-level gender detection F-measure of 96.52 and a hourly women speaking time percentage error bellow 0.6%. It was considered reliable enough to realize large-scale gender equality descriptions. The proposed gender detection system has been packaged as an open-source framework.

International audience

Recently, unsupervised approaches were introduced to analyze the structure of TV programs, relying on the discovery of repeated elements within a program or across multiple episodes of the same program. These methods can discover key repeating elements, such as jingles and separators, however they cannot infer the entire structure of a program. In this paper, we propose a hierarchical use of grammatical inference to yield a temporal grammar of a program from a collection of episodes, discovering both the vocabulary of the grammar and the temporal organization of the words from the vocabulary. Using a set of basic event detectors and simple filtering techniques to detect repeating elements of interest, a symbolic representation of each episode is derived based on minimal domain knowledge. Grammatical inference based on multiple sequence alignment is then used in a hierarchical manner to provide a temporal grammar of the program at various levels of details. Experimental validation is performed on 3 distinct types of programs on 4 datasets. Qualitative analyses show that the grammars inferred at the different levels of the hierarchy are relevant and can be obtained from a fairly limited number of episodes.

In this article, we propose solutions to the problem of speaker diarization of TV talk-shows, a problem for which adapted multimodal approaches, relying on other streams of data than only audio, remain largely under exploited. Hence we propose an original system that leverages prior knowledge on the structure of this type of content, especially the visual information relating to the active speakers, for an improved diarization performance. The architecture of this system can be decomposed into two main stages. First a reliable training set is created, in an unsupervised fashion, for each participant of the TV program being processed. This data is assembled by the association of visual and audio descriptors carefully selected in a clustering cascade. Then, Support Vector Machines are used for the classification of the speech data (of a given TV program). The performance of this new architecture is assessed on two French talk-show collections: Le Grand Échiquier and On n'a pas tout dit. The results show that our new system outperforms state-of-the-art methods, thus evidencing the effectiveness of kernel-based methods, as well as visual cues, in multimodal approaches to speaker diarization of challenging contents such as TV talk-shows.

In this paper we propose a novel multimodal method for identifying unregistered speakers in a TV talk-show using a semi-supervised learning approach based on Support Vector Machines. Our study highlights the fact that specific visual features prove to be very efficient for this particular type of video content which is edited from multi-camera recordings. These visual features, motivated by prior knowledge on the approach followed by the TV director in choosing the appropriate shots, are found to bring a significant improvement in identification accuracy when used together with classic audio Mel-frequency cepstral coefficients (+8% compared to various baseline systems, in particular a standard audio only system).

In this paper we explore the way to allow a user to interactively organize a multimedia database through a dynamic interface, creating its own "audiovisual concepts" freely. The user defines distances on a small subset of documents, using low-level audio and video off-line automatically extracted descriptors. The semi-supervised learning process, relying on support vector regression used in an early fusion context, leads to generate a behavioral model of those descriptors thanks to human interaction, creating a personal audiovisual similarity measure.

We propose in this paper an original approach to the TV stream structuring problem. The goal of our work is to automatically break the TV stream into telecasts and advertisings and to label each telecast with its genre. One can think the TV stream structuring problem can be solved by an alignment of the program guide on the stream. But our study shows that, in average, only 25% of the telecasts per day are presented in the program guide. Hence, our method consists in improving statistically these program guides in order to reduce the TV stream structuring problem to a simple alignment problem. The improvement consists in adding the missing telecasts. We present an original system that lays on the modeling of past TV schedules by a contextual hidden Markov model and a regression tree. Interesting results are presented at the end of the paper

Enabling an intelligent access to multimedia data requires a powerful description language. In this paper we demonstrate why the MPEG-7 standard fails to fulfill this task. We introduce then our proposition: an audio-visual specific description language modular reduced but designed to be extensible. This language is centered on the notions of descriptor and structure with a well-defined semantics. A descriptor can be a low-level feature automatically extracted from the signal or a higher semantic concept that will be used to annotate the video documents. The descriptors can be combined into structures according to defined models that provide description patterns.