International audience

International audience

International audience

Computational power is crucial for the development and deployment of artificial intelligence capabilities, as the large size of deep learning models often requires significant resources. Compression methods aim to reduce model size making artificial intelligence more sustainable and accessible. Compression techniques are often applied uniformly across model layers, without considering their individual characteristics. In this paper, we introduce a customized approach that optimizes compression for each layer individually. Some layers undergo both pruning and/or quantization, while others are only quantized, with fuzzy logic guiding these decisions. The quantization precision is further adjusted based on the importance of each layer. Our experiments on both supervised and self-supervised models using the librispeech dataset show only a slight decrease in performance, with about 85% memory footprint reduction.

International audience

Given the high prevalence of mental disorders and the significant diagnostic delays and difficulties in patient follow-up, voice biomarkers hold the promise of improving access to care and therapeutic follow-up for people with psychiatric disorders. Yet, despite many years of successful research in the field, none of these voice biomarkers are implemented in clinical practice. Beyond the reductive explanation of the lack of explainability of the involved machine learning systems, we look for arguments in the epistemology and sociology of psychiatry. We show that the estimation of diagnoses, the major task in the literature, is of little interest to both clinicians and patients. After tackling the common misbeliefs about diagnosis in psychiatry in a didactic way, we propose a paradigm shift towards the estimation of clinical symptoms and signs, which not only address the limitations raised against diagnosis estimation but also enable the formulation of new machine learning tasks. We hope that this paradigm shift will empower the use of vocal biomarkers in clinical practice. It is however conditional on a change in database labeling practices, but also on a profound change in the speech processing community’s practices towards psychiatry.

This multi-partner-project contribution introduces the midway results of the Horizon 2020 FVLLMONTI project. In this project we develop a new and ultra-efficient class of ANN accelerators, the neural network compute cube$(N^{2}C^{2})$, which is specifically designed to execute complex machine learning tasks in a 3D technology, in order to provide the high computing power and ultra-high efficiency needed for future edgeAI applications. We showcase its effectiveness by targeting the challenging class of Transformer ANNs, tailored for Automatic Speech Recognition and Machine Translation, the two fundamental components of speech-to-speech translation. To gain the full benefit of the accelerator design, we develop disruptive vertical transistor technologies and execute design-technology-co-optimization (DTCO) loops from single device, to cell and compute cube level. Further, a hardware-software-co-optimization is executed, e.g. by compressing the executed speech recognition and translation models for energy efficient executing without substantial loss in precision.

Despite the continuous innovation in voice biomarkers domain for more than a decade and the apparent need for clinicians to have objective diagnostic tools, no device has yet been implemented in real clinical settings or widely adopted by clinicians. After giving a short overview of the literature, we argue that in addition to the factors usually mentioned in the literature (low performance, database sizes, transparency, etc.), an underestimated but crucial factor preventing the use of such systems is the therapeutic relationship. We also discuss the "objectivity" of such systems, and the place of diagnosis in clinical practice and its conceptual limitations. In order to shape useful and relevant voice biomarkers, we propose to estimate symptoms instead of diagnosis, and draw perspectives related to this paradigm, which will require databases annotated with patients’ symptoms rather than only their pathological status.

This article is about the automatic estimation of sleepiness in hypersomnia patients recorded during a reading task. Based on the Multiple Sleep Latency Corpus, our main contribution is to explore new formulations of sleepiness detection in speech by specifying and performing five sleepiness-related classification tasks. We automatically classify three symptoms, and two syndromes, i.e. combinations of symptoms that are closer to clinical reasoning. Another contribution of this paper is the use of a simple and interpretable pipeline integrating selecting voice biomarkers of sleepiness, i.e. features that are both sensible and specific to sleepiness. In particular, specificity is adressed integrating a decorrelation step in the pipeline, which allows to certify that the descriptors selected by the pipeline are indeed specific of sleepiness with respect to 7 cofactors (age, BMI, etc.).

Following the release of the SLEEP corpus during the Interspeech 2019 paralinguistic continuous sleepiness estimation challenge, a paper presented at Interspeech 2020 by Huckvale et al. examined the reasons for the poor performance of the models proposed for this task. Careful analyses of the corpus led to the conclusion that its bias makes it hazardous to use for training machine learning systems, but a perceptual experiment on a subset of this corpus seemed to indicate that human hearing is however able to estimate sleepiness on this corpus.In this study, we present the results of the Endymion replication study, in which the same samples were rated by thirty French-speaking naive listeners. We then discuss the causes of the differences between the two studies and examine the effect of listener and sample characteristics on annotation performances.

International audience

In automatic speech recognition, deep learning models such as transformers are increasingly used for their high performance. However, they suffer from their large size, which makes it very difficult to use them in real contexts. Hence the idea of pruning them. Conventional pruning methods are not optimal and sometimes not efficient since they operate blindly without taking into account the nature of the layers or their number of parameters or their distribution. In this work, we propose to perform a fine-grained analysis of the transformer model layers in order to determine the most efficient pruning approach. We show that it is more appropriate to prune some layers than others and underline the importance of knowing the behavior of the layers to choose the pruning approach.

International audience

Following patients with chronic sleep disorders involves multiple appointments between doctors and patients which often results in episodic follow-ups with unevenly spaced interviews. Speech technologies and virtual doctors can help improve this follow-up. However, there are still some challenges to overcome: sleepiness measurements are diverse and are not always correlated, and most past research focused on detecting nstantaneous sleepiness levels of healthy sleep-deprived subjects. This article presents a large database to assess the sleepiness level of highly phenotyped patients that complain from excessive daytime sleepiness. Based on the Multiple Sleep Latency Test, it differs from existing databases by multiple aspects. First, it is omposed of recordings from patients suffering from excessive daytime sleepiness instead of sleep deprived healthy subjects. Second, it incites the subjects to sleep contrary to existing stressing sleepiness deprivation experimental paradigms. Third, the sleepiness level of the patients is evaluated with different temporal granularities - long term sleepiness and short term sleepiness - and both objective and subjective sleepiness measures are collected. Finally, it relies on the recordings of 94 highly phenotyped patients, allowing to unravel the influences of different physical factors (age, sex, weight, ... ) on voice.

International audience

This paper presents a unified framework for smooth convex regularization of discrete optimal transport problems. In this context, the regularized optimal transport turns out to be equivalent to a matrix nearness problem with respect to Bregman divergences. Our framework thus naturally generalizes a previously proposed regularization based on the Boltzmann-Shannon entropy related to the Kullback-Leibler divergence, and solved with the Sinkhorn-Knopp algorithm. We call the regularized optimal transport distance the rot mover's distance in reference to the classical earth mover's distance. By exploiting alternate Bregman projections, we develop the alternate scaling algorithm and non-negative alternate scaling algorithm, to compute efficiently the regularized optimal plans depending on whether the domain of the regularizer lies within the non-negative orthant or not. We further enhance the separable case with a sparse extension to deal with high data dimensions. We also instantiate our framework and discuss the inherent specificities for well-known regularizers and statistical divergences in the machine learning and information geometry communities. Finally, we demonstrate the merits of our methods with experiments using synthetic data to illustrate the effect of different regularizers, penalties and dimensions, as well as real-world data for a pattern recognition application to audio scene classification.

International audience

International audience

This paper describes an accent identification system for Portuguese, that explores different type of properties: acoustic, phono tactic and prosodic. The system is designed to be used as a pre-processing module for the Portuguese automatic speech recognition system developed at INESC-ID. In terms of variety identification, the overall rate of correct identification is 69.0% if all 7 varieties are considered, and the best results are obtained for Brazilian Portuguese, also the variety that proved easiest to identify in perceptual experiments. When distinguishing between European, Brazilian and African Portuguese, the identification rate goes up to 94.7%. The fact that the prosodic system alone can achieve an identification rate of 77% is also worth investigating.

This paper addresses the problem of modeling prosody for language identification. The aim is to create a system that can be used prior to any linguistic work to show if prosodic differences among languages or dialects can be automatically determined. In previous papers, we defined a prosodic unit, the pseudosyllable. Rhythmic modeling has proven the relevance of the pseudosyllable unit for automatic language identification. In this paper, we propose to model the prosodic variations, that is to say model sequences of prosodic units. This is achieved by the separation of phrase and accentual components of intonation. We propose an independent coding of those components on differentiated scales of duration. Short-term and long-term language-dependent sequences of labels are modeled by n-gram models. The performance of the system is demonstrated by experiments on read speech and evaluated by experiments on spontaneous speech. Finally, an experiment is described on the discrimination of Arabic dialects, for which there is a lack of linguistic studies, notably on prosodic comparisons. We show that our system is able to clearly identify the dialectal areas, leading to the hypothesis that those dialects have prosodic differences.

International audience

In the field of automatic language identification, several, mostly-empirical, arithmetic fusion operations are currently done to make a consensus decision from a set of acoustics-based identification systems whose estimated performance is taken into account by means of weighting techniques. The paper presents how to apply the discriminant factor analysis method formally to compute and use weighting performance confidence indexes at the expert and class levels. Moreover, the observation level is also explored. These confidence indexes allow us not only to qualify the identification decision with additional insight by means of a certainty degree, but also to provide acoustics-based identification systems with powerful uncertainty-based inference techniques where the systems' a priori performance knowledge is a key heuristic-like element to improve language identification capabilities.

We present and merge two speech/music classification approaches that we have developed. The first one is a differentiated modeling approach based on a spectral analysis, which is implemented using GMM (Gaussian mixture model). The other one is based on three original features: entropy modulation, stationary segment duration and number of segments. They are merged with the classical 4 Hertz modulation energy. Our classification system is a fusion of the two approaches. It is divided in two classifications (speech/non-speech and music/non-music) and provides 94% of accuracy for speech detection and 90% for music detection, with one second of input signal. Beside the spectral information and GMM, classically used in speech/music discrimination, simple parameters bring complementary and efficient information.

International audience

In this paper, we present and merge two speech / music classification approaches of that we have developed. The first one is a differentiated modeling approach based on a spectral analysis, which is implemented with GMM. The other one is based on three original features: entropy modulation, stationary segment duration and number of segments. They are merged with the classical 4 Hertz modulation energy. Our classification system is a fusion of the two approaches. It is divided in two classifications (speech/non-speech and music/non-music) and provides 94 % of accuracy for speech detection and 90 % for music detection, with one second of input signal. Beside the spectral information and GMM, classically used in speech / music discrimination, simple parameters bring complementary and efficient information.

This paper deals with an approach to automatic language identification using only prosodic modeling. The actual approach for language identification focuses mainly on phonotactics because it gives the best results. We propose here to evaluate the relevance of prosodic information for language identification with read studio recording and spontaneous telephone speech. For read speech, experiments were performed on the five languages of the MULTEXT database [E. Campoine and J. Veronis, Nov. 1998]. On the MULTEXT corpus, our prosodic system achieved an identification rate of 79% on the five languages discrimination task. For spontaneous speech, experiments are made on the ten languages of the OGI multilingual telephone speech corpus [Y. K. Muthusamy et al., October 1992]. On the OGI MLTS corpus, the results are given for languages pair discrimination tasks, and are compared with results from [F. Cummins et al., 1999]. As a conclusion, if our prosodic system achieves good performance on read speech, it might not take into account the complexity of spontaneous speech prosody.

This paper deals with an approach to Automatic Language Identification based on rhythmic modeling and vowel system modeling. Experiments are performed on read speech for 5 European languages. They show that rhythm and stress may be automatically extracted and are relevant in language identification: using cross-validation, 78% of correct identification is reached with 21 s. utterances. The Vowel System Modeling, tested in the same conditions (cross-validation), is efficient and results in a 70% of correct identification for the 21 s. utterances. Last. merging the two models slightly improves the results.

International audience