International audience

Recent advancements in generative AI, particularly in speech synthesis, have enabled the generation of highly natural-sounding synthetic speech that closely mimics human voices. While these innovations hold promise for applications like assistive technologies, they also pose significant risks, including misuse for fraudulent activities, identity theft, and security threats. Current research on spoofing detection countermeasures remains limited by generalization to unseen deepfake attacks and languages. To address this, we propose a gating mechanism extracting relevant feature from the speech foundation XLS-R model as a front-end feature extractor. For downstream back-end classifier, we employ Multi-kernel gated Convolution (MultiConv) to capture both local and global speech artifacts. Additionally, we introduce Centered Kernel Alignment (CKA) as a similarity metric to enforce diversity in learned features across different MultiConv layers. By integrating CKA with our gating mechanism, we hypothesize that each component helps improving the learning of distinct synthetic speech patterns. Experimental results demonstrate that our approach achieves state-of-the-art performance on in-domain benchmarks while generalizing robustly to out-of-domain datasets, including multilingual speech samples. This underscores its potential as a versatile solution for detecting evolving speech deepfake threats.

International audience

International audience

This paper is a positional paper that surveys existing technologies for animating signing avatars from written language. The main grammatical mechanisms of sign languages are described, and in particular the sign inflecting mechanisms in light of the processes of spatialization and iconicity that characterize these visual-gestural languages. The challenges faced by sign language generation systems using signing avatars are then outlined, as well as unresolved issues in building text-to-sign generation systems.

Document analysis is essential in many industrial applications. However, engineering natural language resources to represent entire documents is still challenging. Besides, available resources in French are scarce and do not cover all possible tasks, especially in specific business applications. In this context, we present a French newsletter dataset and its use to predict the good or bad impact of newsletters on readers. We propose a new representation of newsletters in the form of graphs that consider the newsletters' layout. We evaluate the relevance of the proposed representation to predict a newsletter's performance in terms of open and click rates using graph analysis methods.

We present a processing pipeline for Chinese public procurement document harvesting, with the aim of producing strategic data with greater added value. It consists of three micro-modules: data collection, information extraction, database indexing. The information extraction part is implemented through a hybrid system which combines rule-based and machine learning approaches. Rule-based method is used for extracting information with presenting recurring morphological features, such as dates, amounts and contract awardee information. Machine learning method is used for trade detection in the title of procurement documents.

In this paper, we propose DiFF-RF, an ensemble approach composed of random partitioning binary trees to detect point-wise and collective (as well as contextual) anomalies. Thanks to a distance-based paradigm used at the leaves of the trees, this semi-supervised approach solves a drawback that has been identified in the isolation forest (IF) algorithm. Moreover, taking into account the frequencies of visits in the leaves of the random trees allows to significantly improve the performance of DiFF-RF when considering the presence of collective anomalies. DiFF-RF is fairly easy to train, and good performance can be obtained by using a simple semi-supervised procedure to setup the extra hyper-parameter that is introduced. We first evaluate DiFF-RF on a synthetic data set to i) verify that the limitation of the IF algorithm is overcome, ii) demonstrate how collective anomalies are actually detected and iii) to analyze the effect of the meta-parameters it involves. We assess the DiFF-RF algorithm on a large set of datasets from the UCI repository, as well as four benchmarks related to network intrusion detection applications. Our experiments show that DiFF-RF almost systematically outperforms the IF algorithm and one of its extended variant, but also challenges the one-class SVM baseline, deep learning variational auto-encoder and ensemble of auto-encoder architectures. Finally, DiFF-RF is computationally efficient and can be easily parallelized on multi-core architectures.

Detecting an anomalous human behavior can be a challenging task. In this paper, we present a novel objective function for autoencoders which include a temporal component. Our method is a fully end-to-end semi-supervised approach for video anomaly detection. The autoencoder is trained to reconstruct a sample from a partial input, by interpolating latent codes obtained from this partial input. We show this approach improves over using usual autoencoder objective functions for video anomaly detection and achieves results close to the state of the art on a broad range of datasets. Our code is publicly available on github.

This paper introduces a new similarity measure, the covering similarity, which we formally define for evaluating the similarity between a symbolic sequence and a set of symbolic sequences. A pairwise similarity can also be directly derived from the covering similarity to compare two symbolic sequences. An efficient implementation to compute the covering similarity is proposed which uses a suffix-tree data structure, but other implementations, based on suffix array for instance, are possible and are possibly necessary for handling very large-scale problems. We have used this similarity to isolate attack sequences from normal sequences in the scope of host-based intrusion detection. We have assessed the covering similarity on two well-known benchmarks in the field. In view of the results reported on these two datasets for the state-of-the-art methods, according to the comparative study, we have carried out based on three challenging similarity measures commonly used for string processing, or in bioinformatics, we show that the covering similarity is particularly relevant to address the detection of anomalies in sequences of system calls.

Endowing animated virtual characters with emotionally expressive behaviors is paramount to improving the quality of the interactions between humans and virtual characters. Full-body motion, in particular, with its subtle kinematic variations, represents an effective way of conveying emotionally expressive content. However, before synthesizing expressive full-body movements, it is necessary to identify and understand what qualities of human motion are salient to the perception of emotions and how these qualities can be exploited to generate novel and equally expressive full-body movements. Based on previous studies, we argue that it is possible to perceive and generate expressive full-body movements from a limited set of joint trajectories, including end-effector trajectories and additional constraints such as pelvis and elbow trajectories. Hence, these selected trajectories define a significant and reduced motion space, which is adequate for the characterization of the expressive qualities of human motion and that is both suitable for the analysis and generation of emotionally expressive full-body movements. The purpose and main contribution of this work is the methodological framework we defined and used to assess the validity and applicability of the selected trajectories for the perception and generation of expressive full-body movements. This framework consists of the creation of a motion capture database of expressive theatrical movements, the development of a motion synthesis system based on trajectories re-played or re-sampled and inverse kinematics, and two perceptual studies.

This paper presents an attempt to evaluate three different sets of features extracted from prosodic descriptors and Big Five traits for building an anomaly detector. The Big Five model enables to capture personality information. Big Five traits are extracted from a manual annotation while Prosodic features are extracted directly from the speech signal. Two different anomaly detection methods are evaluated: Gaussian Mixture Model (GMM) and One-Class SVM (OC-SVM), each one combined with a threshold classification to decide the ”normality” of a sample. The different combinations of models and feature sets are evaluated on the SSPNET-Personality corpus which has already been used in several experiments, including a previous work on separating two types of personality profiles in a supervised way. In this work, we propose the above mentioned unsupervised or semi-supervised methods, and discuss their performance, to detect particular audio-clips produced by a speaker with an abnormal personality. Results show that using automatically extracted prosodic features competes with the Big Five traits. The overall detection performance achieved by the best model is around 0.8 (F1-measure)

Recent results in the affective computing sciences point towards the importance of virtual characters capable of conveying affect through their movements. However, in spite of all advances made on the synthesis of expressive motions, almost all of the existing approaches focus on the translation of stylistic content rather than on the generation of new expressive motions. Based on studies that show the importance of end-effector trajectories in the perception and recognition of affect, this paper proposes a new approach for the automatic generation of affective motions. In this approach, expressive content is embedded in a low-dimensional manifold built from the observation of end-effector trajectories. These trajectories are taken from an expressive motion capture database. Body motions are then reconstructed by a multi-chain Inverse Kinematics controller. The similarity between the expressive content of MoCap and synthesized motions is quantitatively assessed through information theory measures.

Virtual characters capable of showing emotional content are considered as more believable and engaging. However, in spite of the numerous psychological studies and machine learning applications trying to decode the most salient features in the expression and perception of affect, there is still no common understanding about how affect is conveyed through body motions. Based on findings reported by the psychology research community and quantitative results obtained in the computer animation domain during the last years, we propose to represent affective bodily movement through a low-dimensional parameterization consisting of the spatio-temporal trajectories of eight main joints in the human body (hands, head, feet, elbows and pelvis). Using a combined evaluation protocol, we show that this low-dimensional parameterization and the features derived from it are a compact and sufficient representation of affective motions that can be used for automatic recognition of affect and the generation of new affective-expressive motions.

This paper proposes some extensions to the work on kernels dedicated to string or time series global alignment based on the aggregation of scores obtained by local alignments. The extensions that we propose allow us to construct, from classical recursive definition of elastic distances, recursive edit distance (or time-warp) kernels that are positive definite if some sufficient conditions are satisfied. The sufficient conditions we end up with are original and weaker than those proposed in earlier works, although a recursive regularizing term is required to get proof of the positive definiteness as a direct consequence of the Haussler's convolution theorem. Furthermore, the positive definiteness is maintained when a symmetric corridor is used to reduce the search space, and thus the algorithmic complexity, which is quadratic in the worst case. The classification experiment we conducted on three classical time-warp distances (two of which are metrics), using support vector machine classifier, leads to the conclusion that when the pairwise distance matrix obtained from the training data is far from definiteness, the positive definite recursive elastic kernels outperform in general the distance substituting kernels for several classical elastic distances we have tested.

In the field of gestural action recognition, many studies have focused on dimensionality reduction along the spatial axis, to reduce both the variability of gestural sequences expressed in the reduced space, and the computational complexity of their processing. It is noticeable that very few of these methods have explicitly addressed the dimensionality reduction along the time axis. This is however a major issue with regard to the use of elastic distances characterized by a quadratic complexity. To partially fill this apparent gap, we present in this paper an approach based on temporal down-sampling associated to elastic kernel machine learning. We experimentally show, on two data sets that are widely referenced in the domain of human gesture recognition, and very different in terms of quality of motion capture, that it is possible to significantly reduce the number of skeleton frames while maintaining a good recognition rate. The method proves to give satisfactory results at a level currently reached by state-of-the-art methods on these data sets. The computational complexity reduction makes this approach eligible for real-time applications.

This paper proposes a framework dedicated to the construction of what we call discrete elastic inner product allowing one to embed sets of nonuniformly sampled multivariate time series or sequences of varying lengths into inner product space structures. This framework is based on a recursive definition that covers the case of multiple embedded time elastic dimensions. We prove that such inner products exist in our general framework and show how a simple instance of this inner product class operates on some prospective applications, while generalizing the euclidean inner product. Classification experimentations on time series and symbolic sequences data sets demonstrate the benefits that we can expect by embedding time series or sequences into elastic inner spaces rather than into classical euclidean spaces. These experiments show good accuracy when compared to the euclidean distance or even dynamic programming algorithms while maintaining a linear algorithmic complexity at exploitation stage, although a quadratic indexing phase beforehand is required.

Abstract—In the last two decades, a lot of protein 3D shapes have been discovered, characterized, and made available thanks to the Protein Data Bank (PDB), that is nevertheless growing very quickly. New scalable methods are thus urgently required to search through the PDB efficiently. This paper presents an approach entitled LNA (Laplacian Norm Alignment) that performs a structural comparison of two proteins with dynamic programming algorithms. This is achieved by characterizing each residue in the protein with scalar features. The feature values are calculated using a Laplacian operator applied on the graph corresponding to the adjacency matrix of the residues. The weighted Laplacian operator we use estimates, at various scales, local deformations of the topology where each residue is located. On some benchmarks, which are widely shared by the community, we obtain qualitatively similar results compared to other competing approaches, but with an algorithm one or two order of magnitudes faster. 180,000 protein comparisons can be done within 1 second with a single recent Graphical Processing Unit (GPU), which makes our algorithm very scalable and suitable for real-time database querying across the web.

We propose a new multi-resolution indexing and retrieval method of the similarity search problem in time series databases. The proposed method is based on a fast-and-dirty filtering scheme that iteratively reduces the search space using several resolution levels. For each resolution level the time series are approximated by an appropriate function. The distance between the time series and the approximating function is computed and stored at indexing-time. At query-time, assigned filters use these pre-computed distances to exclude wide regions of the search space, which do not contain answers to the query, using the least number of query-time distance computations. The resolution level is progressively increased to converge towards higher resolution levels where the exclusion power rises, but the cost of query-time distance computations also increases. The proposed method uses lower bounding distances, so there are no false dismissals, and the search process returns all the possible answers to the query. A post-processing scanning on the candidate response set is performed to filter out any false alarms and return the final response set. We present experimentations that compare our method with sequential scanning on different datasets, using different threshold values and different approximating functions. The experiments show that our new method is faster than sequential scanning by an order of magnitude.

A Tabu move merge split (TMMS) algorithm is proposed for the polygonal approximation problem. TMMS incorporates a tabu principle to avoid premature convergence into local minima. TMMS is compared to optimal, near to optimal top down multi-resolution (TDMR) and classical split and merge heuristics solutions. Experiments show that potential improvements for crudest approximations can be obtained. The evaluation is carried out on 2D geographic maps according to effectiveness and efficiency measures.

In a way similar to the string-to-string correction problem, we address discrete time series similarity in light of a time-series-to-time-series-correction problem for which the similarity between two time series is measured as the minimum cost sequence of edit operations needed to transform one time series into another. To define the edit operations, we use the paradigm of a graphical editing process and end up with a dynamic programming algorithm that we call Time Warp Edit Distance (TWED). TWED is slightly different in form from Dynamic Time Warping (DTW), Longest Common Subsequence (LCSS), or Edit Distance with Real Penalty (ERP) algorithms. In particular, it highlights a parameter that controls a kind of stiffness of the elastic measure along the time axis. We show that the similarity provided by TWED is a potentially useful metric in time series retrieval applications since it could benefit from the triangular inequality property to speed up the retrieval process while tuning the parameters of the elastic measure. In that context, a lower bound is derived to link the matching of time series into downsampled representation spaces to the matching into the original space. The empiric quality of the TWED distance is evaluated on a simple classification task. Compared to Edit Distance, DTW, LCSS, and ERP, TWED has proved to be quite effective on the considered experimental task.

We present in this article a method to wisely replicate information in an unstructured peer-to-peer network. We make no assumption on the network topology. Thematic agents move randomly on the network and estimate the level of redundancy of the specific information they are dealing with. They can delete or create replicas if this estimated redundancy is too high or too low. Experiments show that we can achieve an homogeneous distribution of information in a distributed environment while achieving a high level of fault tolerance.

This paper addresses the dynamic control of multi- joint systems based on learning of sensory-motor transformations. To avoid the dependency of the controllers to the analytical knowledge of the multi- joint system, a non parametric learning approach is developed which identifies non linear mappings between sensory signals and motor commands involved in control motor systems. The learning phase is handled through a General Regression Neural Network (GRNN) that implements a non parametric Nadarayan-Watson regression scheme and a set of local PIDs. The resulting dynamic sensory-motor controller (DSMC) is intensively tested within the scope of hand-arm reaching and tracking movements in a dynamical simulation environment. (DSMC) proves to be very effective and robust. Moreover, it reproduces kinematics behaviors close to captured hand-arm movements.

Focused structured document retrieval employs the concept of best entry point (BEP), which is intended to provide optimal starting-point from which users can browse to relevant document components [4]. In this paper we describe and evaluate a method for finding BEPs in XML documents. Experiments conducted within the framework of INEX 2006 evaluation campaign on the Wikipedia XML collection [2] shown the effectiveness of the proposed approach.

This paper presents an efficient method of learning motion control for autonomous animated characters. The method uses a nonparametric learning approach which identifies nonlinear mappings between sensory signals and motor control. The learning phase is handled through a general regression neural network model simulated by using near neighbors search algorithms (kd-tree). The resulting adaptive model (ASMM) is suitable for the expressive animation of an anthropomorphic hand-arm system involved in reaching or tracking tasks.

The authors propose an algorithm based on a constrained iterative optimization process using a gradient method. The constraints can be applied on the time as well as the spectral dimension. Without any temporal constraints the algorithm produces relatively compact functions which exhibit a secondary lobe structure. Applying B.S. Atal's (1983) method each important secondary lobe is modeled by a different parametric target and associated compact function. After presenting the new decomposition technique, the authors focus on an experiment where they have been able automatically to infer directly from the speech signal articulatory gestures which intervene in a stimulus like /iyiy/ by means of an articulation with two degrees of freedom (lip protrusion versus lip opening). This study reports the first step toward a numerical model of articulatory inversion.>

A new global approach to coarse classification of speech segments is presented. Markov modeling is applied on an analytic approach to coarticulation. Speech signal evolution of diphone-like segments is modelized by a point moving frame by frame in a factorial space. Kinematic segmentation applied to the trajectory covered by this point enables the authors to build stochastic models of these segments. Input parameters of a Markov model are extracted from a skeleton of this trajectory considered as a functional model of overlapping segments. The evaluation of such representations in a recognition task gives some elements of discussion about the relative information contained in steady states versus transient segments and acoustical trajectories in general.>