Alzheimer’s disease (AD) is the most common form of dementia and a progressive, irreversible brain disorder that affects millions worldwide. The majority of existing research on AD classification relies on cross-sectional brain magnetic resonance imaging studies, which consider information from a single time point and fail to account for the progressive nature of AD. Longitudinal analysis, however, is crucial for capturing AD evolution and enabling more accurate diagnosis. To address this gap, we propose GeoHSAF, a novel hippocampus-based geometric learning framework for longitudinal AD classification. To overcome the challenge of missing or inconsistent hippocampal shapes across subjects and time points, our framework includes an interpolation module that predicts intermediate shapes, ensuring temporal continuity. We evaluate the effectiveness of GeoHSAF on three public longitudinal AD datasets: ADNI, OASIS, and AIBL, and benchmark its performance against existing approaches. GeoHSAF achieves new state-of-the-art results on binary classification tasks (AD vs. NC), while also demonstrating strong performance on more challenging triple-class classification tasks (AD vs. NC vs. MCI). Our work is fully reproducible, and all code is available at: https://github.com/ayodejimb/GeoHSAF

Alzheimer’s disease (AD) is a progressive and irreversible brain disorder that remains incurable. Research has shown a strong link between gait and cognition, with AD significantly affecting gait and behavioral patterns. While most preclinical studies analyze gait using mice pawprints, this approach is prone to inconsistencies due to variations in pawprints’ correction by different experimenters. In contrast, skeleton-based behavioral analysis provides a more consistent and cleaner representation. In this work, we collect a new mice dataset and propose a novel skeleton-based geometric deep attention network for disease classification using the mice’s behavioral information from this dataset. We begin our analysis by extracting posture data as skeleton landmark sequences which are then processed by our proposed network for the classification. Our proposed approach demonstrates promising results, making it particularly relevant for preclinical gait research and we conduct an ablation study on our proposed approach to demonstrate its effectiveness. Our work is reproducible and the codes are publicly available.11https://github.com/ayodejimb/Geometric_Deep_Behavioral_Analysis

This paper presents a novel approach for population estimation by focusing on the comparison between two deep learning models: popVAE, which integrates a Variational Autoencoder (VAE) for latent spatial contextual feature extraction, and popCNN, which relies on conventional convolutional layers for spatial contextual feature extraction. The models were evaluated on a geospatial dataset from Tunisia. Our results show that popVAE outperforms popCNN in terms of predictive accuracy, as evidenced by a higher Coefficient of Determination (R2) of 0.8760 and lower Mean Squared Error (MSE) values. The popVAE performance was also compared to baseline models and achieved competitive accuracy. These findings suggest that VAEs offer significant advantages in population estimation tasks by capturing complex spatial dependencies in the data. Our code and training dataset are available1.

International audience

Skeletal motion prediction aims to forecast future movement based on 3D skeleton sequences, crucial for applications such as autonomous driving and virtual reality. However, anticipating the motion of 3D articulated objects is challenging due to their inherent non linearity and stochastic nature. Existing approaches often represent the skeleton as a set of 3D joints, which unfortunately ignores joint relationships and anatomical constraints. Moreover, conventional recurrent neural networks struggle with capturing long-term dependencies in motion contexts. To address these limitations, we propose encoding anatomical constraints through Lie algebra representation, integrating self-attention in transformer networks. Our Motion-Lie Transformer architecture, leveraging Transformers with self-attention, preserves human motion kinematics. Empirical evaluations on datasets like Human3.6M, GTA-IM, and PROX promise competitive performance and accurate 3D human pose estimation.

The Transformer architecture has gained significant popularity in computer vision tasks due to its capacity to generalize and capture long-range dependencies. This characteristic makes it well-suited for generating spatiotemporal tokens from videos. On the other hand, convolutions serve as the fundamental backbone for processing images and videos, as they efficiently aggregate information within small local neighborhoods to create spatial tokens that describe the spatial dimension of a video. While both CNN-based architectures and pure transformer architectures are extensively studied and utilized by researchers, the effective combination of these two backbones has not received comparable attention in the field of activity recognition. In this research, we propose a novel approach that leverages the strengths of both CNNs and Transformers in an hybrid architecture for performing activity recognition using RGB videos. Specifically, we suggest employing a CNN network to enhance the video representation by generating a 128-channel video that effectively separates the human performing the activity from the background. Subsequently, the output of the CNN module is fed into a transformer to extract spatiotemporal tokens, which are then used for classification purposes. Our architecture has achieved new SOTA results with 90.05 %, 99.6%, and 95.09% on HMDB51. UCF101. and ETRI-Activity3D respectively.

Deep learning architectures, albeit successful in most computer vision tasks, were designed for data with an underlying Euclidean structure, which is not usually fulfilled since pre-processed data may lie on a non-linear space. In this article, we propose a geometric deep learning approach using rigid and non-rigid transformations, named KShapenet, for 2D and 3D landmark-based human motion analysis. Landmark configuration sequences are first modeled as trajectories on Kendall's shape space and then mapped to a linear tangent space. The resulting structured data are then input to a deep learning architecture, which includes a layer that optimizes over rigid and non-rigid transformations of landmark configurations, followed by a CNN-LSTM network. We apply KShapenet to 3D human landmark sequences for action and gait recognition, and 2D facial landmark sequences for expression recognition, and demonstrate the competitiveness of the proposed approach with respect to state-of-the-art.

Deep Learning architectures, albeit successful in most computer vision tasks, were designed for data with an underlying Euclidean structure, which is not usually fulfilled since pre-processed data may lie on a non-linear space. In this paper, we propose a geometry aware deep learning approach using rigid and non rigid transformation optimization for skeleton-based action recognition. Skeleton sequences are first modeled as trajectories on Kendall’s shape space and then mapped to the linear tangent space. The resulting structured data are then fed to a deep learning architecture, which includes a layer that optimizes over rigid and non rigid transformations of the 3D skeletons, followed by a CNN-LSTM network. The assessment on two large scale skeleton datasets, namely NTU-RGB+D and NTU-RGB+D 120, has proven that the proposed approach outperforms existing geometric deep learning methods and exceeds recently published approaches with respect to the majority of configurations.

Generative adversarial networks (GANs) are one of the major ML techniques for data augmentation and classification, in the field of image processing, computer vision and natural language processing. However, in the field of data networks and protocols the use of GANs for data generation and classification (at packet level) is very limited or relatively new. Although, GANs specific properties and characteristics can be highly relevant in this context (unsupervised technique). This limitation, is even more critical if we consider network protocols or communication oriented protocols such as SIP VoIP To address this problem, we propose “SIP-GAN” an extension and adaptation of GANs model for SIP, aiming to process and generate SIP traffic at packet level. The proposed generic model includes an encoder, a generator, and a decoder The encoder extracts information from pcap data, associates and converts these SIP data into a GAN image representation. The generator is based on a DCGAN model, that generates new SIP dataset from each extracted image. The decoder combines the generated images and reconstruct a valid pcap file (SIP file). A specific testbed, with a formal and practical analysis, demonstrate the validity of the generated data, from the SIP-GAN model. Also, the experimental and performance results are globally satisfactory, showing the relevance of our proposed SIP-GAN based traffic generator in this context.

The detection and tracking of human landmarks in video streams has gained in reliability partly due to the availability of affordable RGB-D sensors. The analysis of such time-varying geometric data is playing an important role in the automatic human behavior understanding. However, suitable shape representations as well as their temporal evolution, termed trajectories, often lie to nonlinear manifolds. This puts an additional constraint (i.e., nonlinearity) in using conventional Machine Learning techniques. As a solution, this paper accommodates the well-known Sparse Coding and Dictionary Learning approach to study time-varying shapes on the Kendall shape spaces of 2D and 3D landmarks. We illustrate effective coding of 3D skeletal sequences for action recognition and 2D facial landmark sequences for macro- and micro-expression recognition. To overcome the inherent nonlinearity of the shape spaces, intrinsic and extrinsic solutions were explored. As main results, shape trajectories give rise to more discriminative time-series with suitable computational properties, including sparsity and vector space structure. Extensive experiments conducted on commonly-used datasets demonstrate the competitiveness of the proposed approaches with respect to state-of-the-art.

In this paper, an effective approach is proposed for automatic 4D Facial Expression Recognition (FER). It combines two growing but disparate ideas in the domain of computer vision, i.e., computing spatial facial deformations using a Riemannian method and magnifying them by a temporal filtering technique. Key frames highly related to facial expressions are first extracted from a long 4D video through a spectral clustering process, forming the Onset-Apex-Offset flow. It is then analyzed to capture the spatial deformations based on Dense Scalar Fields (DSF), where registration and comparison of neighboring 3D faces are jointly led. The generated temporal evolution of these deformations is further fed into a magnification method to amplify facial activities over time. The proposed approach allows revealing subtle deformations and thus improves the emotion classification performance. Experiments are conducted on the BU-4DFE and BP-4D databases, and competitive results are achieved compared to the state-of-the-art.

Suitable shape representations as well as their temporal evolution, termed trajectories, often lie to non-linear manifolds. This puts an additional constraint (i.e., non-linearity) in using conventional machine learning techniques for the purpose of classification, event detection, prediction, etc. This paper accommodates the well-known Sparse Coding and Dictionary Learning to the Kendall's shape space and illustrates effective coding of 3D skeletal sequences for action recognition. Grounding on the Riemannian geometry of the shape space, an intrinsic sparse coding and dictionary learning formulation is proposed for static skeletal shapes to overcome the inherent non-linearity of the manifold. As a main result, initial trajectories give rise to sparse code functions with suitable computational properties, including sparsity and vector space representation. To achieve action recognition, two different classification schemes were adopted. A bi-directional LSTM is directly performed on sparse code functions, while a linear SVM is applied after representing sparse code functions using Fourier temporal pyramid. Experiments conducted on three publicly available datasets show the superiority of the proposed approach compared to existing Riemannian representations and its competitiveness with respect to other recently-proposed approaches. When the benefits of invariance are maintained from the Kendall's shape representation, our approach not only overcomes the problem of non-linearity but also yields to discriminative sparse code functions.

In this paper we propose a novel gait recognition approach from animated 3D skeletal data. Our approach is based on two disparate ideas from Shape Analysis and Functional Data Analysis (FDA) for a joint geometric-functional analysis. That is, skeletal sequences are viewed as time-parametrized trajectories on the Kendall's shape space when scaling, translation and rotation variations are filtered out from fixed-time 3D skeletons. A Riemannian Functional Principal Component Analysis (RFPCA) is carried out on our manifold-valued trajectories in order to build a new basis of principal functions, termed EigenTrajectories. Thus, each trajectory, could be projected into the eigenbasis which give rise to a compact signature, or EigenScores. The latter is fed to pre-trained `One-vs-All' SVM classifiers for identity recognition and authentication. Based on the geometry of the underlying shape space, tools for re-sampling and synchronizing trajectories are naturally derived to apply the proposed variant of FPCA. We have conducted experiments on a subset of the CMU dataset. Our approach shows promising results compared to the state-of-the-art when a compact and robust signature is considered.

Despite a great deal of progress in face recognition technologies, current solutions are still vulnerable to spoof attacks. In fact, it is easy to access digital replicas of facial biometric information from readily available photos, videos and 3D masks. The literature contains several face anti spoofing methods that try to detect whether the face in the front of the recognition system is real or an artificial replica. However, these methods are not robust and require many improvements since they are sensitive to lightening conditions and pose variations. In order to address these issues, we propose a novel face anti spoofing method based on Multi Color Convolutional Neural Network (CNN) architecture named DeepColorFASD. Our approach investigates the effect of space colors (RGB, HSV and Y CbCr) on CNN architectures and proposes a fusion based voting method for face anti spoofing. In addition, we also explain the resulting feature maps visualizations. We evaluate our system through an experimental study using CASIA FASD: a well-known face anti spoofing database. The results using this challenging database demonstrate that our solution performs better than recent works as measured by Half Total Error Rate (HTER) and ROC curve.

In this paper, we investigate an efficient palmprint texture modeling method that incorporates a robust analysis based on fusing multiple information. In fact, a single descriptor alone may not achieve a high accuracy in palmprint biometric system. Hence, we propose the fusion of various information features extracted by the different descriptors, such as the fractal and the Multi-fractal techniques which produce a robustness to face the numerous challenging and variation of palmprint in unconstrained environments. To increase the performance of palmprint biometric systems, information fusion is proposed as a key phase in multi-characteristic systems. The obtained information can be combined at different levels, i.e., at the feature level, the score level or the decision level. Nevertheless, the feature level fusion is considered more effective than both the matching score and the classifier decision levels, thanks to a feature vector set which contains more and richer information about the input palmprint image. In order to improve the discriminating texture information, our proposed method extracts the fractal dimension features from the preprocessed palmprint images and fuses them with the Multi-fractal dimension features using the Canonical Correlation Analysis (CCA) incorporating the Linear Discriminant Analysis (LDA) in order to reduce the feature dimensionality for each feature set. To demonstrate the feasibility and effectiveness of our proposed method, we performed the experimental results on two benchmark datasets. These results outperform other well-known state of the art methods and produce promising recognition rates by achieving 96.02% for PolyU-Palmprint database and 97.00% for CASIA-Palmprint database.

In this paper, we propose an effective approach for automatic 4D Facial Expression Recognition (FER). The flow of 3D facial scans is first modeled to capture spatial deformations based on the recently-developed Riemannian approach, namely Dense Scalar Fields (DSF), where registration and comparison of neighboring 3D face frames are jointly led. The deformations are then fed into a temporal filtering based magnification step to amplify the slight facial actions over time. The proposed method allows revealing subtle (hidden) deformations which enhances the performance in classification. We evaluate our approach on the BU-4DFE dataset, and the state-of-art accuracy up to 94.18% is achieved, which is superior to the top one so far reported, clearly demonstrating its effectiveness.

This paper describes a novel framework for computing geodesic paths in shape spaces of spherical surfaces under an elastic Riemannian metric. The novelty lies in defining this Riemannian metric directly on the quotient (shape) space, rather than inheriting it from pre-shape space, and using it to formulate a path energy that measures only the normal components of velocities along the path. In other words, this paper defines and solves for geodesics directly on the shape space and avoids complications resulting from the quotient operation. This comprehensive framework is invariant to arbitrary parameterizations of surfaces along paths, a phenomenon termed as gauge invariance. Additionally, this paper makes a link between different elastic metrics used in the computer science literature on one hand, and the mathematical literature on the other hand, and provides a geometrical interpretation of the terms involved. Examples using real and simulated 3D objects are provided to help illustrate the main ideas.

In this paper, we present an automatic approach for facial expression recognition from 3-D video sequences. In the proposed solution, the 3-D faces are represented by collections of radial curves and a Riemannian shape analysis is applied to effectively quantify the deformations induced by the facial expressions in a given subsequence of 3-D frames. This is obtained from the dense scalar field, which denotes the shooting directions of the geodesic paths constructed between pairs of corresponding radial curves of two faces. As the resulting dense scalar fields show a high dimensionality, Linear Discriminant Analysis (LDA) transformation is applied to the dense feature space. Two methods are then used for classification: 1) 3-D motion extraction with temporal Hidden Markov model (HMM) and 2) mean deformation capturing with random forest. While a dynamic HMM on the features is trained in the first approach, the second one computes mean deformations under a window and applies multiclass random forest. Both of the proposed classification schemes on the scalar fields showed comparable results and outperformed earlier studies on facial expression recognition from 3-D video sequences.

We propose a novel geometric framework for analyzing 3D faces, with the specific goals of comparing, matching, and averaging their shapes. Here we represent facial surfaces by radial curves emanating from the nose tips and use elastic shape analysis of these curves to develop a Riemannian framework for analyzing shapes of full facial surfaces. This representation, along with the elastic Riemannian metric, seems natural for measuring facial deformations and is robust to challenges such as large facial expressions (especially those with open mouths), large pose variations, missing parts, and partial occlusions due to glasses, hair, and so on. This framework is shown to be promising from both--empirical and theoretical--perspectives. In terms of the empirical evaluation, our results match or improve upon the state-of-the-art methods on three prominent databases: FRGCv2, GavabDB, and Bosphorus, each posing a different type of challenge. From a theoretical perspective, this framework allows for formal statistical inferences, such as the estimation of missing facial parts using PCA on tangent spaces and computing average shapes.

In this paper we explore the use of shapes of elastic radial curves to model 3D facial deformations, caused by changes in facial expressions. We represent facial surfaces by indexed collections of radial curves on them, emanating from the nose tips, and compare the facial shapes by comparing the shapes of their corresponding curves. Using a past approach on elastic shape analysis of curves, we obtain an algorithm for comparing facial surfaces. We also introduce a quality control module which allows our approach to be robust to pose variation and missing data. Comparative evaluation using a common experimental setup on GAVAB dataset, considered as the most expression-rich and noise-prone 3D face dataset, shows that our approach outperforms other state-of-the-art approaches.

In this paper we explore the use of shapes of noses for performing partial human biometrics. The basic idea is to represent nasal surfaces using indexed collections of iso-curves, and to analyze shapes of noses by comparing their corresponding curves. We extend past work in Riemannian analysis of shapes of closed curves in R3to obtain a similar Riemannian analysis for nasal surfaces. In particular, we obtain algorithms for computing geodesics, computing statistical means, and stochastic clustering. We demonstrate these ideas in two application contexts : authentication and identification. We evaluate performances on a large database involving 2000 scans from FRGC v2 database, and present a hierarchical organization of nose databases to allow for efficient searches.