Abstract. In this paper, we address the problem of registering multiple point clouds corrupted with high anisotropic localization noise. Our approach follows the widely used framework of Gaussian mixture model (GMM) reconstruction with an expectation-maximization (EM) algorithm. Existing methods are based on an implicit assumption of spatially invariant isotropic Gaussian noise. However, this assumption is violated in practice in applications such as single molecule localization microscopy (SMLM). To address this issue, we propose to introduce an explicit localization noise model that decouples shape modeling with the GMM from noise handling. We use this model for multiview point cloud registration by designing an EM algorithm that considers noise-free data as a latent variable, with closed-form solutions at each EM step. The first advantage of our approach is to handle spatially varying and anisotropic Gaussian noise. The second advantage is to leverage the explicit noise model to impose prior knowledge about the noise that may be available from physical sensors. We show on various simulated data that our noise handling strategy improves significantly the robustness to high levels of anisotropic noise. We also demonstrate the performance of our method on real SMLM data.

Point cloud rigid registration is a fundamental problem in 3D computer vision. In the multiview case, we aim to find a set of 6D poses to align a set of objects. Methods based on pairwise registration rely on a subsequent synchronization algorithm, which makes them poorly scalable with the number of views. Generative approaches overcome this limitation, but are based on Gaussian Mixture Models and use an Expectation-Maximization algorithm. Hence, they are not well suited to handle large transformations. Moreover, most existing methods cannot handle high levels of degradations. In this paper, we introduce POLAR (POint cloud LAtent Registration), a multiview registration method able to efficiently deal with a large number of views, while being robust to a high level of degradations and large initial angles. To achieve this, we transpose the registration problem into the latent space of a pretrained autoencoder, design a loss taking degradations into account, and develop an efficient multistart optimization strategy. Our proposed method significantly outperforms state-of-the-art approaches on synthetic and real data. POLAR is available at github.com/pypolar/polar or as a standalone package which can be installed with pip install polaregistration.

Facial expression generation is one of the most challenging and long-sought aspects of character animation, with many interesting applications. The challenging task, traditionally having relied heavily on digital craftspersons, remains yet to be explored. In this article, we introduce a generative framework for generating 3D facial expression sequences (i.e., 4D faces) that can be conditioned on different inputs to animate an arbitrary 3D face mesh. It is composed of two tasks: (1) learning the generative model that is trained over a set of 3D landmark sequences and (2) generating 3D mesh sequences of an input facial mesh driven by the generated landmark sequences. The generative model is based on a Denoising Diffusion Probabilistic Model (DDPM), which has achieved remarkable success in generative tasks of other domains. While it can be trained unconditionally, its reverse process can still be conditioned by various condition signals. This allows us to efficiently develop several downstream tasks involving various conditional generations, by using expression labels, text, partial sequences, or simply a facial geometry. To obtain the full mesh deformation, we then develop a landmark-guided encoder-decoder to apply the geometrical deformation embedded in landmarks on a given facial mesh. Experiments show that our model has learned to generate realistic, quality expressions solely from the dataset of relatively small size, improving over the state-of-the-art methods. Videos and qualitative comparisons with other methods can be found at https://github.com/ZOUKaifeng/4DFM . Code and models will be made available upon acceptance.

Most of previous semi-supervised methods that seek to obtain disentangled representations using variational autoencoders divide the latent representation into two components: the non-interpretable part and the disentangled part that explicitly models the factors of interest. With such models, features associated with high-level factors are not explicitly modeled, and they can either be lost, or at best entangled in the other latent variables, thus leading to bad disentanglement properties. To address this problem, we propose a novel conditional dependency structure where both the labels and their features belong to the latent space. We show using the CelebA dataset that the proposed model can learn meaningful representations, and we provide quantitative and qualitative comparisons with other approaches that show the effectiveness of the proposed method.

This paper compares several linear and non-linear multivariate models for the detection of abnormal patterns in neuroimaging data, when comparing a single subject to a normal control group. The proposed methods learn the manifold spanned by the normal controls using non-linear dimension reduction techniques. The image of a subject is projected on the control group manifold either via a standard projection or through an embedding/reconstruction scheme. A comparison of the reconstruction with the subject's original neuroimaging data allows for the detection of abnormal patterns by way of statistical tests on the residuals. The different abnormality detection methods are assessed on synthetic data and real (MRI) neuroimaging data. The importance of non-linear modeling of the manifold in the reduced-dimension subspace is highlighted, as well as robustness to large abnormalities.

The estimation of one-to-one mappings is one of the most intensively studied topics in the research field of nonrigid registration. Although the computation of such mappings can be now accurately and efficiently performed, the solutions for using them in the context of binary image deformation is much less satisfactory. In particular, warping a binary image with such transformations may alter its discrete topological properties if common resampling strategies are considered. In order to deal with this issue, this paper proposes a method for warping such images according to continuous and bijective mappings while preserving their discrete topological properties (i.e., their homotopy type). Results obtained in the context of the atlas-based segmentation of complex anatomical structures highlight the advantages of the proposed approach.

This paper presents a novel statistical approach for the modeling and analysis of structured random processes observed through multiple event sequences: the hidden Markov multiple event sequence model (HMMESM). This model accounts for several features of these processes: (i) the hidden-observable aspect of the event sequences to be analyzed, (ii) the multiplicity of the observed event sequences, (iii) the non stationary, time-localized character of their events, (iv) the redundancy, complementarity, and strong asynchrony that exist between events across sequences. A first application of this model in functional MRI (fMRI) brain mapping is presented. The developed method shows high robustness to noise and variability of the active fMRI signals

In this paper, a novel functional magnetic resonance imaging (fMRI) brain mapping method is presented within the statistical modeling framework of hidden semi-Markov event sequence models (HSMESMs). Neural activation detection is formulated at the voxel level in terms of time coupling between the sequence of hemodynamic response onsets (HROs) observed in the fMRI signal, and an HSMESM of the hidden sequence of task-induced neural activations. The sequence of HRO events is derived from a continuous wavelet transform (CWT) of the fMRI signal. The brain activation HSMESM is built from the timing information of the input stimulation protocol. The rich mathematical framework of HSMESMs makes these models an effective and versatile approach for fMRI data analysis. Solving for the HSMESM Evaluation and Learning problems enables the model to automatically detect neural activation embedded in a given set of fMRI signals, without requiring any template basis function or prior shape assumption for the fMRI response. Solving for the HSMESM Decoding problem allows to enrich brain mapping with activation lag mapping, activation mode visualizing, and hemodynamic response function analysis. Activation detection results obtained on synthetic and real epoch-related fMRI data demonstrate the superiority of the HSMESM mapping method with respect to a real application case of the statistical parametric mapping (SPM) approach. In addition, the HSMESM mapping method appears clearly insensitive to timing variations of the hemodynamic response, and exhibits low sensitivity to fluctuations of its shape.

Due to the piecewise stationarity assumption required for the observable process of a hidden Markov chain, the application of hidden Markov models (HMMs) to the analysis of event-based random processes remains intricate. For such processes, a new class of HMMs is proposed: the hidden semi-Markov event sequence model (HSMESM). In a HSMESM, the observable process is no more considered as segmental in nature but issued from a detection-characterization preprocessing step. The standard markovian formalism is adapted accordingly. Results obtained in functional MRI sequence analysis validate this novel statistical modeling approach while opening new perspectives in detection-recognition of event-based random processes.