Facial expression generation is one of the most challenging and long-sought aspects of character animation, with many interesting applications. The challenging task, traditionally having relied heavily on digital craftspersons, remains yet to be explored. In this article, we introduce a generative framework for generating 3D facial expression sequences (i.e., 4D faces) that can be conditioned on different inputs to animate an arbitrary 3D face mesh. It is composed of two tasks: (1) learning the generative model that is trained over a set of 3D landmark sequences and (2) generating 3D mesh sequences of an input facial mesh driven by the generated landmark sequences. The generative model is based on a Denoising Diffusion Probabilistic Model (DDPM), which has achieved remarkable success in generative tasks of other domains. While it can be trained unconditionally, its reverse process can still be conditioned by various condition signals. This allows us to efficiently develop several downstream tasks involving various conditional generations, by using expression labels, text, partial sequences, or simply a facial geometry. To obtain the full mesh deformation, we then develop a landmark-guided encoder-decoder to apply the geometrical deformation embedded in landmarks on a given facial mesh. Experiments show that our model has learned to generate realistic, quality expressions solely from the dataset of relatively small size, improving over the state-of-the-art methods. Videos and qualitative comparisons with other methods can be found at https://github.com/ZOUKaifeng/4DFM . Code and models will be made available upon acceptance.

Most of previous semi-supervised methods that seek to obtain disentangled representations using variational autoencoders divide the latent representation into two components: the non-interpretable part and the disentangled part that explicitly models the factors of interest. With such models, features associated with high-level factors are not explicitly modeled, and they can either be lost, or at best entangled in the other latent variables, thus leading to bad disentanglement properties. To address this problem, we propose a novel conditional dependency structure where both the labels and their features belong to the latent space. We show using the CelebA dataset that the proposed model can learn meaningful representations, and we provide quantitative and qualitative comparisons with other approaches that show the effectiveness of the proposed method.

We propose an automatic multiorgan segmentation method for 3-D radiological images of different anatomical contents and modalities. The approach is based on a simultaneous multilabel graph cut optimization of location, appearance, and spatial configuration criteria of target structures. Organ location is defined by target-specific probabilistic atlases (PA) constructed from a training dataset using a fast (2+1)D SURF-based multiscale registration method involving a simple four-parameter transformation. PAs are also used to derive target-specific organ appearance models represented as intensity histograms. The spatial configuration prior is derived from shortest-path constraints defined on the adjacency graph of structures. Thorough evaluations on Visceral project benchmarks and training dataset, as well as comparisons with the state-of-the-art confirm that our approach is comparable to and often outperforms similar approaches in multiorgan segmentation, thus proving that the combination of multiple suboptimal but complementary information sources can yield very good performance.

We propose a new shape analysis approach based on the non-local analysis of local shape variations. Our method relies on a novel description of shape variations, called Local Probing Field (LPF), which describes how a local probing operator transforms a pattern onto the shape. By carefully optimizing the position and orientation of each descriptor, we are able to capture shape similarities and gather them into a geometrically relevant dictionary over which the shape decomposes sparsely. This new representation permits to handle shapes with mixed intrinsic dimensionality (e.g., shapes containing both surfaces and curves) and to encode various shape features such as boundaries. Our shape representation has several potential applications; here we demonstrate its efficiency for shape resampling and point set denoising for both synthetic and real data.

Real-time 3D Echocardiography (RT3DE) has been proven to be an accurate tool for left ventricular (LV) volume assessment. However, identification of the LV endocardium remains a challenging task, mainly because of the low tissue/blood contrast of the images combined with typical artifacts. Several semi and fully automatic algorithms have been proposed for segmenting the endocardium in RT3DE data in order to extract relevant clinical indices, but a systematic and fair comparison between such methods has so far been impossible due to the lack of a publicly available common database. Here, we introduce a standardized evaluation framework to reliably evaluate and compare the performance of the algorithms developed to segment the LV border in RT3DE. A database consisting of 45 multivendor cardiac ultrasound recordings acquired at different centers with corresponding reference measurements from three experts are made available. The algorithms from nine research groups were quantitatively evaluated and compared using the proposed online platform. The results showed that the best methods produce promising results with respect to the experts' measurements for the extraction of clinical indices, and that they offer good segmentation precision in terms of mean distance error in the context of the experts' variability range. The platform remains open for new submissions.

We propose a system for automatic positioning of identified landmarks on anatomical structures generated from medical images. We obtain a rigid registration of 3-D shapes using the Iterative Closest Point (ICP) algorithm where a curvature constraint is added as a supplementary dimension in order to improve robustness. A dichotomic scale search and a local rigid registration allow fast local adjustment. We show the effectiveness of the proposed approach by comparing the automatic positioning of anatomical landmarks against the manual positioning by a trained operator.

Abstract Most surfaces, be it from a fine‐art artifact or a mechanical object, are characterized by a strong self‐similarity. This property finds its source in the natural structures of objects but also in the fabrication processes: regularity of the sculpting technique, or machine tool. In this paper, we propose to exploit the self‐similarity of the underlying shapes for compressing point cloud surfaces which can contain millions of points at a very high precision. Our approach locally resamples the point cloud in order to highlight the self‐similarity of the shape, while remaining consistent with the original shape and the scanner precision. It then uses this self‐similarity to create an ad hoc dictionary on which the local neighborhoods will be sparsely represented, thus allowing for a light‐weight representation of the total surface. We demonstrate the validity of our approach on several point clouds from fine‐arts and mechanical objects, as well as a urban scene. In addition, we show that our approach also achieves a filtering of noise whose magnitude is smaller than the scanner precision.

We derive shortest-path constraints from graph models of structure adjacency relations and introduce them in a joint centroidal Voronoi image clustering and Graph Cut multiobject semiautomatic segmentation framework. The vicinity prior model thus defined is a piecewise-constant model incurring multiple levels of penalization capturing the spatial configuration of structures in multiobject segmentation. Qualitative and quantitative analyses and comparison with a Potts prior-based approach and our previous contribution on synthetic, simulated, and real medical images show that the vicinity prior allows for the correct segmentation of distinct structures having identical intensity profiles and improves the precision of segmentation boundary placement while being fairly robust to clustering resolution. The clustering approach we take to simplify images prior to segmentation strikes a good balance between boundary adaptivity and cluster compactness criteria furthermore allowing to control the trade-off. Compared with a direct application of segmentation on voxels, the clustering step improves the overall runtime and memory footprint of the segmentation process up to an order of magnitude without compromising the quality of the result.

This paper describes a framework for the integration of medical image simulators in the Virtual Imaging Platform (VIP). Simulation is widely involved in medical imaging but its availability is hampered by the heterogeneity of software interfaces and the required amount of computing power. To address this, VIP defines a simulation workflow template which transforms object models from the IntermediAte Model Format (IAMF) into native simulator formats and parallelizes the simulation computation. Format conversions, geometrical scene definition and physical parameter generation are covered. The core simulator executables are directly embedded in the simulation workflow, enabling data parallelism exploitation without modifying the simulator. The template is instantiated on simulators of the four main medical imaging modalities, namely Positron Emission Tomography, Ultrasound imaging, Magnetic Resonance Imaging and Computed Tomography. Simulation examples and performance results on the European Grid Infrastructure are shown.

We propose an application of multi-label “Graph Cut” optimization algorithms to the simultaneous segmentation of multiple anatomical structures, initialized via an over-segmentation of the image computed by a fast centroidal Voronoi diagram (CVD) clustering algorithm. With respect to comparable segmentations computed directly on the voxels of image volumes, we demonstrate performance improvements on both execution speed and memory footprint by, at least, an order of magnitude, making it possible to process large volumes on commodity hardware which could not be processed pixel-wise.

Trabecular bone is made of a complex network of plate and rod structures, the proportion of which evolves with age or disease. Thus the identification of trabecular plates and rods is important in understanding bone fragility. We propose a novel approach based on 3D multi-scale adjacency graph analysis of high resolution 3D tomographic images of bone structures. The purpose of this new method is to classify each voxel of the 3D images in two classes: plate and rod voxels. We show that the use of a multi-scale framework is very efficient at detecting rods with different sizes. We present applications of our method to both synthetic images and experimental bone synchrotron radiation micro-CT images.

In this paper, we present a novel method for medial axis approximation based on Constrained Centroidal Voronoi Diagram of discrete data (image, volume). The proposed approach is based on the shape boundary subsampling controled by a clustering approach which generates a Voronoi Diagram well suited for Medial Axis extraction. The resulting Voronoi Diagram is further filtered in order to capture the correct topology of the medial axis. The main contribution of this paper is the integration of both a curvature maps and a distance map for controlling the local variability of Voronoi cells densities. Examples of complex shape processing prove the effectiveness of the proposed approach.

Abstract In this paper, we propose a novel progressive lossless mesh compression algorithm based on Incremental Parametric Refinement, where the connectivity is uncontrolled in a first step, yielding visually pleasing meshes at each resolution level while saving connectivity information compared to previous approaches. The algorithm starts with a coarse version of the original mesh, which is further refined by means of a novel refinement scheme. The mesh refinement is driven by a geometric criterion, in spirit with surface reconstruction algorithms, aiming at generating uniform meshes. The vertices coordinates are also quantized and transmitted in a progressive way, following a geometric criterion, efficiently allocating the bit budget. With this assumption, the generated intermediate meshes tend to exhibit a uniform sampling. The potential discrepancy between the resulting connectivity and the original one is corrected at the end of the algorithm. We provide a proof‐of‐concept implementation, yielding very competitive results compared to previous works in terms of rate/distortion trade‐off.

In this paper, we propose a generic framework for 3D surface remeshing. Based on a metric-driven Discrete Voronoi Diagram construction, our output is an optimized 3D triangular mesh with a user defined vertex budget. Our approach can deal with a wide range of applications, from high quality mesh generation to shape approximation. By using appropriate metric constraints the method generates isotropic or anisotropic elements. Based on point-sampling, our algorithm combines the robustness and theoretical strength of Delaunay criteria with the efficiency of entirely discrete geometry processing . Besides the general described framework, we show experimental results using isotropic, quadric-enhanced isotropic and anisotropic metrics which prove the efficiency of our method on large meshes, for a low computational cost.

In this paper, we present a wavelet-based progressive compression method for 3-D dynamic meshes. Our method exploits the spatial and temporal redundancy. We encode the geometry of base mesh, the wavelet coefficients and the connectivity of each resolution level in order to reduce the spatial redundancy of intra meshes. For inter mesh coding, we encode the differences of geometry of base meshes and of their wavelet coefficients between adjacent frames to reduce the temporal redundancy. Our proposal is based on the wavelet-based multiresolution analysis which uses a perfect reconstruction filter bank and therefore it enables not only progressive representation but also lossless compression. The simulation results demonstrate that the proposed method is applicable to lossy and lossless compression of 3-D dynamic meshes.

Most of the meshes coming from a variety of source are over sampled and exhibit highly irregular connectivity that prevent efficient wavelet analysis. Remeshing comes as a solution to sampling and connectivity optimization with respect to the geometry. The main drawback of existing subdivision remeshing approaches is the lack of control either on the number of triangles or on the geometrical error. We propose a new remeshing scheme that directly produces wavelet-based multiresolution meshes which does not suffer from the previous limitations. We provide a finer control on the resulting number of triangles by using an adaptive subdivision scheme. We also show a lowering of the local resulting geometrical error by using a new curvature-based subdivision criterion.

Abstract We present a novel clustering algorithm for polygonal meshes which approximates a Centroidal Voronoi Diagram construction. The clustering provides an efficient way to construct uniform tessellations, and therefore leads to uniform coarsening of polygonal meshes, when the output triangulation has many fewer elements than the input mesh. The mesh topology is also simplified by the clustering algorithm. Based on a mathematical framework, our algorithm is easy to implement, and has low memory requirements. We demonstrate the efficiency of the proposed scheme by processing several reference meshes having up to 1 million triangles and very high genus within a few minutes on a low‐ end computer.

This paper extends Lounsbery's multiresolution analysis wavelet-based theory for triangular 3D meshes, which can only be applied to regularly subdivided meshes and thus involves a remeshing of the existing 3D data. Based on a new irregular subdivision scheme, the proposed algorithm can be applied directly to irregular meshes, which can be very interesting when one wants to keep the connectivity and geometry of the processed mesh completely unchanged. This is very convenient in CAD (Computer-Assisted Design), when the mesh has attributes such as texture and color information, or when the 3D mesh is used for simulations, and where a different connectivity could lead to simulation errors. The algorithm faces an inverse problem for which a solution is proposed. For each level of resolution, the simplification is processed in order to keep the mesh as regular as possible. In addition, a geometric criterion is used to keep the geometry of the approximations as close as possible to the original mesh. Several examples on various reference meshes are shown to prove the efficiency of our proposal.

This paper proposes a new lossy to lossless progressive compression scheme for triangular meshes, based on a wavelet multiresolution theory for irregular 3D meshes. Although remeshing techniques obtain better compression ratios for geometric compression, this approach can be very effective when one wants to keep the connectivity and geometry of the processed mesh completely unchanged. The simplification is based on the solving of an inverse problem. Optimization of both the connectivity and geometry of the processed mesh improves the approximation quality and the compression ratio of the scheme at each resolution level. We show why this algorithm provides an efficient means of compression for both connectivity and geometry of 3D meshes and it is illustrated by experimental results on various sets of reference meshes, where our algorithm performs better than previously published approaches for both lossless and progressive compression.

Wavemesh is a powerful scheme for 3D triangular mesh processing. In sharp contrast with other approaches using wavelets for mesh compression which apply only to meshes having subdivision connectivity, wavemesh can simplify, approximate, and compress meshes even if they do not respect this constraint. Results clearly indicate that wavemesh outperforms the other approaches in terms of progressive lossless compression. We propose in this paper an improvement for our scheme : higher efficiency for meshes with large subdivision connectivity sets, as shown by experimental results. Also, in some cases, the enhanced wavemesh can even perform better than monoresolution approaches in terms of connectivity compression.

We propose an adaptive wavelets based multiresolution scheme by using harmonic maps for 3D irregular meshes. This approach extends the previous works by M. Eck et al. (see SIGGRAPH '95, p.173-82, 1995) and M. Lounsbery (see "Multiresolution Analysis for Surfaces of Arbitrary Topological Type", PhD thesis, Department of Computer Science and Engineering, University of Washington, p.129, 1994) which have been developed for regular triangular mesh subdivision. First, we construct parameterizations of the original mesh that results in a remesh having a subdivision connectivity for the wavelets decomposition. Next, the local subdivision based multiresolution scheme is presented. Our algorithm represents effectively a region of interest or a region having complex and high curvature geometry by using bi-orthogonal wavelets. Through the computer simulation tested on some example meshes, we show that the proposed method is more effective than the previous regular subdivision methods.

The new MPEG-4 standard will provide superior services for the user, as it introduces the video objects concept. However, there is no generic segmentation technique able to provide the segmentation maps for such a coding system. We propose a novel mesh-based video segmentation and objects tracking algorithm with both robust motion estimation and modeling of motion discontinuities. In addition, occlusions and uncovered regions are well managed. This allows the mesh deformation without the need to process remeshing in motion occlusion regions. The spatial properties of each frame are considered to make the mesh edges fit the image contents and a temporal smoothness constraint is also implemented. The proposed algorithm is able to segment multiple objects at the same time. Some experimental results are shown.

In this paper, we consider the texture coding of quadrangular cells which compose active meshes. Active mesh algorithms which have been developed for region based video coding mainly use interpolation techniques to quantize the cell texture. We propose a new quantization scheme which associates image warping and an adaptive JPEG scheme. Our approach leads to an interesting trade-off between mesh complexity and image quality.

We propose a wavelet based local subdivision of 3-D surfaces, which can be effectively applied to a mesh with complex and high curvature faces. The proposed approach is an extension of M. Lounsbery's works (1994), which has been developed only for regular triangular mesh subdivision. For such a purpose, a bi-orthogonal wavelet basis is constructed by defining a local inner product and using the lifting scheme. As only one filter bank is used for the global and local analysis of the surface and also one filter bank for synthesis, this method is very effective. Through the computer simulation tested on some example mesh, we show that the proposed local subdivision is very effective.

We propose a new subdivision scheme derived from the regular 1:4 face split, allowing analysis of irregularly subdivided triangular meshes by the wavelet transforms. Some experimental results on real medical meshes prove the efficiency of this approach in multiresolution schemes. In addition we show the effectiveness of the proposed algorithm for lossless compression.