Abstract For decades, real‐time skinning has been the cornerstone of character animation in visual effects and games. Despite its importance, the creation of animatable digital assets remains a labor‐intensive manual process. Existing automated tools frequently struggle with intricate geometries, often necessitating significant manual refinement to reach production standards. We present a robust, fully automated method for generating high‐quality skinning weights from a standard mesh and skeleton in a canonical A‐ or T‐pose. Unlike traditional approaches, our framework offers direct sparsity controls to limit bone influences per vertex – a critical requirement for maintaining performance in large‐scale mobile environments. Furthermore, we address the challenge of Level‐of‐Detail (LoD) management by optimizing weights within a continuous spatial volume rather than on discrete vertices. This allows a single optimization pass to be applied seamlessly across multiple asset resolutions and variations. Central to our approach is a novel parameterized family of functions, we call SkinCells. We demonstrate that our method consistently produces stable, high‐quality results even in complex scenarios where standard biharmonic weight computations fail.

Abstract Real‐time hair simulation is a vital component in creating believable virtual avatars, as it provides a sense of immersion and authenticity. The dynamic behavior of hair, such as bouncing or swaying in response to character movements like jumping or walking, plays a significant role in enhancing the overall realism and engagement of virtual experiences. Current methods for simulating hair have been constrained by two primary approaches: highly optimized physics‐based systems and neural methods. However, state‐of‐the‐art neural techniques have been limited to quasi‐static solutions, failing to capture the dynamic behavior of hair. This paper introduces a novel neural method that breaks through these limitations, achieving efficient and stable dynamic hair simulation while outperforming existing approaches. We propose a fully self‐supervised method which can be trained without any manual intervention or artist generated training data allowing the method to be integrated with hair reconstruction methods to enable automatic end‐to‐end methods for avatar reconstruction. Our approach harnesses the power of compact, memory‐efficient neural networks to simulate hair at the strand level, allowing for the simulation of diverse hairstyles without excessive computational resources or memory requirements. We validate the effectiveness of our method through a variety of hairstyle examples, showcasing its potential for real‐world applications.

We introduce a novel approach to reconstruct simulation-ready garments with intricate appearance. Despite recent advancements, existing methods often struggle to balance the need for accurate garment reconstruction with the ability to generalize to new poses and body shapes or require large amounts of data to achieve this. In contrast, our method only requires a multi-view capture of a single static frame. We represent garments as hybrid mesh-embedded 3D Gaussian splats, where the Gaussians capture near-field shading and high-frequency details, while the mesh encodes far-field albedo and optimized reflectance parameters. We achieve novel pose generalization by exploiting the mesh from our hybrid approach, enabling physics-based simulation and surface rendering techniques, while also capturing fine details with Gaussians that accurately reconstruct garment details. Our optimized garments can be used for simulating garments on novel poses, and garment relighting. Project page: phys-gaussian-cloth.github.io.

Realistic hair motion is crucial for high-quality avatars, but it is often limited by the computational resources available for real-time applications. To address this challenge, we propose a novel neural approach to predict physically plausible hair deformations that generalizes to various body poses, shapes, and hairstyles. Our model is trained using a self-supervised loss, eliminating the need for expensive data generation and storage. We demonstrate our method’s effectiveness through numerous results across a wide range of pose and shape variations, showcasing its robust generalization capabilities and temporally smooth results. Our approach is highly suitable for real-time applications with an inference time of only a few milliseconds on consumer hardware and its ability to scale to predicting the drape of 1000 grooms in 0.3 seconds.

The realism of digital avatars is crucial in enabling telepresence applications with self-expression and customization. While physical simulations can produce realistic motions for clothed humans, they require high-quality garment assets with associated physical parameters for cloth simulations. However, manually creating these assets and calibrating their parameters is labor-intensive and requires specialized expertise. Current methods focus on reconstructing geometry, but don't generate complete assets for physics-based applications. To address this gap, we propose DiffAvatar, a novel approach that performs body and garment co-optimization using differentiable simulation. By integrating physical simulation into the optimization loop and accounting for the complex nonlinear behavior of cloth and its intricate interaction with the body, our framework recovers body and garment geometry and extracts important material parameters in a physically plausible way. Our experiments demonstrate that our approach generates realistic clothing and body shape suitable for downstream applications. We provide additional insights and results on our webpage: people. csail. mit. edu/liyifei/publication/diffavatar.

Virtual garment simulation has become increasingly important with applications in garment design and virtual try-on. However, reproducing garments faithfully remains a cumbersome process. We propose an end-to-end forward pipeline for estimating parameters of shell material models corresponding to real fabrics with minimal input. In contrast to prior work that relies on complex and often expensive capture systems, our method determines yarn model parameters from Young’s moduli determined during standard yarn stretch tests. We use an extended homogenization method to match yarn-level and shell-level hyperelastic energies with respect to a range of surface deformations represented by the first and second fundamental forms, including anisotropic bending. We optimize the parameters of a shell material model involving uncoupled bending and membrane energies. This allows the simulated shell model to exhibit deformation modes motivated by yarn-level physics in real fabrics. Finally, we validate our results with quantitative and visual comparisons against real world fabrics through stretch tests and drape experiments. Using the homogenized parameters, the shell models are capable of capturing the characteristics of underlying yarn patterns and exhibiting distinct behaviors for different yarn materials.

We present Virtual Elastic Objects (VEOs): virtual objects that not only look like their real-world counterparts but also behave like them, even when subject to novel interactions. Achieving this presents multiple challenges: not only do objects have to be captured including the physical forces acting on them, then faithfully reconstructed and rendered, but also plausible material parameters found and simulated. To create VEOs, we built a multi-view capture system that captures objects under the influence of a compressed air stream. Building on recent advances in model-free, dynamic Neural Radiance Fields, we reconstruct the objects and corresponding deformation fields. We propose to use a differentiable, particle-based simulator to use these deformation fields to find representative material parameters, which enable us to run new simulations. To render simulated objects, we devise a method for integrating the simulation results with Neural Radiance Fields. The resulting method is applicable to a wide range of scenarios: it can handle objects composed of inhomogeneous material, with very different shapes, and it can simulate interactions with other virtual objects. We present our results using a newly collected dataset of 12 objects under a variety of force fields, which will be made available upon publication.

We propose a new model and algorithm to capture the high-definition statics of thin shells via coarse meshes. This model predicts global, fine-scale wrinkling at frequencies much higher than the resolution of the coarse mesh; moreover, it is grounded in the geometric analysis of elasticity, and does not require manual guidance, a corpus of training examples, nor tuning of ad hoc parameters. We first approximate the coarse shape of the shell using tension field theory, in which material forces do not resist compression. We then augment this base mesh with wrinkles, parameterized by an amplitude and phase field that we solve for over the base mesh, which together characterize the geometry of the wrinkles. We validate our approach against both physical experiments and numerical simulations, and we show that our algorithm produces wrinkles qualitatively similar to those predicted by traditional shell solvers requiring orders of magnitude more degrees of freedom.

We present a physically accurate low-order elastic shell model that incorporates active material response to dynamically changing stimuli such as heat, moisture, and growth. Our continuous formulation of the geometrically non-linear elastic energy derives from the principles of differential geometry, and as such naturally incorporates shell thickness, non-zero rest curvature, and physical material properties. By modeling the environmental stimulus as local, dynamic changes in the rest metric of the material, we are able to solve for the corresponding shape changes by integrating the equations of motions given this non-Euclidean rest state. We present models for differential growth and shrinking due to moisture and temperature gradients along and across the surface, and incorporate anisotropic growth by defining an intrinsic machine direction within the material. Comparisons with experiments and volumetric finite elements show that our simulations achieve excellent qualitative and quantitative agreement. By combining the reduced-order shell theory with appropriate physical models, our approach accurately captures all the physical phenomena while avoiding expensive volumetric discretization of the shell volume.

It is known that ECG signal are strongly affected by the chest motion of respiration. We utilize this property and derive a respiration related signal with respect to the change of R wave area, and then match the derived signal to the time axis. We discovered that during the apnea section, a lower frequency component compared to the respiration frequency is observed. With the presence and absence of this lower frequency modulation on the ECG signal, we propose a real-time detection method to differentiate the apnea section and normal breathing section in the duration of sleep.