A longstanding goal in computer vision is to model motions from videos, while the representations behind motions, i.e. the invisible physical interactions that cause objects to deform and move, remain largely unexplored. In this paper, we study how to recover the invisible forces from visual observations, e.g., estimating the wind field by observing a leaf falling to the ground. Our key innovation is an end-to-end differentiable inverse graphics framework, which jointly models object geometry, physical properties, and interactions directly from videos. Through backpropagation, our approach enables the recovery of force representations from object motions. We validate our method on both synthetic and real-world scenarios, and the results demonstrate its ability to infer plausible force fields from videos. Furthermore, we show the potential applications of our approach, including physics-based video generation and editing. We hope our approach sheds light on understanding and modeling the physical process behind pixels, bridging the gap between vision and physics. Please check more video results in our project page https://chaoren2357.github.io/seeingthewind/ .

We give a method to simulate large deformations of 3D elastic rods under arbitrary nonlinear isotropic 3D solid materials. Rod elastic energies in existing graphics literature are derived from volumetric models under the small-strain linearization assumptions. While the resulting equations can and are commonly applied to large deformations, the material modeling has been limited to a single material, namely linear Hooke law. Starting from any 3D solid nonlinear isotropic elastic energy density function ψ , we derive our rod elastic energy by subjecting the 3D solid volumetric material to the limit process whereby rod thickness is decreased to zero. This enables us to explain rod stretching, bending and twisting in a unified model. Care must be taken to adequately model cross-sectional in-plane and out-of-plane deformations. Our key insight is to compute the three cross-sectional deformation modes corresponding to bending (in the two directions) and twisting, using linear theory. Then, given any ψ , we use these modes to derive an analytical formula for a 5D "macroscopic" large-deformation rod elastic energy function of the local longitudinal stretch, radial scaling, the two bending curvatures and torsion. Our model matches linear theory for small deformations, including cross-sectional shrinkage due to Poisson's effect, and produces correct bending and torsional constants. Our experiments demonstrate that our energy closely matches volumetric FEM even under large stretches and curvatures, whereas commonly used methods in graphics deviate from it. We also compare to closely related work from mechanics literature; we give an explicit expansion of all energy terms in terms of the rod cross-section diameter, allowing independent adjustment of stretching, bending and twisting. Finally, we observe an inherent limitation in the ability of rod models to control nonlinear bendability and twistability. We propose to "relax" rod physics to more easily control nonlinear bending and twisting in computer graphics applications.

We propose a coupled mesh-adaptation model and physical simulation algorithm to jointly generate, per timestep, optimal adaptive remeshings and implicit solutions for the simulation of frictionally contacting elastodynamics. To do so, we begin with Ferguson et al.'s [2023] recently developed in-timestep remeshing (ITR) framework, which proposes an Incremental Potential based objective for mesh refinement, and a corresponding, locally greedy remeshing algorithm to minimize it. While this initial ITR framework demonstrates significant improvements, its greedy remeshing does not generate optimal meshes, and so does not converge to improving physical solutions with increasing mesh resolution. In practice, due to lack of optimality, the original ITR framework can and will find mesh and state solutions with unnecessarily low-quality geometries and corresponding physical solution artifacts. At the same time, we also identify additional fundamental challenges to adaptive simulation in terms of both ITR's original remeshing objective and its corresponding optimization problem formulation. In this work, in order to extend the ITR framework to high-quality, optimal in-timestep remeshing, we first construct a new remeshing objective function built from simple, yet critical, updates to the Incremental Potential energy, and a corresponding constrained model problem, whose minimizers provide locally optimal remeshings for physical problems. We then propose a new in-timestep remeshing optimization that jointly solves, per-timestep, for a new locally optimal remeshing and the next physical state defined upon it. To evaluate and demonstrate our extension of the ITR framework, we apply it to the optimal r-adaptive ITR simulation of frictionally contacting elasto-dynamics and statics. To enable r-adaptivity we additionally propose a new numerical method to robustly compute derivatives of the L 2 -projection operator necessary for optimal mesh-to-mesh state mappings within solves, a constraint model to enable on-boundary node adaptivity, and an efficient Newton-type optimization method for practically solving each per-timestep r-adaptive ITR solution. We extensively evaluate our method on challenging large-deformation and frictionally contacting scenarios. Here we observe optimal r-adaptivity captures comparable and better accuracy than unadapted meshes orders-of-magnitude larger, with corresponding significant advantages in both computation speedup and decrease in memory usage.

We present a hard-real-time multi-resolution mesh shape deformation technique for skeleton-driven soft-body characters. Producing mesh deformations at multiple levels of detail is very important in many applications in computer graphics. Our work targets applications where the multi-resolution shapes must be generated at fast speeds ("hard-real-time", e.g., a few milliseconds at most and preferably under 1 millisecond), as commonly needed in computer games, virtual reality and Metaverse applications. We assume that the character mesh is driven by a skeleton, and that high-quality character shapes are available in a set of training poses originating from a high-quality (slow) rig such as volumetric FEM simulation. Our method combines multi-resolution analysis, mesh partition of unity, and neural networks, to learn the pre-skinning shape deformations in an arbitrary character pose. Combined with linear blend skinning, this makes it possible to reconstruct the training shapes, as well as interpolate and extrapolate them. Crucially, we simultaneously achieve this at hard real-time rates and at multiple mesh resolution levels. Our technique makes it possible to trade deformation quality for memory and computation speed, to accommodate the strict requirements of modern real-time systems. Furthermore, we propose memory layout and code improvements to boost computation speeds. Previous methods for realtime approximations of quality shape deformations did not focus on hard real-time, or did not investigate the multi-resolution aspect of the problem. Compared to a "naive" approach of separately processing each hierarchical level of detail, our method offers a substantial memory reduction as well as computational speedups. It also makes it possible to construct the shape progressively level by level and interrupt the computation at any time, enabling graceful degradation of the deformation detail. We demonstrate our technique on several examples, including a stylized human character, human hands, and an inverse-kinematics-driven quadruped animal.

Modeling arbitrarily large deformations of surfaces smoothly embedded in three-dimensional space is challenging. We give a new method to represent surfaces undergoing large spatially varying rotations and strains, based on differential geometry, and surface first and second fundamental forms. Methods that penalize the difference between the current shape and the rest shape produce sharp spikes under large strains, and variational methods produce wiggles, whereas our method naturally supports large strains and rotations without any special treatment. For stable and smooth results, we demonstrate that the deformed surface has to locally satisfy compatibility conditions (Gauss-Codazzi equations) on the first and second fundamental forms. We then give a method to locally modify the surface first and second fundamental forms in a compatible way. We use those fundamental forms to define surface plastic deformations, and finally recover output surface vertex positions by minimizing the surface elastic energy under the plastic deformations. We demonstrate that our method makes it possible to smoothly deform triangle meshes to large spatially varying strains and rotations, while meeting user constraints.

Capturing material properties of real-world elastic solids is both challenging and highly relevant to many applications in computer graphics, robotics and related fields. We give a non-intrusive, in-situ and inexpensive approach to measure the nonlinear elastic energy density function of man-made materials and biological tissues. We poke the elastic object with 3d-printed rigid cylinders of known radii, and use a precision force meter to record the contact force as a function of the indentation depth, which we measure using a force meter stand, or a novel unconstrained laser setup. We model the 3D elastic solid using the Finite Element Method (FEM), and elastic energy using a compressible Valanis-Landel material that generalizes Neo-Hookean materials by permitting arbitrary tensile behavior under large deformations. We then use optimization to fit the nonlinear isotropic elastic energy so that the FEM contact forces and indentations match their measured real-world counterparts. Because we use carefully designed cubic splines, our materials are accurate in a large range of stretches and robust to inversions, and are therefore "animation-ready" for computer graphics applications. We demonstrate how to exploit radial symmetry to convert the 3D elastostatic contact problem to the mathematically equivalent 2D problem, which vastly accelerates optimization. We also greatly improve the theory and robustness of stretch-based elastic materials, by giving a simple and elegant formula to compute the tangent stiffness matrix, with rigorous proofs and singularity handling. We also contribute the observation that volume compressibility can be estimated by poking with rigid cylinders of different radii, which avoids optical cameras and greatly simplifies experiments. We validate our method by performing full 3D simulations using the optimized materials and confirming that they match real-world forces, indentations and real deformed 3D shapes. We also validate it using a "Shore 00" durometer, a standard device for measuring material hardness.

Kirchhoff-Love shells are commonly used in many branches of engineering, including in computer graphics, but have so far been simulated only under limited nonlinear material options. We derive the Kirchhoff-Love thin-shell mechanical energy for an arbitrary 3D volumetric hyperelastic material, including isotropic materials, anisotropic materials, and materials whereby the energy includes both even and odd powers of the principal stretches. We do this by starting with any 3D hyperelastic material, and then analytically computing the corresponding thin-shell energy limit. This explicitly identifies and separates in-plane stretching and bending terms, and avoids numerical quadrature. Thus, in-plane stretching and bending are shown to originate from one and the same process (volumetric elasticity of thin objects), as opposed to from two separate processes as done traditionally in cloth simulation. Because we can simulate materials that include both even and odd powers of stretches, we can accommodate standard mesh distortion energies previously employed for 3D solid simulations, such as Symmetric ARAP and Co-rotational materials. We relate the terms of our energy to those of prior work on Kirchhoff-Love thin-shells in computer graphics that assumed small in-plane stretches, and demonstrate the visual difference due to the presence of our exact stretching and bending terms. Furthermore, our formulation allows us to categorize all distinct hyperelastic Kirchhoff-Love thin-shell energies. Specifically, we prove that for Kirchhoff-Love thin-shells, the space of all hyperelastic materials collapses to two-dimensional hyperelastic materials. This observation enables us to create an interface for the design of thin-shell Kirchhoff-Love mechanical energies, which in turn enables us to create thin-shell materials that exhibit arbitrary stiffness profiles under large deformations.

Precision modeling of the hand internal musculoskeletal anatomy has been largely limited to individual poses, and has not been connected into continuous volumetric motion of the hand anatomy actuating across the hand's entire range of motion. This is for a good reason, as hand anatomy and its motion are extremely complex and cannot be predicted merely from the anatomy in a single pose. We give a method to simulate the volumetric shape of hand's musculoskeletal organs to any pose in the hand's range of motion, producing external hand shapes and internal organ shapes that match ground truth optical scans and medical images (MRI) in multiple scanned poses. We achieve this by combining MRI images in multiple hand poses with FEM multibody nonlinear elastoplastic simulation. Our system models bones, muscles, tendons, joint ligaments and fat as separate volumetric organs that mechanically interact through contact and attachments, and whose shape matches medical images (MRI) in the MRI-scanned hand poses. The match to MRI is achieved by incorporating pose-space deformation and plastic strains into the simulation. We show how to do this in a non-intrusive manner that still retains all the simulation benefits, namely the ability to prescribe realistic material properties, generalize to arbitrary poses, preserve volume and obey contacts and attachments. We use our method to produce volumetric renders of the internal anatomy of the human hand in motion, and to compute and render highly realistic hand surface shapes. We evaluate our method by comparing it to optical scans, and demonstrate that we qualitatively and quantitatively substantially decrease the error compared to previous work. We test our method on five complex hand sequences, generated either using keyframe animation or performance animation using modern hand tracking techniques.

Humans routinely sit or lean against supporting surfaces and it is important to shape these surfaces to be comfortable and ergonomic. We give a method to design the geometric shape of rigid supporting surfaces to maximize the ergonomics of physically based contact between the surface and a deformable human. We model the soft deformable human using a layer of FEM deformable tissue surrounding a rigid core, with measured realistic elastic material properties, and large-deformation nonlinear analysis. We define a novel cost function to measure the ergonomics of contact between the human and the supporting surface. We give a stable and computationally efficient contact model that is differentiable with respect to the supporting surface shape. This makes it possible to optimize our ergonomic cost function using gradient-based optimizers. Our optimizer produces supporting surfaces superior to prior work on ergonomic shape design. Our examples include furniture, apparel and tools. We also validate our results by scanning a real human subject's foot and optimizing a shoe sole shape to maximize foot contact ergonomics. We 3D-print the optimized shoe sole, measure contact pressure using pressure sensors, and demonstrate that the real unoptimized and optimized pressure distributions qualitatively match those predicted by our simulation.

Fast and light-weight methods for animating 3D characters are desirable in various applications such as computer games. We present a learning-based approach to enhance skinning-based animations of 3D characters with vivid secondary motion effects. We design a neural network that encodes each local patch of a character simulation mesh where the edges implicitly encode the internal forces between the neighboring vertices. The network emulates the ordinary differential equations of the character dynamics, predicting new vertex positions from the current accelerations, velocities and positions. Being a local method, our network is independent of the mesh topology and generalizes to arbitrarily shaped 3D character meshes at test time. We further represent per-vertex constraints and material properties such as stiffness, enabling us to easily adjust the dynamics in different parts of the mesh. We evaluate our method on various character meshes and complex motion sequences. Our method can be over 30 times more efficient than ground-truth physically based simulation, and outperforms alternative solutions that provide fast approximations.

Composites are increasingly becoming a material of choice in the aerospace and automotive industries. Currently, many composite parts are produced by manually laying up sheets on complex molds. Composite sheet layup requires executing two main tasks: (1) grasping a sheet and (2) draping it on the mold. Automating the layup process requires automation of these two tasks. This paper is focused on the automation of the grasping task using robots. This requires an automated generation of grasp plans to enable robots to hold the sheet during the draping process. We present a simulation-based approach for determining robot grasp locations on the composite sheets. We also present an intervention controller that uses a real-time sheet tracking system during plan execution and can prevent failures. We demonstrate the performance of the developed system using a large complex part.

We present a method for modeling solid objects undergoing large spatially varying and/or anisotropic strains, and use it to reconstruct human anatomy from medical images. Our novel shape deformation method uses plastic strains and the finite element method to successfully model shapes undergoing large and/or anisotropic strains, specified by sparse point constraints on the boundary of the object. We extensively compare our method to standard second-order shape deformation methods, variational methods, and surface-based methods, and demonstrate that our method avoids the spikiness, wiggliness, and other artifacts of previous methods. We demonstrate how to perform such shape deformation both for attached and un-attached (“free flying”) objects, using a novel method to solve linear systems with singular matrices with a known nullspace. Although our method is applicable to general large-strain shape deformation modeling, we use it to create personalized 3D triangle and volumetric meshes of human organs, based on magnetic resonance imaging or computed tomography scans. Given a medically accurate anatomy template of a generic individual, we optimize the geometry of the organ to match the magnetic resonance imaging or computed tomography scan of a specific individual. Our examples include human hand muscles, a liver, a hip bone, and a gluteus medius muscle (“hip abductor”).

Real-time deformable object simulation is important in interactive applications such as games and virtual reality. One common approach to achieve speed is to employ model reduction, a technique whereby the equations of motion of a deformable object are projected to a suitable low-dimensional space. Improving the real-time performance of model-reduced systems has been the subject of much research. While modern GPUs play an important role in real-time simulation and parallel computing, existing model reduction systems typically utilize CPUs and seldom employ GPUs. We give a method to efficiently employ GPUs for vertex position computation in model-reduced simulations. Our CUDA-based algorithm gives a substantial speedup compared to a CPU implementation, thanks to our system architecture that employs a memory layout friendly to GPU memory, reduces the communication between the CPU and GPU, and enables the CPU and GPU to work in parallel.

Abstract Physically based simulation is often combined with geometric mesh animation to add realistic soft‐body dynamics to virtual characters. This is commonly done using constraint‐based simulation whereby a soft‐tissue simulation is constrained to geometric animation of a subpart (or otherwise proxy representation) of the character. We observe that standard constraint‐based simulation suffers from an important flaw that limits the expressiveness of soft‐body dynamics. Namely, under correct physics, the frequency and amplitude of soft‐tissue dynamics arising from constraints (“inertial amplitude”) are coupled, and cannot be adjusted independently merely by adjusting the material properties of the model. This means that the space of physically based simulations is inherently limited and cannot capture all effects typically expected by computer animators. For example, animators need the ability to adjust the frequency, inertial amplitude, gravity sag and damping properties of the virtual character, independently from each other, as these are the primary visual characteristics of the soft‐tissue dynamics. We demonstrate that independence can be achieved by transforming the equations of motion into a non‐inertial reference coordinate frame, then scaling the resulting inertial forces, and then converting the equations of motion back to the inertial frame. Such scaling of inertia makes it possible for the animator to set the character's inertial amplitude independently from frequency. We also provide exact controls for the amount of character's gravity sag, and the damping properties. In our examples, we use linear blend skinning and pose‐space deformation for geometric mesh animation, and the Finite Element Method for soft‐body constrained simulation; but our idea of scaling inertial forces is general and applicable to other animation and simulation methods. We demonstrate our technique on several character examples.

We demonstrate how to acquire complete human hand bone anatomy (meshes) in multiple poses using magnetic resonance imaging (MRI). Such acquisition was previously difficult because MRI scans must be long for high-precision results (over 10 minutes) and because humans cannot hold the hand perfectly still in non-trivial and badly supported poses. We invent a manufacturing process whereby we use lifecasting materials commonly employed in film special effects industry to generate hand molds, personalized to the subject, and to each pose. These molds are both ergonomic and encasing, and they stabilize the hand during scanning. We also demonstrate how to efficiently segment the MRI scans into individual bone meshes in all poses, and how to correspond each bone's mesh to same mesh connectivity across all poses. Next, we interpolate and extrapolate the MRI-acquired bone meshes to the entire range of motion of the hand, producing an accurate data-driven animation-ready rig for bone meshes. We also demonstrate how to acquire not just bone geometry (using MRI) in each pose, but also a matching highly accurate surface geometry (using optical scanners) in each pose, modeling skin pores and wrinkles. We also give a soft tissue Finite Element Method simulation "rig", consisting of novel tet meshing for stability at the joints, spatially varying geometric and material detail, and quality constraints to the acquired skeleton kinematic rig. Given an animation sequence of hand joint angles, our FEM soft tissue rig produces quality hand surface shapes in arbitrary poses in the hand range of motion. Our results qualitatively reproduce important features seen in the photographs of the subject's hand, such as similar overall organic shape and fold formation.

Self-intersecting, or nearly self-intersecting, meshes are commonly found in 2D and 3D computer graphics practice. Self-intersections occur, for example, in the process of artist manual work, as a by-product of procedural methods for mesh generation, or due to modeling errors introduced by scanning equipment. If the space bounded by such inputs is meshed naively, the resulting mesh joins ("glues") self-overlapping parts, precluding efficient further modeling and animation of the underlying geometry. Similarly, near self-intersections force the simulation algorithm to employ an unnecessarily detailed mesh to separate the nearly self-intersecting regions. Our work addresses both of these challenges, by giving an algorithm to generate an "un-glued" simulation mesh, of arbitrary user-chosen resolution, that properly accounts for self-intersections and near self-intersections. In order to achieve this result, we study the mathematical concept of immersion, and give a deterministic and constructive algorithm to determine if the input self-intersecting triangle mesh is the boundary of an immersion. For near self-intersections, we give a robust algorithm to properly duplicate mesh elements and correctly embed the underlying geometry into the mesh element copies. Both the self-intersections and near self-intersections are combined into one algorithm that permits successful meshing at arbitrary resolution. Applications of our work include volumetric shape editing, physically based simulation and animation, and volumetric weight and geodesic distance computation on self-intersecting inputs.

Botanical simulation plays an important role in many fields including visual effects, games and virtual reality. Previous plant simulation research has focused on computing physically based motion, under the assumption that the material properties are known. It is too tedious and impractical to manually set the spatially-varying material properties of complex trees. In this paper, we give a method to set the mass density, stiffness and damping properties of individual tree components (branches and leaves) using a small number of intuitive parameters. Our method is rooted in plant biomechanics literature and builds upon power laws observed in real botanical systems. We demonstrate our materials by simulating them using offline and model-reduced FEM simulators. Our parameters can be tuned directly by artists; but we also give a technique to infer the parameters from ground truth videos of real trees. Our materials produce tree animations that look much more similar to real trees than previous methods, as evidenced by our user study and experiments.

To date, material modeling in physically based computer animation has largely focused on mass and stiffness material properties. However, deformation dynamics is largely affected also by the damping properties. In this paper, we propose an interactive design method for nonlinear isotropic and anisotropic damping of complex three-dimensional solids simulated using the Finite Element Method (FEM). We first give a damping design method and interface whereby the user can set the damping properties so that motion aligned with each of a few chosen example deformations is damped by an independently prescribed amount, whereas the rest of the deformation space follows standard Rayleigh damping, or any viscous damping. Next, we demonstrate how to design nonlinear damping that depends on the magnitude of the deformation along each example deformation, by editing a single spline curve for each example deformation. Our user interface enables an art-directed and intuitive approach to controlling damping in solid simulations. We mathematically prove that our nonlinear anisotropic damping generalizes the frequency-dependent Caughey damping model, when starting from the Rayleigh damping. Finally, we give an inverse design method whereby the damping curve parameters can be inferred automatically from high-level user input, such as the amount of amplitude loss in one oscillation cycle along each of the chosen example deformations. To minimize numerical damping for implicit integration, we introduce an accurate and stable implicit integrator, which removes spurious high-frequency oscillations while only introducing a minimal amount of numerical damping. Our damping can generate effects not possible with previous methods, such as controllable nonlinear decaying envelopes whereby large deformations are damped faster or slower than small deformations, and damping anisotropic effects. We also fit our damping to videos of real-world objects undergoing large deformations, capturing their nonlinear and anisotropic damping dynamics.

We present a system to combine arbitrary triangle mesh animations with physically based Finite Element Method (FEM) simulation, enabling control over the combination both in space and time. The input is a triangle mesh animation obtained using any method, such as keyframed animation, character rigging, 3D scanning, or geometric shape modeling. The input may be non-physical, crude or even incomplete. The user provides weights, specified using a minimal user interface, for how much physically based simulation should be allowed to modify the animation in any region of the model, and in time. Our system then computes a physically-based animation that is constrained to the input animation to the amount prescribed by these weights. This permits smoothly turning physics on and off over space and time, making it possible for the output to strictly follow the input, to evolve purely based on physically based simulation, and anything in between. Achieving such results requires a careful combination of several system components. We propose and analyze these components, including proper automatic creation of simulation meshes (even for non-manifold and self-colliding undeformed triangle meshes), converting triangle mesh animations into animations of the simulation mesh, and resolving collisions and self-collisions while following the input.

We enrich character animations with secondary soft-tissue Finite Element Method (FEM) dynamics computed under arbitrary rigged or skeletal motion. Our method optionally incorporates pose-space deformation (PSD). It runs at milliseconds per frame for complex characters, and fits directly into standard character animation pipelines. Our simulation method does not require any skin data capture; hence, it can be applied to humans, animals, and arbitrary (real-world or fictional) characters. In standard model reduction of three-dimensional nonlinear solid elastic models, one builds a reduced model around a single pose, typically the rest configuration. We demonstrate how to perform multi-model reduction of Finite Element Method (FEM) nonlinear elasticity, where separate reduced models are precomputed around a representative set of object poses, and then combined at runtime into a single fast dynamic system, using subspace interpolation. While time-varying reduction has been demonstrated before for offline applications, our method is fast and suitable for hard real-time applications in games and virtual reality. Our method supports self-contact, which we achieve by computing linear modes and derivatives under contact constraints.

We propose a method to design linear deformation subspaces, unifying linear blend skinning and generalized barycentric coordinates. Deformation subspaces cut down the time complexity of variational shape deformation methods and physics-based animation (reduced-order physics). Our subspaces feature many desirable properties: interpolation, smoothness, shape-awareness, locality, and both constant and linear precision. We achieve these by minimizing a quadratic deformation energy, built via a discrete Laplacian inducing linear precision on the domain boundary. Our main advantage is speed: subspace bases are solutions to a sparse linear system, computed interactively even for generously tessellated domains. Users may seamlessly switch between applying transformations at handles and editing the subspace by adding, removing or relocating control handles. The combination of fast computation and good properties means that designing the right subspace is now just as creative as manipulating handles. This paradigm shift in handle-based deformation opens new opportunities to explore the space of shape deformations.

We present a technique for synthesizing the effects of skin microstructure deformation by anisotropically convolving a high-resolution displacement map to match normal distribution changes in measured skin samples. We use a 10-micron resolution scanning technique to measure several in vivo skin samples as they are stretched and compressed in different directions, quantifying how stretching smooths the skin and compression makes it rougher. We tabulate the resulting surface normal distributions, and show that convolving a neutral skin microstructure displacement map with blurring and sharpening filters can mimic normal distribution changes and microstructure deformations. We implement the spatially-varying displacement map filtering on the GPU to interactively render the effects of dynamic microgeometry on animated faces obtained from high-resolution facial scans.

We demonstrate an interactive method to create heterogeneous continuous deformable materials on complex three-dimensional meshes. The user specifies displacements and internal elastic forces at a chosen set of mesh vertices. Our system then rapidly solves an optimization problem to compute a corresponding heterogeneous spatial distribution of material properties using the Finite Element Method (FEM) analysis. We apply our method to linear and nonlinear isotropic deformable materials. We demonstrate that solving the problem interactively in the full-dimensional space of individual tetrahedron material values is not practical. Instead, we propose a new model reduction method that projects the material space to a low-dimensional space of material modes. Our model reduction accelerates optimization by two orders of magnitude and makes the convergence much more robust, making it possible to interactively design material distributions on complex meshes. We apply our method to precise control of contact forces and control of pressure over large contact areas between rigid and deformable objects for ergonomics. Our tetrahedron-based dithering method can efficiently convert continuous material distributions into discrete ones and we demonstrate its precision via FEM simulation. We physically display our distributions using haptics, as well as demonstrate how haptics can aid in the material design. The produced heterogeneous material distributions can also be used in computer animation applications.

The Finite Element Method is widely used for solid deformable object simulation in film, computer games, virtual reality and medicine. Previous applications of nonlinear solid elasticity employed materials from a few standard families such as linear corotational, nonlinear St.Venant-Kirchhoff, Neo-Hookean, Ogden or Mooney-Rivlin materials. However, the spaces of all nonlinear isotropic and anisotropic materials are infinite-dimensional and much broader than these standard materials. In this paper, we demonstrate how to intuitively explore the space of isotropic and anisotropic nonlinear materials, for design of animations in computer graphics and related fields. In order to do so, we first formulate the internal elastic forces and tangent stiffness matrices in the space of the principal stretches of the material. We then demonstrate how to design new isotropic materials by editing a single stress-strain curve, using a spline interface. Similarly, anisotropic (orthotropic) materials can be designed by editing three curves, one for each material direction. We demonstrate that modifying these curves using our proposed interface has an intuitive, visual, effect on the simulation. Our materials accelerate simulation design and enable visual effects that are difficult or impossible to achieve with standard nonlinear materials.

The Finite Element Method (FEM) is commonly used to simulate isotropic deformable objects in computer graphics. Several applications (wood, plants, muscles) require modeling the directional dependence of the material elastic properties in three orthogonal directions. We investigate linear orthotropic materials, a special class of linear anisotropic materials where the shear stresses are decoupled from normal stresses, as well as general linear (non-orthotropic) anisotropic materials. Orthotropic materials generalize transversely isotropic materials, by exhibiting different stiffness in three orthogonal directions. Orthotropic materials are, however, parameterized by nine values that are difficult to tune in practice, as poorly adjusted settings easily lead to simulation instabilities. We present a user-friendly approach to setting these parameters that is guaranteed to be stable. Our approach is intuitive as it extends the familiar intuition known from isotropic materials. Similarly to linear orthotropic materials, we also derive a stability condition for a subset of general linear anisotropic materials, and give intuitive approaches to tuning them. In order to simulate large deformations, we augment linear corotational FEM simulations with our orthotropic and general anisotropic materials.

The penalty method is a simple and popular approach to resolving contact in computer graphics and robotics. Penalty-based contact, however, suffers from stability problems due to the highly variable and unpredictable net stiffness, and this is particularly pronounced in simulations with time-varying distributed geometrically complex contact. We employ semi-implicit integration, exact analytical contact gradients, symbolic Gaussian elimination and a SVD solver to simulate stable penalty-based frictional contact with large, time-varying contact areas, involving many rigid objects and articulated rigid objects in complex conforming contact and self-contact. We also derive implicit proportional-derivative control forces for real-time control of articulated structures with loops. We present challenging contact scenarios such as screwing a hexbolt into a hole, bowls stacked in perfectly conforming configurations, and manipulating many objects using actively controlled articulated mechanisms in real time.

Abstract This practice and experience paper describes a robust C++ implementation of several non‐linear solid three‐dimensional deformable object strategies commonly employed in computer graphics, named the Vega finite element method (FEM) simulation library. Deformable models supported include co‐rotational linear FEM elasticity, Saint–Venant Kirchhoff FEM model, mass–spring system and invertible FEM models: neo‐Hookean, Saint–Venant Kirchhoff and Mooney–Rivlin. We provide several timestepping schemes, including implicit Newmark and backward Euler integrators, and explicit central differences. The implementation of material models is separated from integration, which makes it possible to employ our code not only for simulation, but also for deformable object control and shape modelling. We extensively compare the different material models and timestepping schemes. We provide practical experience and insight gained while using our code in several computer animation and simulation research projects.

Physically based simulation can produce quality motion of plants, but requires an authoring stage to convert plant "polygon soup" triangle meshes to a format suitable for physically based simulation. We give a system that can author complex simulation-ready plants in a manner of minutes. Our system decomposes the plant geometry, establishes a hierarchy, builds and connects simulation meshes, and detects instances. It scales to anatomically realistic geometry of adult plants, is robust to non-manifold input geometry, gaps between branches or leaves, free-flying leaves not connected to any branch, spurious geometry, and plant self-collisions in the input configuration. We demonstrate the results using a FEM model reduction simulator that can compute large-deformation dynamics of complex plants at interactive rates, subject to user forces, gravity or randomized wind. We also provide plant fracture (with pre-specified patterns), inverse kinematics to easily pose plants, as well as interactive design of plant material properties. We authored and simulated over 100 plants from diverse climates and geographic regions, including broadleaf (deciduous) trees and conifers, bushes and flowers. Our largest simulations involve anatomically realistic adult trees with hundreds of branches and over 100,000 leaves.

We present an interactive animation editor for complex deformable object animations. Given an existing animation, the artist directly manipulates the deformable body at any time frame, and the surrounding animation immediately adjusts in response. The automatic adjustments are designed to respect physics, preserve detail in both the input motion and geometry, respect prescribed bilateral contact constraints, and controllably and smoothly decay in space-time. While the utility of interactive editing for rigid body and articulated figure animations is widely recognized, a corresponding approach to deformable bodies has not been technically feasible before. We achieve interactive rates by combining spacetime model reduction, rotation-strain coordinate warping, linearized elasticity, and direct manipulation. This direct editing tool can serve the final stages of animation production, which often call for detailed, direct adjustments that are otherwise tedious to realize by re-simulation or frame-by-frame editing.

This paper shows a method to extend 3D nonlinear elasticity model reduction to open-loop multi-level reduced deformable structures. Given a volumetric mesh, we decompose the mesh into several subdomains, build a reduced deformable model for each domain, and connect the domains using inertia coupling. This makes model reduction deformable simulations much more versatile: localized deformations can be supported without prohibitive computational costs, parts can be re-used and precomputation times shortened. Our method does not use constraints, and can handle large domain rigid body motion in addition to large deformations, due to our derivation of the gradient and Hessian of the rotation matrix in polar decomposition. We show real-time examples with multi-level domain hierarchies and hundreds of reduced degrees of freedom.

We show how to greatly accelerate self-collision detection (SCD) for reduced deformable models. Given a triangle mesh and a set of deformation modes, our method precomputes Subspace Self-Collision Culling (SSCC) certificates which, if satisfied, prove the absence of self-collisions for large parts of the model. At runtime, bounding volume hierarchies augmented with our certificates can aggressively cull overlap tests and reduce hierarchy updates. Our method supports both discrete and continuous SCD, can handle complex geometry, and makes no assumptions about geometric smoothness or normal bounds. It is particularly effective for simulations with modest subspace deformations, where it can often verify the absence of self-collisions in constant time. Our certificates enable low amortized costs, in time and across many objects in multi-body dynamics simulations. Finally, SSCC is effective enough to support self-collision tests at audio rates, which we demonstrate by producing the first sound simulations of clattering objects.

Keyframe animation is a common technique to generate animations of deformable characters and other soft bodies. With spline interpolation, however, it can be difficult to achieve secondary motion effects such as plausible dynamics when there are thousands of degrees of freedom to animate. Physical methods can provide more realism with less user effort, but it is challenging to apply them to quickly create specific animations that closely follow prescribed animator goals. We present a fast space-time optimization method to author physically based deformable object simulations that conform to animator-specified keyframes. We demonstrate our method with FEM deformable objects and mass-spring systems. Our method minimizes an objective function that penalizes the sum of keyframe deviations plus the deviation of the trajectory from physics. With existing methods, such minimizations operate in high dimensions, are slow, memory consuming, and prone to local minima. We demonstrate that significant computational speedups and robustness improvements can be achieved if the optimization problem is properly solved in a low-dimensional space. Selecting a low-dimensional space so that the intent of the animator is accommodated, and that at the same time space-time optimization is convergent and fast, is difficult. We present a method that generates a quality low-dimensional space using the given keyframes. It is then possible to find quality solutions to difficult space-time optimization problems robustly and in a manner of minutes.

Recent advances have brought real-time physically based simulation within reach, but simulations are still difficult to control in real time. We present interactive simulations of passive systems such as deformable solids or fluids that are not only fast, but also directable: they follow given input trajectories while simultaneously reacting to user input and other unexpected disturbances. We achieve such directability using a real-time controller that runs in tandem with a real-time physically based simulation. To avoid stiff and over-controlled systems where the natural dynamics are overpowered, the injection of control forces has to be minimized. This search for gentle forces can be made tractable in real-time by linearizing the system dynamics around the input trajectory, and then using a time-varying linear quadratic regulator to build the controller. We show examples of controlled complex deformable solids and fluids, demonstrating that our approach generates a requested fixed outcome for reasonable user inputs, while simultaneously providing runtime motion variety.

Simulating sounds produced by realistic vibrating objects is challenging because sound radiation involves complex diffraction and interreflection effects that are very perceptible and important. These wave phenomena are well understood, but have been largely ignored in computer graphics due to the high cost and complexity of computing them at audio rates.We describe a new algorithm for real-time synthesis of realistic sound radiation from rigid objects. We start by precomputing the linear vibration modes of an object, and then relate each mode to its sound pressure field, or acoustic transfer function, using standard methods from numerical acoustics. Each transfer function is then approximated to a specified accuracy using low-order multi-pole sources placed near the object. We provide a low-memory, multilevel, randomized algorithm for optimized source placement that is suitable for complex geometries. At runtime, we can simulate new interaction sounds by quickly summing contributions from each mode's equivalent multipole sources. We can efficiently simulate global effects such as interreflection and changes in sound due to listener location. The simulation costs can be dynamically traded-off for sound quality. We present several examples of sound generation from physically based animations.

In this paper, we present an approach for fast subspace integration of reduced-coordinate nonlinear deformable models that is suitable for interactive applications in computer graphics and haptics. Our approach exploits dimensional model reduction to build reduced-coordinate deformable models for objects with complex geometry. We exploit the fact that model reduction on large deformation models with linear materials (as commonly used in graphics) result in internal force models that are simply cubic polynomials in reduced coordinates. Coefficients of these polynomials can be precomputed, for efficient runtime evaluation. This allows simulation of nonlinear dynamics using fast implicit Newmark subspace integrators, with subspace integration costs independent of geometric complexity. We present two useful approaches for generating low-dimensional subspace bases: modal derivatives and an interactive sketching technique. Mass-scaled principal component analysis (mass-PCA) is suggested for dimensionality reduction. Finally, several examples are given from computer animation to illustrate high performance, including force-feedback haptic rendering of a complicated object undergoing large deformations.