Abstract We introduce affinification, a novel method for accelerating physics‐based animation of elastic solids. During a time‐dependent simulation, our method automatically partitions the space into affine and elastic regions depending on the deformation. As such, we capture localized deformations while significantly reducing computational costs with larger regions of model reduction. We design a new clustering method based on deformation rates to capture affinely deforming regions, and explore multiple heuristics for seeding, pattern generation, and the impact of physical parameters on coarsened regions. We compare our method with the ground truth, showing performance increasing with resolution and recorded simulations up to 17× faster compared to elastic simulations, while retaining similar levels of visual fidelity.

We present GEOPARD, a transformer-based architecture for predicting articulation from a single static snapshot of a 3D shape. The key idea of our method is a pretraining strategy that allows our transformer to learn plausible candidate articulations for 3D shapes based on a geometric-driven search without manual articulation annotation. The search automatically discovers physically valid part motions that do not cause detachments or collisions with other shape parts. Our experiments indicate that this geometric pretraining strategy, along with carefully designed choices in our transformer architecture, yields state-of-the-art results in articulation inference in the PartNet-Mobility dataset.

Abstract Physics simulation is a cornerstone of many computer graphics applications, ranging from video games and virtual reality to visual effects and computational design. The number of techniques for physically‐based modeling and animation has thus skyrocketed over the past few decades, facilitating the simulation of a wide variety of materials and physical phenomena. This report captures the state‐of‐the‐art of multiphysics simulation for computer graphics applications. Although a lot of work has focused on simulating individual phenomena, here we put an emphasis on methods developed by the computer graphics community for simulating various physical phenomena and materials, as well as the interactions between them. These include combinations of discretization schemes, mathematical modeling frameworks, and coupling techniques. For the most commonly used methods we provide an overview of the state‐of‐the‐art and deliver valuable insights into the various approaches. A selection of software frameworks that offer out‐of‐the‐box multiphysics modeling capabilities is also presented. Finally, we touch on emerging trends in physics‐based animation that affect multiphysics simulation, including machine learning‐based methods which have become increasingly popular in recent years.

Interactive applications demand believable characters that respond naturally to dynamic environments. Traditional character animation techniques often struggle to handle arbitrary situations, leading to a growing trend of dynamically selecting motion-captured animations based on predefined features. While Motion Matching has proven effective for locomotion by aligning to target trajectories, animating environment interactions and crowd behaviors remains challenging due to the need to consider surrounding elements. Existing approaches often involve manual setup or lack the naturalism of motion capture. Furthermore, in crowd animation, body animation is frequently treated as a separate process from trajectory planning, leading to inconsistencies between body pose and root motion. To address these limitations, we present Environment-aware Motion Matching , a novel real-time system for full-body character animation that dynamically adapts to obstacles and other agents, emphasizing the bidirectional relationship between pose and trajectory. In a preprocessing step, we extract shape, pose, and trajectory features from a motion capture database. At runtime, we perform an efficient search that matches user input and current pose while penalizing collisions with a dynamic environment. Our method allows characters to naturally adjust their pose and trajectory to navigate crowded scenes.

Adapting motion to new contexts in digital entertainment often demands fast agile prototyping. State-of-the-art techniques use reinforcement learning policies for simulating the underlined motion in a physics engine. Unfortunately, policies typically fail on unseen tasks and it is too time-consuming to fine-tune the policy for every new morphological, environmental, or motion change. We propose a novel point of view on using policy networks as a representation of motion for physics-based character animation. Our policies are compact, tailored to individual motion tasks, and preserve similarity with nearby tasks. This allows us to view the space of all motions as a manifold of policies where sampling substitutes training. We obtain memory-efficient encoding of motion that leverages the characteristics of control policies such as being generative, and robust to small environmental changes. With this perspective, we can sample novel motions by directly manipulating weights and biases through a Diffusion Model. Our newly generated policies can adapt to previously unseen characters, potentially saving time in rapid prototyping scenarios. Our contributions include the introduction of Common Neighbor Policy regularization to constrain policy similarity during motion imitation training making them suitable for generative modeling; a Diffusion Model adaptation for diverse morphology; and an open policy dataset. The results show that we can learn non-linear transformations in the policy space from labeled examples, and conditionally generate new ones. In a matter of seconds, we sample a batch of policies for different conditions that show comparable motion fidelity metrics as their respective trained ones.

Model-based reinforcement learning agents utilizing transformers have shown improved sample efficiency due to their ability to model extended context, resulting in more accurate world models. However, for complex reasoning and planning tasks, these methods primarily rely on continuous representations. This complicates modeling of discrete properties of the real world such as disjoint object classes between which interpolation is not plausible. In this work, we introduce discrete abstract representations for transformer-based learning (DART), a sample-efficient method utilizing discrete representations for modeling both the world and learning behavior. We incorporate a transformer-decoder for auto-regressive world modeling and a transformer-encoder for learning behavior by attending to task-relevant cues in the discrete representation of the world model. For handling partial observability, we aggregate information from past time steps as memory tokens. DART outperforms previous state-of-the-art methods that do not use look-ahead search on the Atari 100k sample efficiency benchmark with a median human-normalized score of 0.790 and beats humans in 9 out of 26 games. We release our code at https://pranaval.github.io/DART/.

Achieving stable simulation of constrained rigid body systems is a primary concern for many computer graphics applications, such as video games, robotic planning, and virtual reality training. In this paper, we present a novel adaptive sub-stepping scheme that achieves stable simulation by adaptively reducing the time step as needed. Our approach employs a diagonalized geometric stiffness matrix as a heuristic to determine when smaller time steps are required, and adjusts the number of sub-steps accordingly. Our method is straightforward to integrate into existing rigid body simulators, and further eliminates manually tuning the number of sub-steps required. We demonstrate the ability of our method to produce stable simulates at real-time frame rates using a number of challenging, complex examples.

We present a framework for the interactive simulation of fluids coupled with rigid bodies that targets heterogeneous distributed computing architectures. Specifically, our proposed approach is well-suited for computer graphics applications that combine servers with large compute capactiy with low-end devices. In this setting, a global large-scale fluid simulation is performed on servers using high-end compute hardware, and local refinement of fluid and rigid body coupling is performed on a client with limited compute resources, such as a tablet or smartphone. We demonstrate the effectiveness of our framework to simulate large and complex scenes involving wind, ocean, and dynamic objects, all while providing plausible interactions through fluid-rigid coupling.

Abstract Physics‐based character motions remain difficult to create and control. We make two contributions towards simpler specification and faster generation of physics‐based control. First, we introduce a novel partwise model predictive control (MPC) method that exploits independent planning for body parts when this proves beneficial, while defaulting to whole‐body motion planning when that proves to be more effective. Second, we introduce a new approach to motion specification, based on specifying an ordered set of contact keyframes. These each specify a small number of pairwise contacts between the body and the environment, and serve as loose specifications of motion strategies. Unlike regular keyframes or traditional trajectory optimization constraints, they are heavily under‐constrained and have flexible timing. We demonstrate a range of challenging contact‐rich motions that can be generated online at interactive rates using this framework. We further show the generalization capabilities of the method.

Abstract Traditional mesh‐based methods for cutting deformable bodies rely on modifying the simulation mesh by deleting, duplicating, deforming or subdividing its elements. Unfortunately, such topological changes eventually lead to instability, reduced accuracy, or computational efficiency challenges. Hence, state of the art algorithms favor the extended finite element method (XFEM), which decouples the cut geometry from the simulation mesh, allowing for stable and accurate cuts at an additional computational cost that is local to the cut region. However, in the 3‐dimensional setting, current XFEM frameworks are limited by the cutting configurations that they support. In particular, intersecting cuts are either prohibited or require sophisticated special treatment. Our work presents a general XFEM formulation that is applicable to the 1‐, 2‐, and 3‐dimensional setting without sacrificing the desirable properties of the method. In particular, we propose a generalized enrichment which supports multiple intersecting cuts of various degrees of non‐linearity by leveraging recent advances in robust mesh‐Boolean technology. This novel strategy additionally enables analytic discontinuous integration schemes required to compute mass, force and elastic energy. We highlight the simplicity, expressivity and accuracy of our XFEM implementation across various scenarios in which intersecting cutting patterns are featured.

In this paper, we propose a physics-based model of suction phenomenon to achieve simulation of deformable objects like suction cups. Our model uses a constraint-based formulation to simulate the variations of pressure inside suction cups. The respective internal pressures are represented as pressure constraints which are coupled with anti-interpenetration and friction constraints. Furthermore, our method is able to detect multiple air cavities using information from collision detection. We solve the pressure constraints based on the ideal gas law while considering several cavity states. We test our model with a number of scenarios reflecting a variety of uses, for instance, a spring loaded jumping toy, a manipulator performing a pick and place task, and an octopus tentacle grasping a soda can. We also evaluate the ability of our model to reproduce the physics of suction cups of varying shapes, lifting objects of different masses, and sliding on a slippery surface. The results show promise for various applications such as the simulation in soft robotics and computer animation.

Motivated by humans' ability to adapt skills in the learning of new ones, this paper presents AdaptNet, an approach for modifying the latent space of existing policies to allow new behaviors to be quickly learned from like tasks in comparison to learning from scratch. Building on top of a given reinforcement learning controller, AdaptNet uses a two-tier hierarchy that augments the original state embedding to support modest changes in a behavior and further modifies the policy network layers to make more substantive changes. The technique is shown to be effective for adapting existing physics-based controllers to a wide range of new styles for locomotion, new task targets, changes in character morphology and extensive changes in environment. Furthermore, it exhibits significant increase in learning efficiency, as indicated by greatly reduced training times when compared to training from scratch or using other approaches that modify existing policies. Code is available at https://motion-lab.github.io/AdaptNet .

We design a nominal controller for animating an articulated physics-based human arm model, including the hands and fingers, to catch and throw objects. The controller is based on a finite state machine that defines the target poses for proportional-derivative control of the hand, as well as the orientation and position of the center of the palm using the solution of an inverse kinematics solver. We then use reinforcement learning to train agents to improve the robustness of the nominal controller for achieving many different goals. Imitation learning based on trajectories output by a numerical optimization is used to accelerate the training process. The success of our controllers is demonstrated by a variety of throwing and catching tasks, including flipping objects, hitting targets, and throwing objects to a desired height, and for several different objects, such as cans, spheres, and rods. We also discuss ways to extend our approach so that more challenging tasks, such as juggling, may be accomplished.

Abstract We use a finite element model to predict the vibration response of objects in a rigid body simulation, such that rigid objects are augmented to provide a plausible elastic collision response between distant objects due to vibration. We start with a generalized eigenvalue decomposition of the elastic model to precompute a response to an impact at any point on an elastic object with fixed boundary conditions. Then, given a collision between objects, we generate an approximate response impulse to distribute to other objects already in contact with the colliding bodies. This can lead to distant impacts causing an object to slip, or a delicate stack of objects to fall. We also use a geodesic distance based spatial attenuation approximation for travelling waves in objects to respond to an impact at one contact with an impulse at other locations. This response ultimately allows a long distance relationship between contacts, both across a single object being struck, but also traversing the contact graph of a larger collection of objects. We qualitatively validate our approach with a ground truth simulation, and demonstrate a number of scenarios where a long distance relationship between contacts is valuable.

Abstract Inspired by frictional behaviour that is observed when sliding matchsticks against one another at different angles, we propose a phenomenological anisotropic friction model for structured surfaces. Our model interpolates isotropic and anisotropic elliptical Coulomb friction parameters for a pair of surfaces with perpendicular and parallel structure directions (e.g. the wood grain direction). We view our model as a special case of an abstract friction model that produces a cone based on state information, specifically the relationship between structure directions. We show how our model can be integrated into LCP and NCP‐based simulators using different solvers with both explicit and fully implicit time‐integration. The focus of our work is on symmetric friction cones, and we therefore demonstrate a variety of simulation scenarios where the friction structure directions play an important part in the resulting motions. Consequently, authoring of friction using our model is intuitive and we demonstrate that our model is compatible with standard authoring practices, such as texture mapping.

Simulating stiff physical systems is a requirement for numerous computer graphics applications, such as VR training for heavy equipment operation. However, iterative linear solvers often perform poorly in such cases, and direct methods involving a factorization of the system matrix are typically preferred for accurate and stable simulations. This can have a detrimental impact on performance, since factorization of the system matrix is costly for complex simulations. In this paper, we present a method for efficiently solving linear systems of stiff physical systems involving contact, where the dynamics are modeled as a mixed linear complementarity problem (MLCP). Our approach is based on a block Bard-type algorithm that applies low-rank downdates to a Cholesky factorization of the system matrix at each pivoting step. Further performance improvements are realized by exploiting low bandwidth characteristics of the factorization. Our method gives up to 3.5x speed-up versus recomputing the factorization based on the index set. Various challenging scenarios are used to demonstrate the advantages of our approach.

Substructuring permits parallelization of physics simulation on multi-core CPUs. We present a new substructuring approach for solving stiff multibody systems containing both bilateral and unilateral constraints. Our approach is based on non-overlapping domain decomposition with the Schur complement method, which we extend to systems involving contact formulated as a mixed bounds linear complementarity problem. At each time step, we alternate between solving the subsystem and interface constraint impulses, which leads to the identification of the active constraints. By using the active constraints to compute the effective mass of subsystems within the interface solve, we obtain an exact solution. We demonstrate that our simulations have preferable behavior compared to standard iterative solvers and substructuring techniques based on the exchange of forces at interface bodies. We observe considerable speedups for structured simulations where a user-defined partitioning can be applied, and moderate speedups for unstructured simulations, such as piles of bodies. In the latter case, we propose an automatic partitioning strategy based on the degree of bodies in the constraint graph. Because our method makes use of direct solvers, we are able to achieve interactive and real-time frame rates for a number of challenging scenarios involving large mass ratios, redundant constraints, and ill-conditioned systems.

Inverse kinematics (IK) is a central component of systems for motion capture, character animation, motion planning, and robotics control. The field of computer graphics has developed fast stationary point solvers methods, such as the Jacobian transpose method and cyclic coordinate descent. Much work with Newton methods focus on avoiding directly computing the Hessian, and instead approximations are sought, such as in the BFGS class of solvers. This paper presents a numerical method for computing the exact Hessian of an IK system with spherical joints. It is applicable to human skeletons in computer animation applications and some, but not all, robots. Our results show that using exact Hessians can give performance advantages and higher accuracy compared to standard numerical methods used for solving IK problems. Furthermore, we provide code and supplementary details that allows researchers to plug-in exact Hessians in their own work with little effort.

This paper focuses on the stable and efficient simulation of articulated rigid body systems for real-time applications. Specifically, we focus on the use of geometric stiffness which can dramatically increase simulation stability. We examine several numerical problems with the inclusion of geometric stiffness in the equations of motion, as proposed by previous work, and address these issues by introducing a novel method for efficiently building the linear system. This offers improved tractability and numerical efficiency. Furthermore, geometric stiffness tends to significantly dissipate kinetic energy. We propose an adaptive damping scheme, inspired by the geometric stiffness, that uses a stability criterion based on the numerical integrator to determine the amount of non-constitutive damping required to stabilize the simulation. With this approach, not only is the dynamical behavior better preserved, but the simulation remains stable for mass ratios of 1,000,000-to-1 at time steps up to 0.1 s. We present a number of challenging scenarios to demonstrate that our method improves efficiency, and that it increases stability by orders of magnitude compared to previous work.

We present a method for the simulation of compliant, articulated structures using a plausible approximate model that focuses on modeling endpoint interaction. We approximate the structure's behavior about a reference configuration, resulting in a first order reduced compliant system, or FORK (-1) S. Several levels of approximation are available depending on which parts and surfaces we would like to have interactive contact forces, allowing various levels of detail to be selected. Our approach is fast and computation of the full structure's state may be parallelized. Furthermore, we present a method for reducing error by combining multiple FORK (-1)S models at different linearization points, through twist blending and matrix interpolation. Our approach is suitable for stiff, articulate grippers, such as those used in robotic simulation, or physics-based characters under static proportional derivative control. We demonstrate that simulations with our method can deal with kinematic chains and loops with non-uniform stiffness across joints, and that it produces plausible effects due to stiffness, damping, and inertia.

We present a method for the simulation of compliant, articulated structures using a plausible approximate model that focuses on modeling endpoint interaction. We approximate the structure's behavior about a reference configuration, resulting in a first order reduced compliant system, or FORK-1S. Several levels of approximation are available depending on which parts and surfaces we would like to have interactive contact forces, allowing various levels of detail to be selected. Our approach is fast and computation of the full structure's state may be parallelized. Our approach is suitable for stiff, articulate grippers, such as those used in robotic simulation, or physics based characters under static proportional derivative control. We demonstrate that simulations with our method can deal with kinematic chains and loops with non-uniform stiffness across joints, and that it produces plausible effects due to stiffness, damping, and inertia.

Contact on a finger pad results in deformation that redistributes blood within the fingertip tissue in a manner correlated to the pressure. We build a data-driven model that relates contact information to the visible changes of the finger nail and surrounding tissue on the back of the finger tip. Our data analysis and model construction makes use of the space of hemoglobin concentrations, as opposed to an RGB color space, which permits the model to be transferred across different fingers and different people. We use principal component analysis to build a compact model which maps well to graphics hardware with an efficient fragment program implementation. We provide a validation of our model, and a demonstration of a grasping controller running in a physically based simulation, where grip strength is visible in both hand posture and the appearance of color changes at the fingertips.

Haptic texture represents the fine-grained attributes of an object's surface and is related to physical characteristics such as roughness and stiffness. We introduce an interactive and mobile scanning system for the acquisition and synthesis of haptic textures that consists of a visually tracked handheld touch probe. The most novel aspect of our work is an estimation method for the contact stiffness of an object based solely on the acceleration and forces measured during stroking of its surface with the handheld probe. We establish an experimental relationship between the estimated stiffness and the contact stiffness observed during compression. We also measure the height-displacement profile of an object's surface enabling us to generate haptic textures. We show an example of mapping the textures on to a coarse surface mesh obtained with an image-based technique, but the textures may also be combined with coarse surface meshes obtained by manual modeling.