Graph neural networks (GNNs) have emerged as a versatile and efficient option for modeling the dynamic behavior of deformable materials. While GNNs generalize readily to arbitrary shapes, mesh topologies, and material parameters, existing architectures struggle to correctly predict the temporal evolution of key physical quantities such as linear and angular momentum. In this work, we propose MomentumGNN—a novel architecture designed to accurately track momentum by construction. Unlike existing GNNs that output unconstrained nodal accelerations, our model predicts per-edge stretching and bending impulses which guarantee the preservation of linear and angular momentum. We train our network in an unsupervised fashion using a physics-based loss, and we show that our method outperforms baselines in a number of common scenarios where momentum plays a pivotal role.

Abstract Facial skin dynamics are inherently challenging to simulate due to a combination of geometric, material, and anatomical complexities. Human skin is a nonlinear layered material with spatially heterogeneous attachments to the underlying tissues. During contact events, localized compression and shear induce mechanical instabilities, leading to fine‐scale wrinkling patterns governed by a delicate interplay of geometry, boundary conditions, and through‐the‐thickness stresses. We present a finite element framework to simulate contact‐induced wrinkling of facial skin. We model skin as a viscoelastic material with time‐dependent relaxation that governs the rate, persistence, and damping of wrinkle formation. We employ high‐order prismatic solid‐shell elements to resolve through‐thickness stresses and high‐frequency deformation modes. Central to our approach, we introduce a continuum‐based formulation of skin ligaments to model heterogeneous skin attachments and provide anatomically inspired mobility constraints. These skin ligaments control the formation and appearance of facial wrinkles by modulating their amplitude, wavelength, and spatial distribution. We evaluate our method on a set of synthetic examples and compare simulations with real‐world footage. These results demonstrate that our skin model produces temporally coherent and visually realistic wrinkle patterns during transient contact.

Abstract Structured sheet materials such as 3D‐printed rod networks, multi‐material thin shells, and multi‐layer laminates exhibit diverse mechanical behaviors. To avoid the computational burden of native‐scale simulations, data‐driven homogenization offers a promising alternative. This process involves probing a representative patch of material—a unit cell—with a set of stretching and bending tests subject to periodic boundary conditions. Because macro‐scale bending moments are not directly available from native‐scale simulations, existing methods exclusively rely on elastic energy data. Unfortunately, using only elastic energy from uniaxial tests is not sufficient for capturing the full moment‐curvature relationship, and imposing biaxial curvature states would necessarily break periodicity. We present a moment‐based homogenization method that infers curvature coupling using only uniaxial bending tests. Our method computes macro‐scale bending moments from native‐scale simulations for a wide range of mechanical models. To this end, we translate internal deformations into elastic stresses and then integrate these stresses through the thickness and across the unit‐cell patch. We use the resulting homogenized bending moments along with energy data to fit neural bending energy density functions. We demonstrate our method on a diverse set of materials, including multi‐material shells, rod networks, and multi‐layer sheets. Our results show improved accuracy compared to existing approaches and realistic double‐curvature behavior when applied to larger samples.

Abstract Human motion data is inherently rich and complex, containing both semantic content and subtle stylistic features that are challenging to model. We propose a novel method for effective disentanglement of the style and content in human motion data to facilitate style transfer. Our approach is guided by the insight that content corresponds to coarse motion attributes while style captures the finer, expressive details. To model this hierarchy, we employ Residual Vector Quantized Variational Autoencoders (RVQ‐VAEs) to learn a coarse‐to‐fine representation of motion. We further enhance the disentanglement by integrating codebook learning with contrastive learning and a novel information leakage loss to organize the content and the style across different codebooks. We harness this disentangled representation using our simple and effective inference‐time technique Quantized Code Swapping , which enables motion style transfer without requiring any fine‐tuning for unseen styles. Our framework demonstrates strong versatility across multiple inference applications, including style transfer, style removal, and motion blending.

Realistic animal motion for virtual production is typically obtained either through motion capture of highly trained performers who accurately mimic animal behavior, or by retargeting ordinary human motion using complex control setups. Both approaches are challenging and often fail to fully reproduce the nuances of natural animal motion, motivating data-driven alternatives. We present an automatic human-to-quadruped puppeteering framework that produces plausible and controllable quadruped motions from ordinary human motion data. Our approach employs a two-stage generative diffusion model trained purely on quadruped motion data. By introducing a structured conditioning and inpainting strategy, our method supports a wide range of actions, including walking, running, jumping, sitting, and lying. Furthermore, we enable fine-grained intuitive control of the quadruped motion such as head movement control and individual limb puppeteering. Experimental results demonstrate improved motion realism and controllability compared to existing retargeting approaches, highlighting the effectiveness of our framework as a tool for animation and virtual production applications.

We introduce ViSkin, a biomechanically principled approach to simulate skin mechanics on personalized avatars. Our model captures the salient characteristics of human skin, i.e., nonlinear stretching properties, anisotropic stiffness, direction-dependent pre-stretch, and heterogeneous sliding behavior. In particular, we introduce a novel representation of Langer lines, which describe the distribution of principal material directions across the human body. We further propose an optimization-based approach for inferring spatially-varying pre-stretch from motion capture data. We implement our new model using a computationally efficient intrinsic representation that simulates skin as a two-dimensional Lagrangian mesh embedded in the three-dimensional body surface. We demonstrate our method on a diverse set of body models, shapes, and poses and compare to experimentally-obtained skin motion data. Our results indicate that our method produces smoother and more plausible skin deformations than a baseline method and shows good accuracy compared to real-world data.

Reinforcement learning (RL) is ubiquitous in the development of modern AI systems. However, state-of-the-art RL agents require extensive, and potentially unsafe, interactions with their environments to learn effectively. These limitations confine RL agents to simulated environments, hindering their ability to learn directly in real-world settings. In this work, we present ActSafe, a novel model-based RL algorithm for safe and efficient exploration. ActSafe learns a well-calibrated probabilistic model of the system and plans optimistically w.r.t. the epistemic uncertainty about the unknown dynamics, while enforcing pessimism w.r.t. the safety constraints. Under regularity assumptions on the constraints and dynamics, we show that ActSafe guarantees safety during learning while also obtaining a near-optimal policy in finite time. In addition, we propose a practical variant of ActSafe that builds on latest model-based RL advancements and enables safe exploration even in high-dimensional settings such as visual control. We empirically show that ActSafe obtains state-of-the-art performance in difficult exploration tasks on standard safe deep RL benchmarks while ensuring safety during learning.

Reinforcement learning (RL) algorithms aim to balance exploiting the current best strategy with exploring new options that could lead to higher rewards. Most common RL algorithms use undirected exploration, i.e., select random sequences of actions. Exploration can also be directed using intrinsic rewards, such as curiosity or model epistemic uncertainty. However, effectively balancing task and intrinsic rewards is challenging and often task-dependent. In this work, we introduce a framework, MaxInfoRL, for balancing intrinsic and extrinsic exploration. MaxInfoRL steers exploration towards informative transitions, by maximizing intrinsic rewards such as the information gain about the underlying task. When combined with Boltzmann exploration, this approach naturally trades off maximization of the value function with that of the entropy over states, rewards, and actions. We show that our approach achieves sublinear regret in the simplified setting of multi-armed bandits. We then apply this general formulation to a variety of off-policy model-free RL methods for continuous state-action spaces, yielding novel algorithms that achieve superior performance across hard exploration problems and complex scenarios such as visual control tasks.

We present a novel differentiable approach to quantifying and optimizing stability in robotic systems addressing an open challenge in the field of robot analysis, control, design, and optimization. Our method leverages differentiable simulation over extended time horizons to estimate a robustness metric based on the Lyapunov exponents. The proposed metric offers several properties, including a natural extension to limit cycles (commonly encountered in robotics tasks and locomotion) and independence from the trajectory path for states converging to the attractor. We showcase, with an ad-hoc JAX gradient-based optimization framework, remarkable flexibility in tackling the robustness challenge. Our approach is tested through diverse scenarios of varying complexity, encompassing high-degree-of-freedom systems and contact-rich environments. The positive outcomes across these cases highlight the potential of our method in quantifying and possibly enhancing system robustness.

Robotic manipulation of deformable objects remains challenging due to the high dimensional configuration space and complex dynamics. In this work we demonstrate how the abstraction level used for modeling deformable objects can significantly impact the difficulty of the motion planning problem. We specifically focus on buckling—a nonlinear instability phenomenon that arises in response to compression of slender deformable objects. Using deformable linear objects (DLOs) as a case of study, we show that eliminating resistance to compression in the simulation model while penalizing compressed states in the planning objective increases both robustness and performance. We demonstrate our approach on a set of simulation examples and validate our results through physical robot experiments.

Robotic systems are routinely used in the logistics industry to enhance operational efficiency, but the design of robot workspaces remains a complex and manual task, which limits the system’s flexibility to changing demands. This paper aims to automate robot workspace design by proposing a computational framework to generate a budget-minimizing layout by selectively placing stationary robots on a floor grid to sort packages from given input and output locations. Finding a good layout that minimizes the hardware budget while ensuring motion feasibility is a challenging combinatorial problem with nonconvex motion constraints. We propose a new optimization-based approach that models layout planning as a subgraph optimization problem subject to network flow constraints. Our core insight is to abstract away motion constraints from the layout optimization by precomputing a kinematic reachability graph and then extract the optimal layout on this ground graph. We validate the motion feasibility of our approach by proposing a simple task assignment and motion planning technique. We benchmark our algorithm on problems with various grid resolutions and number of outputs and show improvements in memory efficiency over a heuristic search algorithm. In addition, we demonstrate that our algorithm can be extended to handle various types of robot manipulators and conveyor belts, box payload constraints, and cost assignments.

Deploying reinforcement learning (RL) safely in the real world is challenging, as policies trained in simulators must face the inevitable *sim-to-real gap*. Robust safe RL techniques are provably safe, however difficult to scale, while domain randomization is more practical yet prone to unsafe behaviors. We address this gap by proposing SPiDR, short for Sim-to-real via Pessimistic Domain Randomization—a scalable algorithm with provable guarantees for safe sim-to-real transfer. SPiDR uses domain randomization to incorporate the uncertainty about the sim-to-real gap into the safety constraints, making it versatile and highly compatible with existing training pipelines. Through extensive experiments on sim-to-sim benchmarks and two distinct real-world robotic platforms, we demonstrate that SPiDR effectively ensures safety despite the sim-to-real gap while maintaining strong performance.

3D cellular metamaterials are valued for many unique and useful mechanical properties. They enable lightweight, high-strength structures, with a wide range of directional stiffness profiles and possible auxetic behaviour. Infill patterns based on triply-periodic minimal surfaces (TPMS) are commonly used in additive manufacturing due to their high strength-to-weight ratio and near-isotropic mechanical behaviour. While existing work provides a wide range of cellular metamaterials to choose from, optimization of these patterns remains a significant challenge due to the diverse space of possible surface topologies and the lack of a unified parameterization. As a promising alternative, Voronoi diagrams with star-shaped distance metrics have been shown to provide a continuous parameterization of 2D cellular metamaterials, opening a rich space of possible designs. Extending the work of [Zhou et al. 2025], we provide a novel, differentiable construction of 3D volumetric Voronoi diagrams with star-shaped metrics. We integrate our formulation into a complete pipeline for mechanical metamaterial optimization, demonstrating the flexibility of star-shaped metric Voronoi diagrams to create periodic structures with a diverse range of directional stiffness profiles and stress-strain curves. Furthermore, we demonstrate the applicability of this framework to heterogeneous, smoothly graded cellular structures.

Temporary bar structures made of reusable standardized components are widely used in construction, events, and exhibitions. They are economical, easy to assemble, and can be disassembled and reused in various structural arrangements for various purposes. However, existing reusable temporary structures are either limited to modular yet repetitive designs or require bespoke components, which restricts their reuse potential. Instead of designing bespoke kit of parts for limited reuse, this paper investigates how to design and build diverse freeform structures from one homogeneous kit of parts. We propose a computational framework to generate multi-tangent bar structures, a widely used jointing system, which allows bars to be joined at any point along their length with standard connectors. We present a mathematical formulation and a numerical scheme to optimize the bar spatial positions and contact assignment simultaneously, while ensuring that the constraints of tangency, collision, joint connectivity, and bar length are satisfied. Together with simulated case studies, we present two physical prototypes that reuse the same kit of parts using an augmented reality-guided assembly workflow. • Computational design of diverse freeform structures from uniform kit of parts. • Modeling of tangency, collision, connectivity, and bar-length constraints. • Simultaneous optimization of geometry and connectors for uniform bars and couplers. • Augmented reality-assisted rapid prefabrication and on-site assembly process. • Two pavilion-scale prototypes demonstrate reusability of one parts kit.

Cellular patterns, from planar ornaments to architectural surfaces and mechanical metamaterials, blend aesthetics with functionality. Homogeneous patterns like isohedral tilings offer simplicity and symmetry but lack flexibility, particularly for heterogeneous designs. They cannot smoothly interpolate between tilings or adapt to double-curved surfaces without distortion. Voronoi diagrams provide a more adaptable patterning solution. They can be generalized to star-shaped metrics, enabling diverse cell shapes and continuous grading by interpolating metric parameters. Martínez et al. [2019] explored this idea in 2D using a rasterization-based algorithm to create compelling patterns. However, this discrete approach precludes gradient-based optimization, limiting control over pattern quality. We introduce a novel, closed-form, fully differentiable formulation for Voronoi diagrams with piecewise linear star-shaped metrics, enabling optimization of site positions and metric parameters to meet aesthetic and functional goals. It naturally extends to arbitrary dimensions, including curved 3D surfaces. For improved on-surface patterning, we propose a per-sector parameterization of star-shaped metrics, ensuring uniform cell shapes in non-regular neighborhoods. We demonstrate our approach by generating diverse patterns, from homogeneous to continuously graded designs, with applications in decorative surfaces and metamaterials.

This work presents a motion retargeting approach for legged robots, aimed at transferring the dynamic and agile movements to robots from source motions. In particular, we guide the imitation learning procedures by transferring motions from source to target, effectively bridging the morphological disparities while ensuring the physical feasibility of the target system. In the first stage, we focus on motion retargeting at the kinematic level by generating kinematically feasible whole-body motions from keypoint trajectories. Following this, we refine the motion at the dynamic level by adjusting it in the temporal domain while adhering to physical constraints. This process facilitates policy training via reinforcement learning, enabling precise and robust motion tracking. We demonstrate that our approach successfully transforms noisy motion sources, such as hand-held camera videos, into robot-specific motions that align with the morphology and physical properties of the target robots. Moreover, we demonstrate terrain-aware motion retargeting to perform BackFlip on top of a box. We successfully deployed these skills to four robots with different dimensions and physical properties in the real world through hardware experiments.

We propose a self-supervised approach for learning physics-based subspaces for real-time simulation. Existing learning-based methods construct subspaces by approximating pre-defined simulation data in a purely geometric way. However; this approach tends to produce highenergy configurations, leads to entangled latent space dimensions, and generalizes poorly beyond the training set. To overcome these limitations, we propose a self-supervised approach that directly minimizes the system's mechanical energy during training. We show that our method leads to learned subspaces that reflect physical equilibrium constraints, resolve overfitting issues of previous methods, and offer interpretable latent space parameters.

Control policies trained using deep reinforcement learning often generate stiff, high-frequency motions in response to unexpected disturbances. To promote more natural and compliant balance recovery strategies, we propose a simple modification to the typical reinforcement learning training process. Our key insight is that stiff responses to perturbations are due to an agent’s incentive to maximize task rewards at all times, even as perturbations are being applied. As an alternative, we introduce an explicit recovery stage where tracking rewards are given irrespective of the motions generated by the control policy. This allows agents a chance to gradually recover from disturbances before attempting to carry out their main tasks. Through an in-depth analysis, we highlight both the compliant nature of the resulting control policies, as well as the benefits that compliance brings to legged locomotion. In our simulation and hardware experiments, the compliant policy achieves more robust, energy-efficient, and safe interactions with the environment.

Precise reconstruction and manipulation of the crumpled cloths is challenging due to the high dimensionality of cloth models, as well as the limited observation at self-occluded regions. We leverage the recent progress in the field of single-view reconstruction to template-based reconstruct the crumpled cloths from their top-view depth observations only, with our proposed sim-real registration protocols. In contrast to previous implicit cloth representations, our reconstruction mesh explicitly describes the positions and visibilities of the entire cloth mesh vertices, enabling more efficient dual-arm and single-arm target-oriented manipulations. Experiments demonstrate that our TRTM system can be applied to daily cloths that have similar topologies as our template mesh, but with different shapes, sizes, patterns, and physical properties. Videos, datasets, pre-trained models, and code can be downloaded from our project website: https://wenbwa.github.io/TRTM/.

We present Sim-FSVGD for learning robot dynamics from data. As opposed to traditional methods, Sim-FSVGD leverages low-fidelity physical priors, e.g., in the form of simulators, to regularize the training of neural network models. While learning accurate dynamics already in the low data regime, Sim-FSVGD scales and excels also when more data is available. We empirically show that learning with implicit physical priors results in accurate mean model estimation as well as precise uncertainty quantification. We demonstrate the effectiveness of Sim-FSVGD in bridging the sim-to-real gap on a high-performance RC racecar system. Using model-based RL, we demonstrate a highly dynamic parking maneuver with drifting, using less than half the data compared to the state of the art.

We study the problem of nonepisodic reinforcement learning (RL) for nonlinear dynamical systems, where the system dynamics are unknown and the RL agent has to learn from a single trajectory, i.e., without resets. We propose **N**on**e**pisodic **O**ptistmic **RL** (NeoRL), an approach based on the principle of optimism in the face of uncertainty. NeoRL uses well-calibrated probabilistic models and plans optimistically w.r.t. the epistemic uncertainty about the unknown dynamics. Under continuity and bounded energy assumptions on the system, we provide a first-of-its-kind regret bound of $\mathcal{O}(\beta_T \sqrt{T \Gamma_T})$ for general nonlinear systems with Gaussian process dynamics. We compare NeoRL to other baselines on several deep RL environments and empirically demonstrate that NeoRL achieves the optimal average cost while incurring the least regret.

Abstract Modeling contact between deformable solids is a fundamental problem in computer animation, mechanical design, and robotics. Existing methods based on C 0 ‐discretizations—piece‐wise linear or polynomial surfaces—suffer from discontinuities and irregularities in tangential contact forces, which can significantly affect simulation outcomes and even prevent convergence. In this work, we show that these limitations can be overcome with a smooth surface representation based on Implicit Moving Least Squares (IMLS). In particular, we propose a self collision detection scheme tailored to IMLS surfaces that enables robust and efficient handling of challenging self contacts. Through a series of test cases, we show that our approach offers advantages over existing methods in terms of accuracy and robustness for both forward and inverse problems.

Computing intrinsic distances on discrete surfaces is at the heart of many minimization problems in geometry processing and beyond. Solving these problems is extremely challenging as it demands the computation of on-surface distances along with their derivatives. We present a novel approach for intrinsic minimization of distance-based objectives defined on triangle meshes. Using a variational formulation of shortest-path geodesics, we compute first and second-order distance derivatives based on the implicit function theorem, thus opening the door to efficient Newton-type minimization solvers. We demonstrate our differentiable geodesic distance framework on a wide range of examples, including geodesic networks and membranes on surfaces of arbitrary genus, two-way coupling between hosting surface and embedded system, differentiable geodesic Voronoi diagrams, and efficient computation of Karcher means on complex shapes. Our analysis shows that second-order descent methods based on our differentiable geodesics outperform existing first-order and quasi-Newton methods by large margins.

We present a computational approach for modeling the mechanical behavior of flexible scaled sheet materials---3D-printed hard scales embedded in a soft substrate. Balancing strength and flexibility, these structured materials find applications in protective gear, soft robotics, and 3D-printed fashion. To unlock their full potential, however, we must unravel the complex relation between scale pattern and mechanical properties. To address this problem, we propose a contact-aware homogenization approach that distills native-level simulation data into a novel macromechanical model. This macro-model combines piecewise-quadratic uniaxial fits with polar interpolation using circular harmonics, allowing for efficient simulation of large-scale patterns. We apply our approach to explore the space of isohedral scale patterns, revealing a diverse range of anisotropic and nonlinear material behaviors. Through an extensive set of experiments, we show that our models reproduce various scale-level effects while offering good qualitative agreement with physical prototypes on the macro-level.

Navigating topological transitions in cellular mechanical systems is a significant challenge for existing simulation methods. While abstract models lack predictive capabilities at the cellular level, explicit network representations struggle with topology changes, and per-cell representations are computationally too demanding for large-scale simulations. To address these challenges, we propose a novel cell-centered approach based on differentiable Voronoi diagrams. Representing each cell with a Voronoi site, our method defines shape and topology of the interface network implicitly. In this way, we substantially reduce the number of problem variables, eliminate the need for explicit contact handling, and ensure continuous geometry changes during topological transitions. Closed-form derivatives of network positions facilitate simulation with Newton-type methods for a wide range of per-cell energies. Finally, we extend our differentiable Voronoi diagrams to enable coupling with arbitrary rigid and deformable boundaries. We apply our approach to a diverse set of examples, highlighting splitting and merging of cells as well as neighborhood changes. We illustrate applications to inverse problems by matching soap foam simulations to real-world images. Comparative analysis with explicit cell models reveals that our method achieves qualitatively comparable results at significantly faster computation times.

Mobile robots are becoming safer and more affordable, and their presence in the workspace is increasing. However, many tasks that involve reasoning, long-term planning or human preferences are still hard to automate. While some solutions in specialised areas slowly emerge, an alternative to full autonomy can be to actively leverage intuition and experience of human operators. To do this, suitable interfaces and modes of interaction have to be explored. Inspired by Real-Time Strategy games, we implement a Mixed Reality interface that can be used with either a Microsoft HoloLens 2 headset or a tablet. The interface allows users to interact with multiple mobile robots simultaneously. We conduct a user study to compare the headset and tablet versions of the interface in different scenarios inspired by a real-world construction setting. We show that, while performance and preference of interface are dependent on the task and the complexity of the required interaction, users are able to solve non-trivial tasks on both platforms using our system.

Trajectory optimization methods have achieved an exceptional level of performance on real-world robots in recent years. These methods heavily rely on accurate analytical models of the dynamics, yet some aspects of the physical world can only be captured to a limited extent. An alternative approach is to leverage machine learning techniques to learn a differentiable dynamics model of the system from data. In this work, we use trajectory optimization and model learning for performing highly dynamic and complex tasks with robotic systems in absence of accurate analytical models of the dynamics. We show that a neural network can model highly nonlinear behaviors accurately for large time horizons, from data collected in only 25 minutes of interactions on two distinct robots: (i) the Boston Dynamics Spot and an (ii) RC car. Furthermore, we use the gradients of the neural network to perform gradient-based trajectory optimization. In our hardware experiments, we demonstrate that our learned model can represent complex dynamics for both the Spot and Radio-controlled (RC) car, and gives good performance in combination with trajectory optimization methods.

Real-world robotic manipulation tasks remain an elusive challenge, since they involve both fine-grained environment interaction, as well as the ability to plan for long-horizon goals. Although deep reinforcement learning (RL) methods have shown encouraging results when planning end-to-end in high-dimensional environments, they remain fundamentally limited by poor sample efficiency due to inefficient exploration, and by the complexity of credit assignment over long horizons. In this work, we present Efficient Learning of High-Level Plans from Play (ELF-P), a framework for robotic learning that bridges motion planning and deep RL to achieve long-horizon complex manipulation tasks. We leverage task-agnostic play data to learn a discrete behavioral prior over object-centric primitives, modeling their feasibility given the current context. We then design a high-level goal-conditioned policy which (1) uses primitives as building blocks to scaffold complex long-horizon tasks and (2) leverages the behavioral prior to accelerate learning. We demonstrate that ELF-P has significantly better sample efficiency than relevant baselines over multiple realistic manipulation tasks and learns policies that can be easily transferred to physical hardware.

We propose a computational approach for designing fully-integrated compliant mechanisms with bio-inspired joints that are stabilized and actuated by elastic elements. Similar to human knees or finger phalanges, our mechanisms leverage sliding between pairs of contacting surfaces to generate complex motions. Due to the vast design space, however, finding surface shapes that lead to ideal approximations of given target motions is a challenging and time-consuming task. To assist users in this process, our computational design tool combines forward and inverse simulation strategies that allow for guided and automated exploration of the parameter space. We demonstrate the potential of our method on a set of compliant mechanism with different joint geometries and validate our simulation results on 3D-printed prototypes.

Task and motion planning is one of the key problems in robotics today. It is often formulated as a discrete task allocation problem combined with continuous motion planning. Many existing approaches to TAMP involve explicit descriptions of task primitives that cause discrete changes in the kinematic relationship between the actor and the objects. In this work we propose an alternative, fully differentiable approach which supports a large number of TAMP problem instances. Rather than explicitly enumerating task primitives, actions are instead represented implicitly as part of the solution to a nonlinear optimization problem. We focus on decision making for robotic manipulators, specifically for pick and place tasks, and explore the efficacy of the model through a number of simulated experiments including multiple robots, objects and interactions with the environment. We also show several possible extensions.

We present Ungar, an open-source library to aid the implementation of high-dimensional optimal control problems (OCPs). We adopt modern template metaprogramming techniques to enable the compile-time modeling of complex systems while retaining maximum runtime efficiency. Our framework provides syntactic sugar to allow for expressive formulations of a rich set of structured dynamical systems. While the core modules depend only on the header-only Eigen and Boost.Hana libraries, we bundle our codebase with optional packages and custom wrappers for automatic differentiation, code generation, and nonlinear programming. Finally, we demonstrate the versatility of Ungar in various model predictive control applications, namely, four-legged locomotion and collaborative loco-manipulation with multiple one-armed quadruped robots. Ungar is available under the Apache License 2.0 at https://github.com/fdevinc/ungar.

Reinforcement learning algorithms commonly seek to optimize policies for solving one particular task. How should we explore an unknown dynamical system such that the estimated model allows us to solve multiple downstream tasks in a zero-shot manner? In this paper, we address this challenge, by developing an algorithm -- OPAX -- for active exploration. OPAX uses well-calibrated probabilistic models to quantify the epistemic uncertainty about the unknown dynamics. It optimistically---w.r.t. to plausible dynamics---maximizes the information gain between the unknown dynamics and state observations. We show how the resulting optimization problem can be reduced to an optimal control problem that can be solved at each episode using standard approaches. We analyze our algorithm for general models, and, in the case of Gaussian process dynamics, we give a sample complexity bound and show that the epistemic uncertainty converges to zero. In our experiments, we compare OPAX with other heuristic active exploration approaches on several environments. Our experiments show that OPAX is not only theoretically sound but also performs well for zero-shot planning on novel downstream tasks.

Many flexible structures are characterized by a small number of compliant modes , i.e., large-deformation paths that can be traversed with little mechanical effort, whereas resistance to other deformations is much stiffer. Predicting the compliant modes for a given flexible structure, however, is challenging. While linear eigenmodes capture the small-deformation behavior, they quickly divert into states of unrealistically high energy for larger displacements. Moreover, they are inherently unable to predict nonlinear phenomena such as buckling, stiffening, multistability, and contact. To address this limitation, we propose Nonlinear Compliant Modes —a physically principled extension of linear eigenmodes for large-deformation analysis. Instead of constraining the entire structure to deform along a given eigenmode, our method only prescribes the projection of the system’s state onto the linear mode while all other degrees of freedom follow through energy minimization. We evaluate the potential of our method on a diverse set of flexible structures, ranging from compliant mechanisms to topology-optimized joints and structured materials. As validated through experiments on physical prototypes, our method correctly predicts a broad range of nonlinear effects that linear eigenanalysis fails to capture.

Nonlinear metamaterials with tailored mechanical properties have applications in engineering, medicine, robotics, and beyond. While modeling their macromechanical behavior is challenging in itself, finding structure parameters that lead to ideal approximation of high-level performance goals is a challenging task. In this work, we propose Neural Metamaterial Networks (NMN)---smooth neural representations that encode the nonlinear mechanics of entire metamaterial families. Given structure parameters as input, NMN return continuously differentiable strain energy density functions, thus guaranteeing conservative forces by construction. Though trained on simulation data, NMN do not inherit the discontinuities resulting from topo-logical changes in finite element meshes. They instead provide a smooth map from parameter to performance space that is fully differentiable and thus well-suited for gradient-based optimization. On this basis, we formulate inverse material design as a nonlinear programming problem that leverages neural networks for both objective functions and constraints. We use this approach to automatically design materials with desired strain-stress curves, prescribed directional stiffness and Poisson ratio profiles. We furthermore conduct ablation studies on network nonlinearities and show the advantages of our approach compared to native-scale optimization.

Stripe patterns are ubiquitous in nature and everyday life. While the synthesis of these patterns has been thoroughly studied in the literature, their potential to control the mechanics of structured materials remains largely unexplored. In this work, we introduce Differentiable Stripe Patterns---a computational approach for automated design of physical surfaces structured with stripe-shaped bi-material distributions. Our method builds on the work by Knöppel and colleagues [2015] for generating globally-continuous and equally-spaced stripe patterns. To unlock the full potential of this design space, we propose a gradient-based optimization tool to automatically compute stripe patterns that best approximate macromechanical performance goals. Specifically, we propose a computational model that combines solid shell finite elements with XFEM for accurate and fully-differentiable modeling of elastic bi-material surfaces. To resolve non-uniqueness problems in the original method, we furthermore propose a robust formulation that yields unique and differentiable stripe patterns. We combine these components with equilibrium state derivatives into an end-to-end differentiable pipeline that enables inverse design of mechanical stripe patterns. We demonstrate our method on a diverse set of examples that illustrate the potential of stripe patterns as a design space for structured materials. Our simulation results are experimentally validated on physical prototypes.

Soft robotics offers unique advantages in manipulating fragile or deformable objects, human-robot interaction, and exploring inaccessible terrain. However, designing soft robots that produce large, targeted deformations is challenging. In this paper, we propose a new methodology for designing soft robots that combines optimization-based design with a simple and cost-efficient manufacturing process. Our approach is centered around the concept of robotic skins---thin fabrics with 3D-printed reinforcement patterns that augment and control plain silicone actuators. By decoupling shape control and actuation, our approach enables a simpler and cost-efficient manufacturing process. Unlike previous methods that rely on empirical design heuristics for generating desired deformations, our approach automatically discovers complex reinforcement patterns without any need for domain knowledge or human intervention. This is achieved by casting reinforcement design as a nonlinear constrained optimization problem and using a novel, three-field topology optimization approach tailored to fabrics with 3D-printed reinforcements. We demonstrate the potential of our approach by designing soft robotic actuators capable of various motions such as bending, contraction, twist, and combinations thereof. We also demonstrate applications of our robotic skins to robotic grasping with a soft three-finger gripper and locomotion tasks for a soft quadrupedal robot.

We present a method for computational modeling, mechanical characterization, and macro-scale simulation of discrete interlocking materials (DIM)---3D-printed chainmail fabrics made of quasi-rigid interlocking elements. Unlike conventional elastic materials for which deformation and restoring force are directly coupled, the mechanics of DIM are governed by contacts between individual elements that give rise to anisotropic deformation constraints. To model the mechanical behavior of these materials, we propose a computational approach that builds on three key components. ( a ): we explore the space of feasible deformations using native-scale simulations at the per-element level. ( b ): based on this simulation data, we introduce the concept of strain-space boundaries to represent deformation limits for in- and out-of-plane deformations, and ( c ): we use the strain-space boundaries to drive an efficient macro-scale simulation model based on homogenized deformation constraints. We evaluate our method on a set of representative discrete interlocking materials and validate our findings against measurements on physical prototypes.

We present a computational framework for planning the assembly sequence of bespoke frame structures. Frame structures are one of the most commonly used structural systems in modern architecture, providing resistance to gravitational and external loads. Building frame structures requires traversing through several partially built states. If the assembly sequence is planned poorly, these partial assemblies can exhibit substantial deformation due to self-weight, slowing down or jeopardizing the assembly process. Finding a good assembly sequence that minimizes intermediate deformations is an interesting yet challenging combinatorial problem that is usually solved by heuristic search algorithms. In this paper, we propose a new optimization-based approach that models sequence planning using a series of topology optimization problems. Our key insight is that enforcing temporal coherent constraints in the topology optimization can lead to sub-structures with small deformations while staying consistent with each other to form an assembly sequence. We benchmark our algorithm on a large data set and show improvements in both performance and computational time over greedy search algorithms. In addition, we demonstrate that our algorithm can be extended to handle assembly with static or dynamic supports. We further validate our approach by generating a series of results in multiple scales, including a real-world prototype with a mixed reality assistant using our computed sequence and a simulated example demonstrating a multi-robot assembly application.

Multi-Robot Systems (MRS) present many advantages over single robots, e.g. improved stability and payload capacity. Being able to operate or teleoperate these systems is therefore of high interest in industries such as construction or logistics. However, controlling the collective motion of a MRS can place a significant cognitive burden on the operator. We present a Mixed Reality (MR) control interface, which allows an operator to specify payload target poses for a MRS in real-time, while effectively keeping the system away from unfavorable configurations. To this end, we solve the inverse kinematics problem for each arm individually and leverage redundant degrees of freedom to optimize for a secondary objective. Using the manipulability index as a secondary objective in particular, allows us to significantly improve the tracking and singularity avoidance capabilities of our MRS in comparison to the unoptimized scenario. This enables more secure and intuitive teleoperation. We simulate and test our approach on different setups and over different input trajectories, and analyse the convergence properties of our method. Finally, we show that the method also works well when deployed on to a dual-arm ABB YuMi robot.

This work presents a motion capture-driven locomotion controller for quadrupedal robots that replicates the non-periodic footsteps and subtle body movement of animal motions. We adopt a nonlinear model predictive control (NMPC) formulation that generates optimal base trajectories and stepping locations. By optimizing both footholds and base trajectories, our controller effectively tracks retargeted animal motions with natural body movements and highly irregular strides. We demonstrate our approach with prerecorded animal motion capture data. In simulation and hardware experiments, our motion controller enables quadrupedal robots to robustly reproduce fundamental characteristics of a target animal motion regardless of the significant morphological disparity.

A central aspect of robotic motion planning is collision avoidance, where a multitude of different approaches are currently in use. Optimization-based motion planning is one method, that often heavily relies on distance computations between robots and obstacles. These computations can easily become a bottleneck, as they do not scale well with the complexity of the robots or the environment. To improve performance, many different methods suggested to use collision primitives, i.e. simple shapes that approximate the more complex rigid bodies, and that are simpler to compute distances to and from. However, each pair of primitives requires its own specialized code, and certain pairs are known to suffer from numerical issues. In this paper, we propose an easy-to-use, unified treatment of a wide variety of primitives. We formulate distance computation as a minimization problem, which we solve iteratively. We show how to take derivatives of this minimization problem, allowing it to be seamlessly integrated into a trajectory optimization method. We demonstrate that the resulting method can be used to plan smooth and collision-free paths based on a variety of single- and multi-robot scenarios with different obstacles.

Garments with the ability to provide kinesthetic force-feedback on-demand can augment human capabilities in a non-obtrusive way, enabling numerous applications in VR haptics, motion assistance, and robotic control. However, designing such garments is a complex, and often manual task, particularly when the goal is to resist multiple motions with a single design. In this work, we propose a computational pipeline for designing connecting structures between active components—one of the central challenges in this context. We focus on electrostatic (ES) clutches that are compliant in their passive state while strongly resisting elongation when activated. Our method automatically computes optimized connecting structures that efficiently resist a range of pre-defined body motions on demand. We propose a novel dual-objective optimization approach to simultaneously maximize the resistance to motion when clutches are active, while minimizing resistance when inactive. We demonstrate our method on a set of problems involving different body sites and a range of motions. We further fabricate and evaluate a subset of our automatically created designs against manually created baselines using mechanical testing and in a VR pointing study.

The rigid-block equilibrium (RBE) method uses a penalty formulation to measure structural infeasibility or to guide the design of stable discrete-element assemblies from unstable geometry. However, RBE is a purely force-based formulation, and it incorrectly describes stability when complex interface geometries are involved. To overcome this issue, this paper introduces the coupled rigid-block analysis (CRA) method, a more robust approach building upon RBE’s strengths. The CRA method combines equilibrium and kinematics in a penalty formulation in a nonlinear programming problem. An extensive benchmark campaign is used to show how CRA enables accurate modelling of complex three-dimensional discrete-element assemblies formed by rigid blocks. In addition, an interactive stability-aware design process to guide user design towards structurally-sound assemblies is proposed. Finally, the potential of our method for real-world problems are demonstrated by designing complex and scaffolding-free physical models.

Abstract Strain limiting is a widely used approach for simulating biphasic materials such as woven textiles and biological tissue that exhibit a soft elastic regime followed by a hard deformation limit. However, existing methods are either based on slowly converging local iterations, or offer no guarantees on convergence. In this work, we propose a new approach to strain limiting based on second order cone programming (SOCP). Our work is based on the key insight that upper bounds on per‐triangle deformations lead to convex quadratic inequality constraints. Though nonlinear, these constraints can be reformulated as inclusion conditions on convex sets, leading to a second order cone programming problem—a convex optimization problem that a) is guaranteed to have a unique solution and b) allows us to leverage efficient conic programming solvers. We first cast strain limiting with anisotropic bounds on stretching as a quadratically constrained quadratic program (QCQP), then show how this QCQP can be mapped to a second order cone programming problem. We further propose a constraint reflection scheme and empirically show that it exhibits superior energy‐preservation properties compared to conventional end‐of‐step projection methods. Finally, we demonstrate our prototype implementation on a set of examples and illustrate how different deformation limits can be used to model a wide range of material behaviors.

We present a sparse Gauss-Newton solver for accelerated sensitivity analysis with applications to a wide range of equilibrium-constrained optimization problems. Dense Gauss-Newton solvers have shown promising convergence rates for inverse problems, but the cost of assembling and factorizing the associated matrices has so far been a major stumbling block. In this work, we show how the dense Gauss-Newton Hessian can be transformed into an equivalent sparse matrix that can be assembled and factorized much more efficiently. This leads to drastically reduced computation times for many inverse problems, which we demonstrate on a diverse set of examples. We furthermore show links between sensitivity analysis and nonlinear programming approaches based on Lagrange multipliers and prove equivalence under specific assumptions that apply for our problem setting.

Recent monocular human performance capture approaches have shown compelling dense tracking results of the full body from a single RGB camera. However, existing methods either do not estimate clothing at all or model cloth deformation with simple geometric priors instead of taking into account the underlying physical principles. This leads to noticeable artifacts in their reconstructions, e.g. baked-in wrinkles, implausible deformations that seemingly defy gravity, and intersections between cloth and body. To address these problems, we propose a person-specific, learning-based method that integrates a simulation layer into the training process to provide for the first time physics supervision in the context of weakly supervised deep monocular human performance capture. We show how integrating physics into the training process improves the learned cloth deformations, allows modeling clothing as a separate piece of geometry, and largely reduces cloth-body intersections. Relying only on weak 2D multi-view supervision during training, our approach leads to a significant improvement over current state-of-the-art methods and is thus a clear step towards realistic monocular capture of the entire deforming surface of a clothed human.

Current reinforcement learning (RL) methods use simulation models as simple black-box oracles. In this paper, with the goal of improving the performance exhibited by RL algorithms, we explore a systematic way of leveraging the additional information provided by an emerging class of differentiable simulators. Building on concepts established by Deterministic Policy Gradients (DPG) methods, the neural network policies learned with our approach represent deterministic actions. In a departure from standard methodologies, however, learning these policies does not hinge on approximations of the value function that must be learned concurrently in an actor-critic fashion. Instead, we exploit differentiable simulators to directly compute the analytic gradient of a policy’s value function with respect to the actions it outputs. This, in turn, allows us to efficiently perform locally optimal policy improvement iterations. Compared against other state-of-the-art RL methods, we show that with minimal hyper-parameter tuning our approach consistently leads to better asymptotic behavior across a set of payload manipulation tasks that demand a high degree of accuracy and precision.

Teleoperation provides a way for human operators to guide robots in situations where full autonomy is challenging or where direct human intervention is required. It can also be an important tool to teach robots in order to achieve autonomous behaviour later on. The increased availability of collaborative robot arms and Virtual Reality (VR) devices, provides ample opportunity for development of novel teleoperation methods. Since robot arms are often kinematically different from human arms, mapping human motions to a robot in real-time is not trivial. Additionally, a human operator might steer the robot arm toward singularities or its workspace limits, which can lead to undesirable behaviour. This is further accentuated for the orchestration of multiple robots. In this paper, we present a VR interface targeted to multi-arm payload manipulation, which can closely match real-time input motion. Allowing the user to manipulate the payload rather than mapping their motions to individual arms we are able to simultaneously guide multiple collaborative arms. By releasing a single rotational degree of freedom, and by using a local optimization method, we can improve each arm’s manipulability index, which in turn lets us avoid kinematic singularities and workspace limitations. We apply our approach to predefined trajectories as well as real-time teleoperation on different robot arms and compare performance in terms of end-effector position error and relevant joint motion metrics.

Teleoperating robotic arms is a challenging task that requires years of training to master. It is mentally demanding, as the operator must internally compute transformations, or rely on muscle memory, to perform even the simplest tasks. Alternative methods that rely on embodiment –the immersive, first person experience of controlling the robot from its point of view are recently becoming more popular, thanks to the emergence of mixed reality devices. These methods create an intuitive experience by tracking the users motions, and retargetting them to the robot. However, even recent hardware fails at achieving total immersion, due to inherent discrepancies such as latency, imperfect tracking, and the differences between human and robot motor systems. Thus, performing even simple pick-and-place tasks with these systems, while more intuitive, is still cumbersome, and far from the level of human performance.In this paper we propose an immersive system that aims to bridge this gap. The system tracks the user’s motion and retargets them to the robot as usual, but it also detects the user’s intent, that is, the task they wish to perform. Based on this knowledge, the system can autocorrect the motion when it is about to fail, in a seamless manner, such that the task is successfully performed. We evaluate the efficacy of our autocorrection system in a user study. The results show a statistically significant performance improvement in terms of operation accuracy and time.

We combine Boston Dynamics Spot®with a light-weight, external robot arm to perform dynamic grasping maneuvers. While Spot is a reliable, robust and easy-to-control mobile robot, these highly desirable qualities come with the price that the control access granted to the user is restricted. Consequently Spot’s behavior must largely be treated as a black box, which causes difficulties when combined with a moving payload such as a robotic arm. We overcome the arising challenges by building a model of the combined platform, fitting the corresponding model parameters using experimental data and a straight-forward optimization framework. We use this model to generate control commands for the physical platform using trajectory optimization. We demonstrate that even with a simple model, and control trajectories deployed in a feed-forward manner, the combined platform is capable of executing grasping tasks in a dynamic fashion. Furthermore, we show how the platform can use the additional degrees of freedom of the legs to extend the reachability of the arm.

In this paper, we explore the challenge of generating animal-like walking motions for legged robots. To this end, we propose a versatile and robust control pipeline that combines a state-of-the-art model-based controller with a data-driven technique that is commonly used in computer animation. We demonstrate the efficacy of our control framework on a variety of quadrupedal robots in simulation. We show, in particular, that our approach can automatically reproduce key characteristics of animal motions, including speed-specific gaits, unscripted footfall patterns for nonperiodic motions, and natural small variations in overall body movements.

We present a control-aware design optimization method for quadrupedal robots. In particular, we show that it is possible to analytically differentiate typical, inverse dynamics-based whole body controllers with respect to design parameters, and that gradient-based methods can be used to efficiently improve an initial morphological design according to well-established metrics. We apply our design optimization method to various types of quadrupedal robots, including designs that feature closed kinematic chains. The methodology we present enables a principled comparison of different types of optimized legged robot designs. Our experiments, for example, suggest that mechanically-coupled three-link leg designs present notable advantages in terms of performance and efficiency over the common two-link leg designs used in most quadrupedal robots today.

Recent advances in implicit neural representations show great promise when it comes to generating numerical solutions to partial differential equations. Compared to conventional alternatives, such representations employ parameterized neural networks to define, in a mesh-free manner, signals that are highly-detailed, continuous, and fully differentiable. In this work, we present a novel machine learning approach for topology optimization---an important class of inverse problems with high-dimensional parameter spaces and highly nonlinear objective landscapes. To effectively leverage neural representations in the context of mesh-free topology optimization, we use multilayer perceptrons to parameterize both density and displacement fields. Our experiments indicate that our method is highly competitive for minimizing structural compliance objectives, and it enables self-supervised learning of continuous solution spaces for topology optimization problems.

This paper presents an interactive design system that allows the user to create and fabricate stylized sculptures in water-based clay, using a standard 6-axis robot arm. This system facilitates the materialization of abstract design intentions into clay, through the algorithmic formulation of sculpting styles, the optimal path planning of the sculpting toolpaths, and a subtractive robotic fabrication process using customized tools. Unlike other precision-driven fabrication technologies, the authors embrace artistic uncertainty by conducting manual and robotic sculpting experiments and incorporating prominent parameters that affect the fabrication quality. The versatility of the described approach is demonstrated by designing a series of sculpting styles over a wide range of 3D models and robotically fabricating them in clay. Additionally, the paper explores various strategies for designing stylized robotic sculpting patterns by generating toolpaths informed by different techniques.

We present an algorithmic approach to designing animatronic figures - expressive robotic characters whose movements are driven by a large number of actuators. The input to our design system provides a high-level specification of the space of motions the character should be able to perform. The output consists of a fully functional mechatronic blueprint. We cast the design task as a search problem in a vast combinatorial space of possible solutions. To find an optimal design in this space, we propose an efficient best-first search algorithm that is guided by an admissible heuristic. The objectives guiding the search process demand that the design remains free of singularities and self-collisions at any point in the high-dimensional space of motions the character is expected to be able to execute. To identify worst-case self-collision scenarios for multi degree-of-freedom closed-loop mechanisms, we additionally develop an elegant technique inspired by the concept of adversarial attacks. We demonstrate the efficacy of our approach by creating designs for several animatronic figures of varying complexity.

We present a novel class of robots belonging to Constrained Collaborative Mobile Agents (CCMA) family which consists of ground mobile bases with non-holonomic constraints. Moreover, these mobile robots are constrained by closed-loop kinematic chains consisting of revolute joints which can be either passive or actuated. We also describe a novel trajectory optimization method which is general with respect to number of mobile robots, topology of the closed- loop kinematic chains and placement of the actuators at the revolute joints. We also extend the standalone trajectory optimization method to optimize concurrently the design parameters and the control policy. We describe various CCMA system examples, in simulation, differing in design, topology, number of mobile robots and actuation space. The simulation results for standalone trajectory optimization with fixed design parameters is presented for CCMA system examples. We also show how this method can be used for tasks other than end-effector positioning such as internal collision avoidance and external obstacle avoidance. The concurrent design and control policy optimization is demonstrated, in simulations, to increase the CCMA system workspace and manipulation capabilities. Finally, the trajectory optimization method is validated in experiments through two 4-DOF prototypes consisting of 3 tracked mobile bases.

We consider the design of under-actuated articulated mechanism that are able to maintain stable static balance. Our method augments an user-provided design with counter-weights whose mass and attachment locations are automatically computed. The optimized counterweights adjust the center of gravity such that, for bounded external perturbations, the mechanism returns to its original configuration. Using our sketch-based system, we present several examples illustrating a wide range of user-provided designs can be successfully converted into statically-balanced mechanisms. We further validate our results with a set of physical prototypes.

Hot-wire cutting is a subtractive fabrication technique used to carve foam and similar materials. Conventional machines rely on straight wires and are thus limited to creating piecewise ruled surfaces. In this work, we propose a method that exploits a dual-arm robot setup to actively control the shape of a flexible, heated rod as it cuts through the material. While this setting offers great freedom of shape, using it effectively requires concurrent reasoning about three tightly coupled sub-problems: 1) modeling the way in which the shape of the rod and the surface it sweeps are governed by the robot's motions; 2) approximating a target shape through a sequence of surfaces swept by the equilibrium shape of an elastic rod; and 3) generating collision-free motion trajectories that lead the robot to create desired sweeps with the deformable tool. We present a computational framework for robotic hot wire cutting that addresses all three sub-problems in a unified manner. We evaluate our approach on a set of simulated results and physical artefacts generated with our robotic fabrication system.

We present a differentiable dynamics solver that is able to handle frictional contact for rigid and deformable objects within a unified framework. Through a principled mollification of normal and tangential contact forces, our method circumvents the main difficulties inherent to the non-smooth nature of frictional contact. We combine this new contact model with fully-implicit time integration to obtain a robust and efficient dynamics solver that is analytically differentiable. In conjunction with adjoint sensitivity analysis, our formulation enables gradient-based optimization with adaptive trade-offs between simulation accuracy and smoothness of objective function landscapes. We thoroughly analyse our approach on a set of simulation examples involving rigid bodies, visco-elastic materials, and coupled multi-body systems. We furthermore showcase applications of our differentiable simulator to parameter estimation for deformable objects, motion planning for robotic manipulation, trajectory optimization for compliant walking robots, as well as efficient self-supervised learning of control policies.

We present a computational method for designing compliant mechanical systems that exhibit large-amplitude oscillations. The technical core of our approach is an optimization-driven design tool that combines sensitivity analysis for optimization with the Harmonic Balance Method for simulation. By establishing dynamic force equilibrium in the frequency domain, our formulation avoids the major limitations of existing alternatives: it handles nonlinear forces, side-steps any transient process, and automatically produces periodic solutions. We introduce design objectives for amplitude optimization and trajectory matching that enable intuitive high-level authoring of large-amplitude motions. Our method can be applied to many types of mechanical systems, which we demonstrate through a set of examples involving compliant mechanisms, flexible rod networks, elastic thin shell models, and multi-material solids. We further validate our approach by manufacturing and evaluating several physical prototypes.

Expressive robots are useful in many contexts, from industrial to entertainment applications. However, designing expressive robot behaviors requires editing a large number of unintuitive control parameters. We present an interactive, data-driven system that allows editing of these complex parameters in a semantic space. Our system combines a physics-based simulation that captures the robot's motion capabilities, and a crowd-powered framework that extracts relationships between the robot's motion parameters and the desired semantic behavior. These relationships enable mixed-initiative exploration of possible robot motions. We specifically demonstrate our system in the context of designing emotionally expressive behaviors. A user-study finds the system to be useful for more quickly developing desirable robot behaviors, compared to manual parameter editing.

We present a concept of constrained collaborative mobile agents (CCMA) system, which consists of multiple wheeled mobile agents constrained by a passive kinematic chain. This mobile robotic system is modular in nature, the passive kinematic chain can be easily replaced with different designs and morphologies for different functions and task adaptability. Depending solely on the actuation of the mobile agents, this mobile robotic system can manipulate or position an end-effector. However, the complexity of the system due to presence of several mobile agents, passivity of the kinematic chain and the nature of the constrained collaborative manipulation requires development of an optimization framework. We therefore present an optimization framework for forward simulation and kinematic control of this system. With this optimization framework, the number of deployed mobile agents, actuation schemes, the design and morphology of the passive kinematic chain can be easily changed, which reinforces the modularity and collaborative aspects of the mobile robotic system. We present results, in simulation, for spatial 4-DOF to 6-DOF CCMA system examples. Finally, we present experimental quantitative results for two different fabricated 4-DOF prototypes, which demonstrate different actuation schemes, control and collaborative manipulation of an end-effector.

This paper presents a method for optimizing visco-elastic material parameters of a finite element simulation to best approximate the dynamic motion of real-world soft objects. We compute the gradient with respect to the material parameters of a least-squares error objective function using either direct sensitivity analysis or an adjoint state method. We then optimize the material parameters such that the simulated motion matches real-world observations as closely as possible. In this way, we can directly build a useful simulation model that captures the visco-elastic behaviour of the specimen of interest. We demonstrate the effectiveness of our method on various examples such as numerical coarsening, custom-designed objective functions, and of course real-world flexible elastic objects made of foam or 3D printed lattice structures, including a demo application in soft robotics.

Creating animations for robotic characters is very challenging due to the constraints imposed by their physical nature. In particular, the combination of fast motions and unavoidable structural deformations leads to mechanical oscillations that negatively affect their performances. Our goal is to automatically transfer motions created using traditional animation software to robotic characters while avoiding such artifacts. To this end, we develop an optimization-based, dynamics-aware motion retargeting system that adjusts an input motion such that visually salient low-frequency, large amplitude vibrations are suppressed. The technical core of our animation system consists of a differentiable dynamics simulator that provides constraint-based two-way coupling between rigid and flexible components. We demonstrate the efficacy of our method through experiments performed on a total of five robotic characters including a child-sized animatronic figure that features highly dynamic drumming and boxing motions.

We present a computational framework for robotic animation of real-world string puppets. Also known as marionettes, these articulated figures are typically brought to life by human puppeteers. The puppeteer manipulates rigid handles that are attached to the puppet from above via strings. The motions of the marionette are therefore governed largely by gravity, the pull forces exerted by the strings, and the internal forces arising from mechanical articulation constraints. This seemingly simple setup conceals a very challenging and nuanced control problem, as marionettes are, in fact, complex coupled pendulum systems. Despite this, in the hands of a master puppeteer, marionette animation can be nothing short of mesmerizing. Our goal is to enable autonomous robots to animate marionettes with a level of skill that approaches that of human puppeteers. To this end, we devise a predictive control model that accounts for the dynamics of the marionette and kinematics of the robot puppeteer. The input to our system consists of a string puppet design and a target motion, and our trajectory planning algorithm computes robot control actions that lead to the marionette moving as desired. We validate our methodology through a series of experiments conducted on an array of marionette designs and target motions. These experiments are performed both in simulation and using a physical robot, the human-sized, dual arm ABB YuMi ® IRB 14000.

In our everyday life, we interact with and benefit from objects with a wide range of material properties. In contrast, personal fabrication machines (e.g., desktop 3D printers) currently only support a much smaller set of materials. Our goal is to close the gap between current limitations and the future of multi-material printing by enabling people to explore the reuse of material from everyday objects into their custom designs. To achieve this, we develop a library of embeddables--everyday objects that can be cut, worked and embedded into 3D printable designs. We describe a design space that characterizes the geometric and material properties of embeddables. We then develop Medley---a design tool whereby users can import a 3D model, search for embeddables with desired material properties, and interactively edit and integrate their geometry to fit into the original design. Medley also supports the final fabrication and embedding process, including instructions for carving or cutting the objects, and generating optimal paths for inserting embeddables. To validate the expressiveness of our library, we showcase numerous examples augmented by embeddables that go beyond the objects' original printed materials.

Low-cost fabrication machines (e.g., 3D printers) offer the promise of creating custom-designed objects by a range of users. To maximize performance, generative design methods such as topology optimization can automatically optimize properties of a design based on high-level specifications. Though promising, such methods require people to map their design ideas--often unintuitively--to a small number of mathematical input parameters, and the relationship between those parameters and a generated design is often unclear, making it difficult to iterate a design. We present Forte, a sketch-based, real-time interactive tool for people to directly express and iterate on their designs via 2D topology optimization. Users can ask the system to add structures, provide a variation with better performance, or optimize internal material layouts. Users can globally control how much to 'deviate' from the initial sketch, or perform local suggestive editing, which interactively prompts the system to update based on the new information. Design sessions with 10 participants demonstrate that Forte empowers designers to create and explore a range of optimized designs with custom forms and styles.

In this paper, we address the challenge of robotic manipulation of elastically deforming objects. To this end, we model elastic objects using the Finite Element Method. Through a quasi-static assumption, we leverage sensitivity analysis to mathematically model how changes in the robot's configuration affect the deformed shape of the object being manipulated. This enables an interactive, simulation-based control methodology, wherein user-specified deformations for the elastic objects are automatically mapped to joint angle commands. The optimization formulation we introduce is general, operates directly within a robot's workspace and can readily incorporate joint limits as well as collision avoidance between the links. We validate our control methodology on a YuMi® IRB 14000, which we use to manipulate a variety of elastic objects.

From smart toys and household appliances to personal robots, electromechanical devices play an increasingly important role in our daily lives. Rather than relying on gadgets that are mass-produced, our goal is to enable casual users to custom-design such devices based on their own needs and preferences. To this end, we present a computational design system that leverages the power of digital fabrication and the emergence of affordable electronics such as sensors and microcontrollers. The input to our system consists of a 3D representation of the desired device's shape, and a set of user-preferred off-the-shelf components. Based on this input, our method generates an optimized, 3D printable enclosure that can house the required components. To create these designs automatically, we formalize a new spatio-temporal model that captures the entire assembly process, including the placement of the components within the device, mounting structures and attachment strategies, the order in which components must be inserted, and collision-free assembly paths. Using this model as a technical core, we then leverage engineering design guidelines and efficient numerical techniques to optimize device designs. In a user study, which also highlights the challenges of designing such devices, we find our system to be effective in reducing the entry barriers faced by casual users in creating such devices. We further demonstrate the versatility of our approach by designing and fabricating three devices with diverse functionalities.

Abstract We present a computational approach to designing transformables, physical characters that can shape‐shift to take on vastly different forms. The design process begins with a morphological description of an input character and a target object that it should transform into. Guided by a set of objectives that model the core attributes of desirable transformable designs, optimized embeddings are interactively generated. Intuitively, embeddings represent tightly folded character configurations that fit within the target object. From any feasible embedding, skin meshes are then generated for each body part of the character. The process for generating these 3D models is based on a segmentation of the target object, which is achieved through a growth‐based model applied to a multiple level set representation of the transformable. A set of transformation‐aware post‐processing algorithms ensure the feasibility of the final designs. Building on this technical core, our computational design system provides many opportunities for users to inject their intuition and personal preferences into the process of creating transformables, while shielding them from tasks that are challenging and tedious. As a result, they can intuitively explore the vast space of design possibilities. We demonstrated the effectiveness of our computational approach by creating a variety of transformable designs, three of which we fabricate.

We present a computation-driven approach to design optimization and motion synthesis for robotic creatures that locomote using arbitrary arrangements of legs and wheels. Through an intuitive interface, designers first create unique robots by combining different types of servomotors, 3D printable connectors, wheels and feet in a mix-and-match manner. With the resulting robot as input, a novel trajectory optimization formulation generates walking, rolling, gliding and skating motions. These motions emerge naturally based on the components used to design each individual robot. We exploit the particular structure of our formulation and make targeted simplifications to significantly accelerate the underlying numerical solver without compromising quality. This allows designers to interactively choreograph stable, physically-valid motions that are agile and compelling. We furthermore develop a suite of user-guided, semi-automatic, and fully-automatic optimization tools that enable motion-aware edits of the robot's physical structure. We demonstrate the efficacy of our design methodology by creating a diverse array of hybrid legged/wheeled mobile robots which we validate using physics simulation and through fabricated prototypes.

We present the first computational approach that can transform three-dimensional (3D) meshes, created by traditional modeling programs, directly into instructions for a computer-controlled knitting machine. Knitting machines are able to robustly and repeatably form knitted 3D surfaces from yarn but have many constraints on what they can fabricate. Given user-defined starting and ending points on an input mesh, our system incrementally builds a helix-free, quad-dominant mesh with uniform edge lengths, runs a tracing procedure over this mesh to generate a knitting path, and schedules the knitting instructions for this path in a way that is compatible with machine constraints. We demonstrate our approach on a wide range of 3D meshes.

Elastically deforming wire structures are lightweight, durable, and can be bent within minutes using CNC bending machines. We present a computational technique for the design of kinetic wire characters, tailored for fabrication on consumer-grade hardware. Our technique takes as input a network of curves or a skeletal animation, then estimates a cable-driven, compliant wire structure which matches user-selected targets or keyframes as closely as possible. To enable large localized deformations, we shape wire into functional spring-like entities at a discrete set of locations. We first detect regions where changes to local stiffness properties are needed, then insert bendable entities of varying shape and size. To avoid a discrete optimization, we first optimize stiffness properties of generic, non-fabricable entities which capture well the behavior of our bendable designs. To co-optimize stiffness properties and cable forces, we formulate an equilibrium-constrained minimization problem, safeguarding against inelastic deformations. We demonstrate our method on six fabricated examples, showcasing rich behavior including large deformations and complex, spatial motion.

We present a novel computational approach to design the robotic devices from high-level motion specifications. Our computational system uses a library of modular components-actuators, mounting brackets, and connectors-to define the space of possible robot designs. The process of creating a new robot begins with a set of input trajectories that specify how its end effectors and/or body should move. By searching through the combinatorial set of possible arrangements of modular components, our method generates a functional, as-simple-as-possible robotic device that is capable of tracking the input motion trajectories. To significantly improve the efficiency of this discrete optimization process, we propose a novel heuristic that guides the search for appropriate designs. Briefly, our heuristic function estimates how much an intermediate robot design needs to change before it becomes able to execute the target motion trajectories. We demonstrate the effectiveness of our computational design method by automatically creating a variety of robotic manipulators and legged robots. To generate these results, we define our own robotic kit that includes off-the-shelf actuators and 3-D printable connectors. We validate our results by fabricating two robotic devices designed with our method.

We present a computational design system that allows novices and experts alike to easily create custom robotic devices using modular electromechanical components. The core of our work consists of a design abstraction that models the way in which these components can be combined to form complex robotic systems. We use this abstraction to develop a visual design environment that enables an intuitive exploration of the space of robots that can be created using a given set of actuators, mounting brackets and 3d-printable components. Our computational system also provides support for design auto-completion operations, which further simplifies the task of creating robotic devices. Once robot designs are finished, they can be tested in physical simulations and iteratively improved until they meet the individual needs of their users. We demonstrate the versatility of our computational design system by creating an assortment of legged and wheeled robotic devices. To test the physical feasibility of our designs, we fabricate a wheeled device equipped with a 5-DOF arm and a quadrupedal robot.

Quadrupedal locomotion can be described as a constrained optimization problem that is very hard to solve due to the high dimensional, nonlinear and non-smooth system dynamics. In this paper, we propose a formulation that can be solved within few seconds using sequential quadratic programming. This method considers only a simplified model that just sufficiently represents the system dynamics. The output is a very coarse plan, which can be accurately and robustly followed on a real system using hierarchical whole-body control combined with inverted pendulum-based reactive stepping. Using the fully torque controllable quadrupedal robot ANYmal, we present successful experiments for walking, trotting, and gait transitions even under substantial external disturbances.

We present a class of tendon-actuated soft robots, which promise to be low-cost and accessible to non-experts. The fabrication techniques we introduce are largely based on traditional techniques for fabricating plush toys, and so we term the robots created using our approach “plush robots.” A plush robot moves by driving internal winches that pull in (or let out) tendons routed through its skin. We provide a forward simulation model for predicting a plush robot's deformation behavior given some contractions of its internal winches. We also leverage this forward model for use in an interactive control scheme, in which the user provides a target pose for the robot, and optimal contractions of the robot's winches are automatically computed in real-time. We fabricate two examples to demonstrate the use of our system, and also discuss the design challenges inherent to plush robots.

Modular robots can be rapidly reconfigured into customized articulated legged morphologies capable of mobile manipulation and inspection. However, current gait generation methods do not keep pace with the speed of physical reconfiguration. This work focuses on quickly creating gaits for modular legged robots. We build on a recent method that uses trajectory optimization to design quasi-static gaits given only robot geometry and foot contact patterns. We develop methods to automatically generate contact patterns for new gaits and transitions between them. We show the utility of these methods applied to robots with many limbs, such that limbs can be fluidly reassigned to locomotion, manipulation, or inspection tasks, or to adapt gaits to hardware failures online. We demonstrate gait and transition generation with our modular hexapod and dodecapod robots. The robots switch between gaits that use all limbs for locomotion and those that leave some limbs free to pick up objects or position a camera.

We present a computational approach to creating animated plushies, soft robotic plush toys specifically-designed to reenact user-authored motions. Our design process is inspired by muscular hydrostat structures, which drive highly versatile motions in many biological systems. We begin by instrumenting simulated plush toys with a large number of small, independently-actuated, virtual muscle-fibers. Through an intuitive posing interface, users then begin animating their plushie. A novel numerical solver, reminiscent of inverse-kinematics, computes optimal contractions for each muscle-fiber such that the soft body of the plushie deforms to best match user input. By analyzing the co-activation patterns of the fibers that contribute most to the plushie's motions, our design system generates physically-realizable winch-tendon networks. Winch-tendon networks model the motorized cable-driven actuation mechanisms that drive the motions of our real-life plush toy prototypes. We demonstrate the effectiveness of our computational approach by co-designing motions and actuation systems for a variety of physically-simulated and fabricated plushies.

We present a computational tool for designing compliant mechanisms. Our method takes as input a conventional, rigidly-articulated mechanism defining the topology of the compliant design. This input can be both planar or spatial, and we support a number of common joint types which, whenever possible, are automatically replaced with parameterized flexures. As the technical core of our approach, we describe a number of objectives that shape the design space in a meaningful way, including trajectory matching, collision avoidance, lateral stability, resilience to failure, and minimizing motor torque. Optimal designs in this space are obtained as solutions to an equilibrium-constrained minimization problem that we solve using a variant of sensitivity analysis. We demonstrate our method on a set of examples that range from simple four-bar linkages to full-fledged animatronics, and verify the feasibility of our designs by manufacturing physical prototypes.

Telescoping structures are valuable for a variety of applications where mechanisms must be compact in size and yet easily deployed. So far, however, there has been no systematic study of the types of shapes that can be modeled by telescoping structures, nor practical tools for telescopic design. We present a novel geometric characterization of telescoping curves, and explore how free-form surfaces can be approximated by networks of such curves. In particular we consider piecewise helical space curves with torsional impulses, which significantly generalize the linear telescopes found in typical engineering designs. Based on this principle we develop a system for computational design and fabrication which allows users to explore the space of telescoping structures; inputs to our system include user sketches or arbitrary meshes, which are then converted to a curve skeleton. We prototype applications in animation, fabrication, and robotics, using our system to design a variety of both simulated and fabricated examples.

The papers in this special section were presented at the 14th Annual ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA 2015), which was held in Los Angeles, California on August 7-9, 2015.

The design of legged robots is often inspired by animals evolved to excel at different tasks. However, while mimicking morphological features seen in nature can be very powerful, robots may need to perform motor tasks that their living counterparts do not. In the absence of designs that can be mimicked, an alternative is to resort to mathematical models that allow the relationship between a robot's form and function to be explored. In this paper, we propose such a model to co-design the motion and leg configurations of a robot such that a measure of performance is optimized. The framework begins by planning trajectories for a simplified model consisting of the center of mass and feet. The framework then optimizes the length of each leg link while solving for associated full-body motions. Our model was successfully used to find optimized designs for legged robots performing tasks that include jumping, walking, and climbing up a step. Although our results are preliminary and our analysis makes a number of simplifying assumptions, our findings indicate that the cost function, the sum of squared joint torques over the duration of a task, varies substantially as the design parameters change.

The widespread availability of high-quality motion capture data and the maturity of solutions to animate virtual characters has paved the way for the next generation of interactive virtual worlds exhibiting intricate interactions between characters and the environments they inhabit. However, current motion synthesis techniques have not been designed to scale with complex environments and contact-rich motions, requiring environment designers to manually embed motion semantics in the environment geometry in order to address online motion synthesis. This paper presents an automated approach for analyzing both motions and environments in order to represent the different ways in which an environment can afford a character to move. We extract the salient features that characterize the contact-rich motion repertoire of a character and detect valid transitions in the environment where each of these motions may be possible, along with additional semantics that inform which surfaces of the environment the character may use for support during the motion. The precomputed motion semantics can be easily integrated into standard navigation and animation pipelines in order to greatly enhance the motion capabilities of virtual characters. The computational efficiency of our approach enables two additional applications. Environment designers can interactively design new environments and get instant feedback on how characters may potentially interact, which can be used for iterative modeling and refinement. End users can dynamically edit virtual worlds and characters will automatically accommodate the changes in the environment in their movement strategies.

Everyday tools and objects often need to be customized for an unplanned use or adapted for specific user, such as adding a bigger pull to a zipper or a larger grip for a pen. The advent of low-cost 3D printing offers the possibility to rapidly construct a wide range of such adaptations. However, while 3D printers are now affordable enough for even home use, the tools needed to design custom adaptations normally require skills that are beyond users with limited 3D modeling experience.

We present a computational tool for designing ornamental curve networks---structurally-sound physical surfaces with user-controlled aesthetics. In contrast to approaches that leverage texture synthesis for creating decorative surface patterns, our method relies on user-defined spline curves as central design primitives. More specifically, we build on the physically-inspired metaphor of an embedded elastic curve that can move on a smooth surface, deform, and connect with other curves. We formalize this idea as a globally coupled energy-minimization problem, discretized with piece-wise linear curves that are optimized in the parametric space of a smooth surface. Building on this technical core, we propose a set of interactive design and editing tools that we demonstrate on manually-created layouts and semi-automated deformable packings. In order to prevent excessive compliance, we furthermore propose a structural analysis tool that uses eigenanalysis to identify potentially large deformations between geodesically-close curves and guide the user in strengthening the corresponding regions. We used our approach to create a variety of designs in simulation, validated with a set of 3D-printed physical prototypes.

Quadrupedal locomotion on sloped terrains poses different challenges than walking in a mostly flat environment. The robot's configuration needs to be explicitly controlled in order to avoid slipping and kinematic limits. To this end, information about the terrain's inclination is required for carefully planning footholds, the pose of the main body, and modulation of the ground reaction forces. This is even more important for dynamic trotting, as only two support legs are available to compensate for gravity and drive a desired motion. We propose a reliable method for estimating the parameters of the terrain quadrupedal robots move on, in the face of limited perception capabilities and drifting robot pose estimates. By fusing inertial measurements, kinematic data from joint encoders and contact information from force sensors, the local inclination can be robustly estimated and used to optimize the contact forces to reduce slippage. The estimated terrain information, namely the pitch and roll angles of the ground plane, is exploited in an extended version of our previous model-based control approach. Our improved control framework enabled StarlETH, a medium-sized, fully autonomous, torque-controllable quadrupedal robot, to trot on slopes of up to 21°.

One powerful aspect of 3D printing is its ability to extend, repair, or more generally modify everyday objects. However, nearly all existing work implicitly assumes that whole objects are to be printed from scratch. Designing objects as extensions or enhancements of existing ones is a laborious process in most of today's 3D authoring tools. This paper presents a framework for 3D printing to augment existing objects that covers a wide range of attachment options. We illustrate the framework through three exemplar attachment techniques -- print-over, print-to-affix and print-through, implemented in Encore, a design tool that supports a set of analysis metrics relating to viability, durability and usability that are visualized for the user to explore design options and tradeoffs. Encore also generates 3D models for production, addressing issues such as support jigs and contact geometry between the attached part and the original object. Our validation helps to illustrate the strengths and weaknesses of each technique. For example, print-over is stronger than print-to-affix with adhesives, and all the techniques' strengths are affected by surface curvature.

We present a method for interactive editing of planar linkages. Given a working linkage as input, the user can make targeted edits to the shape or motion of selected parts while preserving other, e.g., functionally-important aspects. In order to make this process intuitive and efficient, we provide a number of editing tools at different levels of abstraction. For instance, the user can directly change the structure of a linkage by displacing joints, edit the motion of selected points on the linkage, or impose limits on the size of its enclosure. Our method safeguards against degenerate configurations during these edits, thus ensuring the correct functioning of the mechanism at all times. Linkage editing poses strict requirements on performance that standard approaches fail to provide. In order to enable interactive and robust editing, we build on a symbolic kinematics approach that uses closed-form expressions instead of numerical methods to compute the motion of a linkage and its derivatives. We demonstrate our system on a diverse set of examples, illustrating the potential to adapt and personalize the structure and motion of existing linkages. To validate the feasibility of our edited designs, we fabricated two physical prototypes.

We present an interactive design system that allows casual users to quickly create 3D-printable robotic creatures. Our approach automates the tedious parts of the design process while providing ample room for customization of morphology, proportions, gait and motion style. The technical core of our framework is an efficient optimization-based solution that generates stable motions for legged robots of arbitrary designs. An intuitive set of editing tools allows the user to interactively explore the space of feasible designs and to study the relationship between morphological features and the resulting motions. Fabrication blueprints are generated automatically such that the robot designs can be manufactured using 3D-printing and off-the-shelf servo motors. We demonstrate the effectiveness of our solution by designing six robotic creatures with a variety of morphological features: two, four or five legs, point or area feet, actuated spines and different proportions. We validate the feasibility of the designs generated with our system through physics simulations and physically-fabricated prototypes.

We present a computational tool for fabrication-oriented design of flexible rod meshes. Given a deformable surface and a set of deformed poses as input, our method automatically computes a printable rod mesh that, once manufactured, closely matches the input poses under the same boundary conditions. The core of our method is formed by an optimization scheme that adjusts the cross-sectional profiles of the rods and their rest centerline in order to best approximate the target deformations. This approach allows us to locally control the bending and stretching resistance of the surface with a single material, yielding high design flexibility and low fabrication cost.

We present an interactive tool for designing physical surfaces made from flexible interlocking quadrilateral elements of a single size and shape. With the element shape fixed, the design task becomes one of finding a discrete structure---i.e., element connectivity and binary orientations---that leads to a desired geometry. In order to address this challenging problem of combinatorial geometry, we propose a forward modeling tool that allows the user to interactively explore the space of feasible designs. Paralleling principles from conventional modeling software, our approach leverages a library of base shapes that can be instantiated, combined, and extended using two fundamental operations: merging and extrusion. In order to assist the user in building the designs, we furthermore propose a method to automatically generate assembly instructions. We demonstrate the versatility of our method by creating a diverse set of digital and physical examples that can serve as personalized lamps or decorative items.

Developing control methods that allow legged robots to move with skill and agility remains one of the grand challenges in robotics. In order to achieve this ambitious goal, legged robots must possess a wide repertoire of motor skills. A scalable control architecture that can represent a variety of gaits in a unified manner is therefore desirable. Inspired by the motor learning principles observed in nature, we use an optimization approach to automatically discover and fine-tune parameters for agile gaits. The success of our approach is due to the controller parameterization we employ, which is compact yet flexible, therefore lending itself well to learning through repetition. We use our method to implement a flying trot, a bound and a pronking gait for StarlETH, a fully autonomous quadrupedal robot.

We present a design system for linkage-based characters, combining form and function in an aesthetically-pleasing manner. Linkage-based character design exhibits a mix of discrete and continuous problems, making for a highly unintuitive design space that is difficult to navigate without assistance. Our system significantly simplifies this task by allowing users to interactively browse different topology options, thus guiding the discrete set of choices that need to be made. A subsequent continuous optimization step improves motion quality and, crucially, safeguards against singularities. We demonstrate the flexibility of our method on a diverse set of character designs, and then realize our designs by physically fabricating prototypes.

We present a new approach to clothing simulation using low-dimensional linear subspaces with temporally adaptive bases. Our method exploits full-space simulation training data in order to construct a pool of low-dimensional bases distributed across pose space. For this purpose, we interpret the simulation data as offsets from a kinematic deformation model that captures the global shape of clothing due to body pose. During subspace simulation, we select low-dimensional sets of basis vectors according to the current pose of the character and the state of its clothing. Thanks to this adaptive basis selection scheme, our method is able to reproduce diverse and detailed folding patterns with only a few basis vectors. Our experiments demonstrate the feasibility of subspace clothing simulation and indicate its potential in terms of quality and computational efficiency.

Quadrupedal animals move through their environments with unmatched agility and grace. An important part of this is the ability to choose between different gaits in order to travel optimally at a certain speed or to robustly deal with unanticipated perturbations. In this paper, we present a control framework for a quadrupedal robot that is capable of locomoting using several gaits. We demonstrate the flexibility of the algorithm by performing experiments on StarlETH, a recently-developed quadrupedal robot. We implement controllers for a static walk, a walking trot, and a running trot, and show that smooth transitions between them can be performed. Using this control strategy, StarlETH is able to trot unassisted in 3D space with speeds of up to 0.7m/s, it can dynamically navigate over unperceived 5-cm high obstacles and it can recover from significant external pushes.

Computer graphics techniques allow artists to realize their imaginative visions, leading to immersive virtual worlds that capture the imagination of audiences world-wide. And now, thanks to advancements in rapid manufacturing devices, tangible links between these vivid virtual worlds and our own can be created. But in order to unleash the full potential of this technology, a key challenge lies in determining the fundamental principles and design paradigms that allow digital content to be processed into forms that are suitable for fabrication. A particularly challenging task is that of creating physical representations of animated virtual characters. This paper discusses several techniques that can be applied towards this goal. In particular, a method for controlling the deformation behavior of real-world objects is described, and a computational design system that allows casual users to create animated mechanical characters is presented. In addition, this paper shows that control algorithms developed for physics-based character animation can also be applied to legged robots, allowing them to move with skill and purpose.

We present an interactive design system that allows non-expert users to create animated mechanical characters. Given an articulated character as input, the user iteratively creates an animation by sketching motion curves indicating how different parts of the character should move. For each motion curve, our framework creates an optimized mechanism that reproduces it as closely as possible. The resulting mechanisms are attached to the character and then connected to each other using gear trains, which are created in a semi-automated fashion. The mechanical assemblies generated with our system can be driven with a single input driver, such as a hand-operated crank or an electric motor, and they can be fabricated using rapid prototyping devices. We demonstrate the versatility of our approach by designing a wide range of mechanical characters, several of which we manufactured using 3D printing. While our pipeline is designed for characters driven by planar mechanisms, significant parts of it extend directly to non-planar mechanisms, allowing us to create characters with compelling 3D motions.

We present a method for fabrication-oriented design of actuated deformable characters that allows a user to automatically create physical replicas of digitally designed characters using rapid manufacturing technologies. Given a deformable character and a set of target poses as input, our method computes a small set of actuators along with their locations on the surface and optimizes the internal material distribution such that the resulting character exhibits the desired deformation behavior. We approach this problem with a dedicated algorithm that combines finite-element analysis, sparse regularization, and constrained optimization. We validate our pipeline on a set of two- and three-dimensional example characters and present results in simulation and physically-fabricated prototypes.

Abstract We present a practical and inexpensive method for creating physical objects that cast different color shadow images when illuminated by prescribed lighting configurations. The input to our system is a number of lighting configurations and corresponding desired shadow images. Our approach computes attenuation masks, which are then printed on transparent materials and stacked to form a single multi‐layer attenuator. When illuminated with the input lighting configurations, this multi‐layer attenuator casts the prescribed color shadow images. Alternatively, our method can compute layers so that their permutations produce different prescribed shadow images under fixed lighting. Each multi‐layer attenuator is quick and inexpensive to produce, can generate multiple full‐color shadows, and can be designed to respond to different types of natural or synthetic lighting setups. We illustrate the effectiveness of our multi‐layer attenuators in simulation and in reality, with the sun as a light source.

Abstract We present ‘Smart Scribbles’—a new scribble‐based interface for user‐guided segmentation of digital sketchy drawings. In contrast to previous approaches based on simple selection strategies, Smart Scribbles exploits richer geometric and temporal information, resulting in a more intuitive segmentation interface. We introduce a novel energy minimization formulation in which both geometric and temporal information from digital input devices is used to define stroke‐to‐stroke and scribble‐to‐stroke relationships. Although the minimization of this energy is, in general, an NP‐hard problem, we use a simple heuristic that leads to a good approximation and permits an interactive system able to produce accurate labellings even for cluttered sketchy drawings. We demonstrate the power of our technique in several practical scenarios such as sketch editing, as‐rigid‐as‐possible deformation and registration, and on‐the‐fly labelling based on pre‐classified guidelines.

We present a method for controlling the motions of active deformable characters. As an underlying principle, we require that all motions be driven by internal deformations. We achieve this by dynamically adapting rest shapes in order to induce deformations that, together with environment interactions, result in purposeful and physically-plausible motions. Rest shape adaptation is a powerful concept and we show that by restricting shapes to suitable subspaces, it is possible to explicitly control the motion styles of deformable characters. Our formulation is general and can be combined with arbitrary elastic models and locomotion controllers. We demonstrate the efficiency of our method by animating curve, shell, and solid-based characters whose motion repertoires range from simple hopping to complex walking behaviors.

We present a method that brings the benefits of physics-based simulations to traditional animation pipelines. We formulate the equations of motions in the subspace of deformations defined by an animator's rig. Our framework fits seamlessly into the workflow typically employed by artists, as our output consists of animation curves that are identical in nature to the result of manual keyframing. Artists can therefore explore the full spectrum between handcrafted animation and unrestricted physical simulation. To enhance the artist's control, we provide a method that transforms stiffness values defined on rig parameters to a non-homogeneous distribution of material parameters for the underlying FEM model. In addition, we use automatically extracted high-level rig parameters to intuitively edit the results of our simulations, and also to speed up computation. To demonstrate the effectiveness of our method, we create compelling results by adding rich physical motions to coarse input animations. In the absence of artist input, we create realistic passive motion directly in rig space.

We develop an integrated set of gaits and skills for a physics-based simulation of a quadruped. The motion repertoire for our simulated dog includes walk, trot, pace, canter, transverse gallop, rotary gallop, leaps capable of jumping on-and-off platforms and over obstacles, sitting, lying down, standing up, and getting up from a fall. The controllers use a representation based on gait graphs, a dual leg frame model, a flexible spine model, and the extensive use of internal virtual forces applied via the Jacobian transpose. Optimizations are applied to these control abstractions in order to achieve robust gaits and leaps with desired motion styles. The resulting gaits are evaluated for robustness with respect to push disturbances and the traversal of variable terrain. The simulated motions are also compared to motion data captured from a filmed dog.

We present a control strategy for physically-simulated walking motions that generalizes well across gait parameters, motion styles, character proportions, and a variety of skills. The control is realtime, requires no character-specific or motion-specific tuning, is robust to disturbances, and is simple to compute. The method works by integrating tracking, using proportional-derivative control; foot placement, using an inverted pendulum model; and adjustments for gravity and velocity errors, using Jacobian transpose control. High-level gait parameters allow for forwards-and-backwards walking, various walking speeds, turns, walk-to-stop, idling, and stop-to-walk behaviors. Character proportions and motion styles can be authored interactively, with edits resulting in the instant realization of a suitable controller. The control is further shown to generalize across a variety of walking-related skills, including picking up objects placed at any height, lifting and walking with heavy crates, pushing and pulling crates, stepping over obstacles, ducking under obstacles, and climbing steps.

We present a method for precomputing robust task-based control policies for physically simulated characters. This allows for characters that can demonstrate skill and purpose in completing a given task, such as walking to a target location, while physically interacting with the environment in significant ways. As input, the method assumes an abstract action vocabulary consisting of balance-aware, step-based controllers. A novel constrained state exploration phase is first used to define a character dynamics model as well as a finite volume of character states over which the control policy will be defined. An optimized control policy is then computed using reinforcement learning. The final policy spans the cross-product of the character state and task state, and is more robust than the conrollers it is constructed from. We demonstrate real-time results for six locomotion-based tasks and on three highly-varied bipedal characters. We further provide a game-scenario demonstration.

Simulated characters in simulated worlds require simulated skills. We develop control strategies that enable physically-simulated characters to dynamically navigate environments with significant stepping constraints, such as sequences of gaps. We present a synthesis-analysis-synthesis framework for this type of problem. First, an offline optimization method is applied in order to compute example control solutions for randomly-generated example problems from the given task domain. Second, the example motions and their underlying control patterns are analyzed to build a low-dimensional step-to-step model of the dynamics. Third, this model is exploited by a planner to solve new instances of the task at interactive rates. We demonstrate real-time navigation across constrained terrain for physics-based simulations of 2D and 3D characters. Because the framework sythesizes its own example data, it can be applied to bipedal characters for which no motion data is available.

Modeling the large space of possible human motions requires scalable techniques. Generalizing from example motions or example controllers is one way to provide the required scalability. We present techniques for generalizing a controller for physics-based walking to significantly different tasks, such as climbing a large step up, or pushing a heavy object. Continuation methods solve such problems using a progressive sequence of problems that trace a path from an existing solved problem to the final desired-but-unsolved problem. Each step in the continuation sequence makes progress towards the target problem while further adapting the solution. We describe and evaluate a number of choices in applying continuation methods to adapting walking gaits for tasks involving interaction with the environment. The methods have been successfully applied to automatically adapt a regular cyclic walk to climbing a 65 cm step, stepping over a 55 cm sill, pushing heavy furniture, walking up steep inclines, and walking on ice. The continuation path further provides parameterized solutions to these problems.

To increase memory bandwidth, many programmable Digital Signal Processors (DSPs) employ two on-chip data memories. This architectural feature supports higher memory bandwidth by allowing multiple data memory accesses to occur in parallel. Exploiting dual memory banks, however, is a challenging problem for compilers. This, in part, is due to the instruction-level parallelism, small numbers of registers, and highly specialized register capabilities of most DSPs. In this paper, we present a new methodology based on a genetic algorithm for assigning data to dual-bank memories. Our approach is global, and integrates several important issues in memory assignment within a single model. Special effort is made to identify those data objects that could potentially benefit from an assignment to a specific memory, or perhaps duplication in both memories. Our computational results show that the GA is able to achieve a 54% reduction in the number of memory cycles and a reduction in the range of 7% to 42% in the total number of cycles when tested with well-known DSP kernels and applications.

If you were told that some object A was perfectly (or somewhat, or not at all) in some direction d (e.g., west, above-right) of some reference object B, where in space would you look for A? Cognitive experiments suggest that you would mentally build a spatial template. Using essentially angular deviation, you would partition the space into regions where the relationship "in direction d of B" holds (to various extents) and regions where it does not hold. You would then be able to locate the objects for which the relationship holds best, and find A. Spatial templates, therefore, represent directional spatial relationships to reference objects (e.g., "east of the post office"). Note that other names can also be found in the literature (e.g., fuzzy landscape, applicability structure, potential field). There exists a very simple and yet cognitively plausible way to mathematically model a spatial template without sacrificing the geometry of the reference object (i.e., the object is not approximated through its centroid or minimum bounding rectangle). In case of 2D raster data, exact calculation of the model can easily be achieved but is computationally expensive, and tractable approximation algorithms were proposed. In case of 2D vector data, exact calculation of the model is not conceivable. In previous work, we introduced the concept of the F-template. We discussed the case of 2D raster data and designed, based on this concept, an efficient approximation algorithm for spatial template computation. The algorithm is faster, gives better results, and is more flexible than its competitors. Here, comparable advances are presented in the case of 2D vector data. These advances are of particular interest for spatial query processing in Geographic Information Systems.

To increase memory bandwidth, many programmable Digital-Signal Processors (DSPs) employ two on-chip data memories. This architectural feature supports higher memory bandwidth by allowing multiple data memory accesses to occur in parallel. Exploiting dual memory banks, however, is a challenging problem for compilers. This, in part, is due to the instruction-level parallelism, small numbers of registers, and highly specialized register capabilities of most DSPs. In this paper, we present a new methodology based on a Memetic Algorithm (MA) for assigning data to dual-bank memories. Our approach is global, and integrates several important issues in memory assignment within a single model. Special effort is made to identify those data objects that could potentially benefit from an assignment to a specific memory, or perhaps duplication in both memories. Our computational results show that the MA is able to achieve a 54% reduction in the number of memory cycles and a reduction in the range of 7%–42% in the total number of cycles when tested with well-known DSP kernels and applications. Our computational results also show that, when compared with the Genetic Algorithm in Ref. 3, the memetic algorithm is able to find solutions that, on average, have 7%–20% less cost, with the biggest improvements being found for larger problem instances.