Many non-photorealistic rendering (NPR) styles, such as feature lines and hatching, are defined by image-space structure inherited from hand-drawn media. While recent path-space formulations like the stylized rendering equation (SRE) enable stylization to interact naturally with light transport, they provide no mechanism for enforcing image-space consistency. We present a conceptual framework for lifting image-space stylizations into path-space in a principled, SRE-compatible manner. Our key observation is that image-space consistency can be achieved by establishing geometrically-driven mappings from image-space to path-space. We demonstrate this through two complementary stylizations: feature lines and tone. For feature line rendering, we introduce a conditional lifting based on partial path-space variation, where a geometric path parametrization is combined with parallel transport to preserve image-space structure under distribution effects. This enables a curvature-aware, stochastic, geometry-driven formulation of line detection that generalizes prior ray-based methods. For tone-based styles, such as hatching and halftone, we propose a canonical lifting anchored to a material-independent mapping, motivated by stationary-phase arguments from wave optics. Its locally invertible approximation enables evaluation of image-space tone fields at arbitrary path vertices while preserving image-space structure under complex light transport. Both methods are implemented as ordinary style functions under the SRE and work with existing estimation and sampling strategies; demonstrating how image-space structure can be preserved within path-space rendering, enabling a broader class of expressive, physically-grounded NPR styles.

We estimate the three Herschel-Bulkley parameters (yield stress σ Y , power-law index n , and consistency parameter η ) for shear-dependent fluid-like materials possibly with large-scale inclusions, for which rheometers may fail to provide a useful measurement. We perform experiments using the unknown material for dam-break (or column collapse) setups and capture video footage. We then use simulations to optimize for the material parameters. For better match up with the simple shear flow encountered in a rheometer, we modify the flow rule for the elasto-viscoplastic Herschel-Bulkley model. Analyzing the loss landscape for optimization, we realize a similarity relation ; material parameters far away within this relation would result in matched simulations, making it hard to distinguish the parameters. We found that by exploiting the setup dependency of the similarity relation, we can improve on the estimation using multiple setups, which we propose by analyzing the Hessian of the similarity relation. We validate the efficacy of our method by comparing the estimations to the measurements from a rheometer (for materials without large-scale inclusions) and show applications to materials with large-scale inclusions, including various salad or pasta sauces, and congee.

Generative models based on deep neural networks often have a high-dimensional latent space, ranging sometimes to a few hundred dimensions or even higher, which typically makes them hard for a user to explore directly. We propose differential subspace search to allow efficient iterative user exploration in such a space, without relying on domain- or data-specific assumptions. We develop a general framework to extract low-dimensional subspaces based on a local differential analysis of the generative model, such that a small change in such a subspace would provide enough change in the resulting data. We do so by applying singular value decomposition to the Jacobian of the generative model and forming a subspace with the desired dimensionality spanned by a given number of singular vectors stochastically selected on the basis of their singular values, to maintain ergodicity. We use our framework to present 1D subspaces to the user via a 1D slider interface. Starting from an initial location, the user finds a new candidate in the presented 1D subspace, which is in turn updated at the new candidate location. This process is repeated until no further improvement can be made. Numerical simulations show that our method can better optimize synthetic black-box objective functions than the alternatives that we tested. Furthermore, we conducted a user study using complex generative models and the results show that our method enables more efficient exploration of high-dimensional latent spaces than the alternatives.

The materials around us usually exist as mixtures of constituents, each constituent with possibly a different elasto-viscoplastic property. How can we describe the material property of such a mixture is the core question of this paper. We propose a nonlinear blending model that can capture intriguing flowing behaviors that can differ from that of the individual constituents (Fig. 1). We used a laboratory device, rheometer , to measure the flowing properties of various fluid-like foods, and found that an elastic Herschel-Bulkley model has nice agreements with the measured data even for the mixtures of these foods. We then constructed a blending model such that it qualitatively agrees with the measurements and is closed in the parameter space of the elastic Herschel-Bulkley model. We provide validations through comparisons between the measured and estimated properties using our model, and comparisons between simulated examples and captured footages. We show the utility of our model for producing interesting behaviors of various mixtures.

Robotic manipulation of deformable objects is a difficult problem especially because of the complexity of the many different ways an object can deform. Searching such a high-dimensional state space makes it difficult to recognize, track, and manipulate deformable objects. In this paper, we introduce a predictive, model-driven approach to address this challenge, using a precomputed, simulated database of deformable object models. Mesh models of common deformable garments are simulated with the garments picked up in multiple different poses under gravity, and stored in a database for fast and efficient retrieval. To validate this approach, we developed a comprehensive pipeline for manipulating clothing as in a typical laundry task. First, the database is used for category and the pose estimation is used for a garment in an arbitrary position. A fully featured 3-D model of the garment is constructed in real time, and volumetric features are then used to obtain the most similar model in the database to predict the object category and pose. Second, the database can significantly benefit the manipulation of deformable objects via nonrigid registration, providing accurate correspondences between the reconstructed object model and the database models. Third, the accurate model simulation can also be used to optimize the trajectories for the manipulation of deformable objects, such as the folding of garments. Extensive experimental results are shown for the above tasks using a variety of different clothings. Note to Practitioners-This paper provides an open source, extensible, 3-D database for dissemination to the robotics and graphics communities. Model-driven methods are proliferating, and they need to be applied, tested, and validated in real environments. A key idea we have exploited is to have an innovative and novel use of simulation. This database will serve as infrastructure for developing advanced robotic machine learning algorithms. We want to address this machine learning idea ourselves, but we expect the dissemination of the database to other researchers with different agendas and task applications, which will bring wide progress in this area. Our proposed methods, as mentioned earlier, can be easily applied to interrelated areas. One example is that the 3-D shape-based matching algorithm can be used for other objects, such as bottles, papers, and food. After integrating with other robotic systems, the use of the robot can be easily extended to other tasks, such as making food, cleaning room, and fetching objects, to assist our daily life.

We propose a technique to simulate granular materials that exploits the dual strengths of discrete and continuum treatments. Discrete element simulations provide unmatched levels of detail and generality, but prove excessively costly when applied to large scale systems. Continuum approaches are computationally tractable, but limited in applicability due to built-in modeling assumptions; e.g., models suitable for granular flows typically fail to capture clogging, bouncing and ballistic motion. In our hybrid approach, an oracle dynamically partitions the domain into continuum regions where safe, and discrete regions where necessary. The domains overlap along transition zones, where a Lagrangian dynamics mass-splitting coupling principle enforces agreement between the two simulation states. Enrichment and homogenization operations allow the partitions to evolve over time. This approach accurately and efficiently simulates scenarios that previously required an entirely discrete treatment.

Clouds play an important role when synthesizing realistic images of outdoor scenes. The realistic display of clouds is therefore one of the important research topics in computer graphics. In order to display realistic clouds, we need methods for modeling, rendering, and animating clouds realistically. It is also important to control the shapes and appearances of clouds to create certain visual effects. In this paper, we explain our efforts and research results to meet such requirements, together with related researches on the visual simulation of clouds.

We propose a framework for developing a new class of imaging systems that are thin and flexible. Such an imaging sheet can be flexed at will and wrapped around everyday objects to capture unconventional fields of view. Our approach is to use a lens array attached to a sheet with a 2D grid of pixels. A major challenge with this type of a system is that its sampling of the scene varies with the curvature of the sheet. To avoid undesirable aliasing effects due to under-sampling in high curvature regions of the sheet, we design a deformable lens array with adaptive optical properties. We show that the material and geometric properties of the lens array can be optimized so that the object-side point spread function corresponding to each pixel widens with the curvature of the sheet at that pixel. This intrinsic adaptation of focal length is passive (without the use of actuators or other control mechanisms), and enables a sheet camera to capture images without aliasing, irrespective of its shape. We have designed a 33×33 lens array, fabricated it using silicone rubber, and conducted several experiments to verify its optical adaptation characteristics. We conclude with a discussion on the advantages of our proposed approach as well as future work.

Robotic manipulation of deformable objects remains a challenging task. One such task is to iron a piece of cloth autonomously. Given a roughly flattened cloth, the goal is to have an ironing plan that can iteratively apply a regular iron to remove all the major wrinkles by a robot. We present a novel solution to analyze the cloth surface by fusing two surface scan techniques: a curvature scan and a discontinuity scan. The curvature scan can estimate the height deviation of the cloth surface, while the discontinuity scan can effectively detect sharp surface features, such as wrinkles. We use this information to detect the regions that need to be pulled and extended before ironing, and the other regions where we want to detect wrinkles and apply ironing to remove the wrinkles. We demonstrate that our hybrid scan technique is able to capture and classify wrinkles over the surface robustly. Given detected wrinkles, we enable a robot to iron them using shape features. Experimental results show that using our wrinkle analysis algorithm, our robot is able to iron the cloth surface and effectively remove the wrinkles.

Deformable objects such as garments are highly unstructured, making them difficult to recognize and manipulate. In this paper, we propose a novel method to teach a two-arm robot to efficiently track the states of a garment from an unknown state to a known state by iterative regrasping. The problem is formulated as a constrained weighted evaluation metric for evaluating the two desired grasping points during regrasping, which can also be used for a convergence criterion The result is then adopted as an estimation to initialize a regrasping, which is then considered as a new state for evaluation. The process stops when the predicted thin shell conclusively agrees with reconstruction. We show experimental results for regrasping a number of different garments including sweater, knitwear, pants, and leggings, etc.

Robotic manipulation of deformable objects remains a challenging task. One such task is folding a garment autonomously. Given start and end folding positions, what is an optimal trajectory to move the robotic arm to fold a garment? Certain trajectories will cause the garment to move, creating wrinkles, and gaps, other trajectories will fail altogether. We present a novel solution to find an optimal trajectory that avoids such problematic scenarios. The trajectory is optimized by minimizing a quadratic objective function in an off-line simulator, which includes material properties of the garment and frictional force on the table. The function measures the dissimilarity between a user folded shape and the folded garment in simulation, which is then used as an error measurement to create an optimal trajectory. We demonstrate that our two-arm robot can follow the optimized trajectories, achieving accurate and efficient manipulations of deformable objects.

Building LEGO sculptures requires accounting for the target object's shape, colors, and stability. In particular, finding a good layout of LEGO bricks that prevents the sculpture from collapsing (due to its own weight) is usually challenging, and it becomes increasingly difficult as the target object becomes larger or more complex. We devise a force-based analysis for estimating physical stability of a given sculpture. Unlike previous techniques for Legolization, which typically use heuristic-based metrics for stability estimation, our force-based metric gives 1) an ordering in the strength so that we know which structure is more stable, and 2) a threshold for stability so that we know which one is stable enough. In addition, our stability analysis tells us the weak portion of the sculpture. Building atop our stability analysis, we present a layout refinement algorithm that iteratively improves the structure around the weak portion, allowing for automatic generation of a LEGO brick layout from a given 3D model, accounting for color information, required workload (in terms of the number of bricks) and physical stability. We demonstrate the success of our method with real LEGO sculptures built up from a wide variety of 3D models, and compare against previous methods.

We consider the simulation of dense foams composed of microscopic bubbles, such as shaving cream and whipped cream. We represent foam not as a collection of discrete bubbles, but instead as a continuum. We employ thematerial point method(MPM) to discretize a hyperelastic constitutive relation augmented with the Herschel-Bulkley model of non-Newtonian viscoplastic flow, which is known to closely approximate foam behavior. Since large shearing flows in foam can produce poor distributions of material points, a typical MPM implementation can produce non-physical internal holes in the continuum. To address these artifacts, we introduce a particle resampling method for MPM. In addition, we introduce an explicit tearing model to prevent regions from shearing into artificially thin, honey-like threads. We evaluate our method's efficacy by simulating a number of dense foams, and we validate our method by comparing to real-world footage of foam.

We present a computational approach for designing wire meshes , i.e., freeform surfaces composed of woven wires arranged in a regular grid. To facilitate shape exploration, we map material properties of wire meshes to the geometric model of Chebyshev nets . This abstraction is exploited to build an efficient optimization scheme. While the theory of Chebyshev nets suggests a highly constrained design space, we show that allowing controlled deviations from the underlying surface provides a rich shape space for design exploration. Our algorithm balances globally coupled material constraints with aesthetic and geometric design objectives that can be specified by the user in an interactive design session. In addition to sculptural art, wire meshes represent an innovative medium for industrial applications including composite materials and architectural façades. We demonstrate the effectiveness of our approach using a variety of digital and physical prototypes with a level of shape complexity unobtainable using previous methods.

We present a technique for computing the shape of a transparent object that can generate user-defined caustic patterns. The surface of the object generated using our method is smooth. Thanks to this property, the resulting caustic pattern is smooth, natural, and highly detailed compared to the results btained using previous methods. Our method consists of two processes. First, we use a differential geometry approach to compute a smooth mapping between the distributions of the incident light and the light reaching the screen. Second, we utilize this mapping to compute the surface of the object. We solve Poisson's equation to compute both the mapping and the surface of the object.

Abstract Pixel art is a kind of digital art that through per‐pixel manipulation enables production of a diverse array of artistic images. In this paper, we present a new way for people to experience and express pixel art. Our digital art consists of a set of sticks made of acrylate resin, each of which refracts light from a parallel light source, in certain directions. Artistic users are able to easily rearrange these sticks and view their digital art through the refracted light projection on any planar surface. As we demonstrate in this paper, a user can generate various artistic images using only a single set of sticks. We additionally envision that our pixel art with rearrangeable sticks would have great entertainment appeal, e.g., as an art puzzle.

Clouds play an important role in creating realistic images of outdoor scenes. Many methods have therefore been proposed for displaying realistic clouds. However, the realism of the resulting images depends on many parameters used to render them and it is often difficult to adjust those parameters manually. This paper proposes a method for addressing this problem by solving an inverse rendering problem: given a non-uniform synthetic cloud density distribution, the parameters for rendering the synthetic clouds are estimated using photographs of real clouds. The objective function is defined as the difference between the color histograms of the photograph and the synthetic image. Our method searches for the optimal parameters using genetic algorithms. During the search process, we take into account the multiple scattering of light inside the clouds. The search process is accelerated by precomputing a set of intermediate images. After ten to twenty minutes of precomputation, our method estimates the optimal parameters within a minute.

Abstract Photo‐realistic rendering of inhomogeneous participating media with light scattering in consideration is important in computer graphics, and is typically computed using Monte Carlo based methods. The key technique in such methods is the free path sampling, which is used for determining the distance (free path) between successive scattering events. Recently, it has been shown that efficient and unbiased free path sampling methods can be constructed based on Woodcock tracking. The key concept for improving the efficiency is to utilize space partitioning (e.g., kd‐tree or uniform grid), and a better space partitioning scheme is important for better sampling efficiency. Thus, an estimation framework for investigating the gain in sampling efficiency is important for determining how to partition the space. However, currently, there is no estimation framework that works in 3D space. In this paper, we propose a new estimation framework to overcome this problem. Using our framework, we can analytically estimate the sampling efficiency for any typical partitioned space. Conversely, we can also use this estimation framework for determining the optimal space partitioning. As an application, we show that new space partitioning schemes can be constructed using our estimation framework. Moreover, we show that the differences in the performances using different schemes can be predicted fairly well using our estimation framework.

Realistic rendering of participating media is one of the major subjects in computer graphics. Monte Carlo techniques are widely used for realistic rendering because they provide unbiased solutions, which converge to exact solutions. Methods based on Monte Carlo techniques generate a number of light paths, each of which consists of a set of randomly selected scattering events. Finding a new scattering event requires free path sampling to determine the distance from the previous scattering event, and is usually a time-consuming process for inhomogeneous participating media. To address this problem, we propose an adaptive and unbiased sampling technique using kd-tree based space partitioning. A key contribution of our method is an automatic scheme that partitions the spatial domain into sub-spaces (partitions) based on a cost model that evaluates the expected sampling cost. The magnitude of performance gain obtained by our method becomes larger for more inhomogeneous media, and rises to two orders compared to traditional free path sampling techniques.

Abstract A rendering system for interior scenes is proposed in this paper. The light reaches the interior scene, usually through small regions, such as windows or abat‐jours, which we call portals. To provide a solution, suitable for rendering interior scenes with portals, we extend the traditional precomputed radiance transfer approaches. In our approach, a bounding sphere, which we call a shell, of the interior, centered at each portal, is created and the light transferred from the shell towards the interior through the portal is precomputed. Each shell acts as an environment light source and its intensity distribution is determined by rendering images of the scene, viewed from the center of the shell. By updating the intensity distribution of the shell at each frame, we are able to handle dynamic objects outside the shells. The material of the portals can also be modified at run time (e.g. changing from transparent glass to frosted glass). Several applications are shown, including the illumination of a cathedral, lit by skylight at different times of a day, and a car, running in a town, at interactive frame rates, with a dynamic viewpoint.

Information on appearances of three-dimensional objects are transmitted via the Internet, and displaying objects plays an important role in a lot of areas such as movies and video games. An autostereogram is one of the ways to represent three-dimensional objects taking advantage of binocular parallax, by which depths of objects are perceived. In previous methods, the colors of objects were ignored when constructing autostereogram images. In this paper, we propose a method to construct autostereogram images taking into account color variation of objects, such as shading. We also propose a technique to embed color information in autostereogram images. Using our technique, autostereogram images shown on a display change, and viewers can enjoy perceiving three-dimensional object and its colors in two stages. Color information is embedded in a monochrome autostereogram image, and colors appear when the correct color table is used. Our technique can also be used for steganography to hide color information. Moreover, file size gets smaller than that of an original autostereogram image because indexed colors are used. Therefore, the 3D images constructed using our method are useful for transmission through the cyberworld.

In the computer graphics field, improving reality and reducing computational cost are both very important, thus a lot of research has been done to improve these two factors. For reality, physically based lighting models play an important role. Especially shadows and interreflections of light have a great impact on perceptual reality. To improve computational speed, exploiting the parallel processing ability of GPUs and using precomputation are major techniques. In this paper, we introduce some methods using precomputation and the GPU for efficient photo-realistic rendering.

In this paper, we propose a fast global illumination solution for interactive lighting design. Using our method, light sources and the viewpoint are movable, and the characteristics of materials can be modified (assuming low-frequency BRDF) during rendering. Our solution is based on particle tracing (a variation of photon mapping) and final gathering. We assume that objects in the input scene are static, and pre-compute potential light paths for particle tracing and final gathering. To perform final gathering fast, we propose an efficient technique called Hierarchical Histogram Estimation for rapid estimation of radiances from the distribution of the particles. The rendering process of our method can be fully implemented on the GPU and our method achieves interactive frame rates for rendering scenes with even more than 100,000 triangles.