We present Perm, a learned parametric representation of human 3D hair designed to facilitate various hair-related applications. Unlike previous work that jointly models the global hair structure and local curl patterns, we propose to disentangle them using a PCA-based strand representation in the frequency domain, thereby allowing more precise editing and output control. Specifically, we leverage our strand representation to fit and decompose hair geometry textures into low- to high-frequency hair structures, termed guide textures and residual textures, respectively. These decomposed textures are later parameterized with different generative models, emulating common stages in the hair grooming process. We conduct extensive experiments to validate the architecture design of Perm, and finally deploy the trained model as a generic prior to solve task-agnostic problems, further showcasing its flexibility and superiority in tasks such as single-view hair reconstruction, hairstyle editing, and hair-conditioned image generation. More details can be found on our project page: https://cs.yale.edu/homes/che/projects/perm/.

Spectral information plays a crucial role in many domains, including remote sensing, cultural heritage analysis, food inspection, and material appearance modeling. Spectral measurements, such as hyperspectral imaging, provide a powerful means of acquiring this information but often require expensive equipment and time-consuming capture procedures.

We present 3DGH, an unconditional generative model for 3D human heads with composable hair and face components. Unlike previous work that entangles the modeling of hair and face, we propose to separate them using a novel data representation with template-based 3D Gaussian Splatting, in which deformable hair geometry is introduced to capture the geometric variations across different hairstyles. Based on this data representation, we design a 3D GAN-based architecture with dual generators and employ a cross-attention mechanism to model the inherent correlation between hair and face. The model is trained on synthetic renderings using carefully designed objectives to stabilize training and facilitate hair-face separation. We conduct extensive experiments to validate the design choice of 3DGH, and evaluate it both qualitatively and quantitatively by comparing with several state-of-the-art 3D GAN methods, demonstrating its effectiveness in unconditional full-head image synthesis and composable 3D hairstyle editing. More details will be available on our project page: https://c-he.github.io/projects/3dgh/.

The fashion industry has a real need to preview fabric designs using the actual threads they intend to use, ensuring that the designs they envisage can be physically realized. Unfortunately, today's fabric rendering relies on either hand-tuned parameters or parameters acquired from already fabricated cloth. Furthermore, existing curve-based scattering models are not suitable for this problem: they are either not naturally differentiable due to discrete fiber count parameters, or require a more detailed geometry representation, introducing extra complexity. In this work, we bridge this gap by presenting a novel pipeline that captures and digitizes physical threads and predicts the appearance of the fabric based on the weaving pattern. We develop a practical thread scattering model based on simulations of multiple fiber scattering within a thread. Using a cost-efficient multi-view setup, we capture threads of diverse colors and materials. We apply differentiable rendering to digitize threads, demonstrating that our model significantly improves the reconstruction accuracy compared to existing models, matching both reflection and transmission. We leverage a two-scale rendering technique to efficiently render woven cloth. We validate that our digital threads, combined with simulated woven yarn geometry, can accurately predict the fabric appearance by comparing to real samples. We show how our work can aid designs using diverse thread profiles, woven patterns, and textured design patterns.

Automation of inspection for quality control is needed to overcome the errors and delays inherent in manual processes. Machine learning methods have the potential to greatly improve automated inspection. However, machine learning techniques require training data that are precisely labeled and reflect the distribution of defects to be detected. Physically collecting suitable training data requires significant time and prolongs the overall time for system development. To address this challenge, a new study is presented that explores synthetic data generation for a state-of-the-art anomaly detection (AD) model in the electric motor housing (EMH) surface inspection. The study successfully demonstrates using synthetic data for anomaly detection and presents a comparison of detection performance by models trained solely on synthetic data and models trained on both synthetic and real data. The study shows that real data combined with synthetic data can increase overall model performance. The study also addresses current challenges in using synthetic data and proposes directions for future work.

We present an implicit neural representation to learn the spatio-temporal space of kinematic motions. Unlike previous work that represents motion as discrete sequential samples, we propose to express the vast motion space as a continuous function over time, hence the name Neural Motion Fields (NeMF). Specifically, we use a neural network to learn this function for miscellaneous sets of motions, which is designed to be a generative model conditioned on a temporal coordinate $t$ and a random vector $z$ for controlling the style. The model is then trained as a Variational Autoencoder (VAE) with motion encoders to sample the latent space. We train our model with a diverse human motion dataset and quadruped dataset to prove its versatility, and finally deploy it as a generic motion prior to solve task-agnostic problems and show its superiority in different motion generation and editing applications, such as motion interpolation, in-betweening, and re-navigating. More details can be found on our project page: https://cs.yale.edu/homes/che/projects/nemf/.

Abstract Despite the ubiquitous use of materials maps in modern rendering pipelines, their editing and control remains a challenge. In this paper, we present an example‐based material control method to augment input material maps based on user‐provided material photos. We train a tileable version of MaterialGAN and leverage its material prior to guide the appearance transfer, optimizing its latent space using differentiable rendering. Our method transfers the micro and meso‐structure textures of user provided target(s) photographs, while preserving the structure and quality of the input material. We show our methods can control existing material maps, increasing realism or generating new, visually appealing materials.

Procedural modeling is now the de facto standard of material modeling in industry. Procedural models can be edited and are easily extended, unlike pixel-based representations of captured materials. In this article, we present a semi-automatic pipeline for general material proceduralization. Given Spatially Varying Bidirectional Reflectance Distribution Functions (SVBRDFs) represented as sets of pixel maps, our pipeline decomposes them into a tree of sub-materials whose spatial distributions are encoded by their associated mask maps. This semi-automatic decomposition of material maps progresses hierarchically, driven by our new spectrum-aware material matting and instance-based decomposition methods. Each decomposed sub-material is proceduralized by a novel multi-layer noise model to capture local variations at different scales. Spatial distributions of these sub-materials are modeled either by a by-example inverse synthesis method recovering Point Process Texture Basis Functions (PPTBF) [ 30 ] or via random sampling. To reconstruct procedural material maps, we propose a differentiable rendering-based optimization that recomposes all generated procedures together to maximize the similarity between our procedural models and the input material pixel maps. We evaluate our pipeline on a variety of synthetic and real materials. We demonstrate our method’s capacity to process a wide range of material types, eliminating the need for artist designed material graphs required in previous work [ 38 , 53 ]. As fully procedural models, our results expand to arbitrary resolution and enable high-level user control of appearance.

Relating small-scale structures to large-scale appearance is a key element in material appearance design. Bi-scale material design requires finding small-scale structures - meso-scale geometry and micro-scale BRDFs - that produce a desired large-scale appearance expressed as a macro-scale BRDF. The adjustment of small-scale geometry and reflectances to achieve a desired appearance can become a tedious trial-and-error process. We present a learning-based solution to fit a target macro-scale BRDF with a combination of a meso-scale geometry and micro-scale BRDF. We confront challenges in representation at both scales. At the large scale we need macro-scale BRDFs that are both compact and expressive. At the small scale we need diverse combinations of geometric patterns and potentially spatially varying micro-BRDFs. For large-scale macro-BRDFs, we propose a novel 2D subset of a tabular BRDF representation that well preserves important appearance features for learning. For small-scale details, we represent geometries and BRDFs in different categories with different physical parameters to define multiple independent continuous search spaces. To build the mapping between large-scale macro-BRDFs and small-scale details, we propose an end-to-end model that takes the subset BRDF as input and performs classification and parameter estimation on small-scale details to find an accurate reconstruction. Compared with other fitting methods, our learning-based solution provides higher reconstruction accuracy and covers a wider gamut of appearance.

This study investigates the potential impact of subsurface light transport on gloss perception for the purposes of broadening our understanding of visual appearance in computer graphics applications. Gloss is an important attribute for characterizing material appearance. We hypothesize that subsurface scattering of light impacts the glossiness perception. However, gloss has been traditionally studied as a surface-related quality and the findings in the state-of-the-art are usually based on fully opaque materials, although the visual cues of glossiness can be impacted by light transmission as well. To address this gap and to test our hypothesis, we conducted psychophysical experiments and found that subjects are able to tell the difference in terms of gloss between stimuli that differ in subsurface light transport but have identical surface qualities and object shape. This gives us a clear indication that subsurface light transport contributes to a glossy appearance. Furthermore, we conducted additional experiments and found that the contribution of subsurface scattering to gloss varies across different shapes and levels of surface roughness. We argue that future research on gloss should include transparent and translucent media and to extend the perceptual models currently limited to surface scattering to more general ones inclusive of subsurface light transport.

Non-photorealistic rendering (NPR) and image processing algorithms are widely assumed as a proxy for drawing. However, this assumption is not well assessed due to the difficulty in collecting and registering freehand drawings. Alternatively, tracings are easier to collect and register, but there is no quantitative evaluation of tracing as a proxy for freehand drawing. In this paper, we compare tracing, freehand drawing, and computer-generated drawing approximation (CGDA) to understand their similarities and differences. We collected a dataset of 1,498 tracings and freehand drawings by 110 participants for 100 image prompts. Our drawings are registered to the prompts and include vector-based timestamped strokes collected via stylus input. Comparing tracing and freehand drawing, we found a high degree of similarity in stroke placement and types of strokes used over time. We show that tracing can serve as a viable proxy for freehand drawing because of similar correlations between spatio-temporal stroke features and labeled stroke types. Comparing hand-drawn content and current CGDA output, we found that 60% of drawn pixels corresponded to computer-generated pixels on average. The overlap tended to be commonly drawn content, but people's artistic choices and temporal tendencies remained largely uncaptured. We present an initial analysis to inform new CGDA algorithms and drawing applications, and provide the dataset for use by the community.

Procedural textures are powerful tools that have been used in graphics for decades. In contrast to the alternative exemplar-based texture synthesis techniques, procedural textures provide user control and fast texture generation with low-storage cost and unlimited texture resolution. However, creating procedural models for complex textures requires a time-consuming process of selecting a combination of procedures and parameters. We present an example-based framework to automatically select procedural models and estimate parameters. In our framework, we consider textures categorized by commonly used high level classes. For each high level class we build a data-driven inverse modeling system based on an extensive collection of real-world textures and procedural texture models in the form of node graphs. We use unsupervised learning on collected real-world images in a texture class to learn sub-classes. We then classify the output of each of the collected procedural models into these sub-classes. For each of the collected models we train a convolutional neural network (CNN) to learn the parameters to produce a specific output texture. To use our framework, a user provides an exemplar texture image within a high level class. The system first classifies the texture into a sub-class, and selects the procedural models that produce output in that sub-class. The pre-trained CNNs of the selected models are used to estimate the parameters of the texture example. With the predicted parameters, the system can generate appropriate procedural textures for the user. The user can easily edit the textures by adjusting the node graph parameters. In a last optional step, style transfer augmentation can be applied to the fitted procedural textures to recover details lost in the procedural modeling process. We demonstrate our framework for four high level classes and show that our inverse modeling system can produce high-quality procedural textures for both structural and non-structural textures.

A 3D modeling system with all-inclusive functionality is too demanding for a casual 3D modeler to learn. There has been a shift towards more approachable systems, with easy-to-learn, intuitive interfaces. However, most modeling systems still employ mouse and keyboard interfaces, despite the ubiquity of tablet devices and the benefits of multi-touch interfaces. We introduce an alternative 3D modeling and fabrication paradigm using developable surfaces, inspired by traditional papercrafting, and we implement it as a complete system designed for a multi-touch tablet, allowing a user to fabricate 3D scenes. We demonstrate the modeling and fabrication process of assembling complex 3D scenes from a collection of simpler models, in turn shaped through operations applied to virtual paper. Our fabrication method facilitates the assembly of the scene with real paper by automatically converting scenes into a series of cutouts with appropriately added fiducial markers and supporting structures. Our system assists users in creating occluded supporting structures to help maintain the spatial and rigid properties of a scene without compromising its aesthetic qualities. We demonstrate several 3D scenes modeled and fabricated in our system, and evaluate the faithfulness of our fabrications relative to their virtual counterparts and 3D-printed fabrications.

We introduce the problem of computing a human-perceived softness measure for virtual 3D objects. As the virtual objects do not exist in the real world, we do not directly consider their physical properties but instead compute the human-perceived softness of the geometric shapes. In an initial experiment, we find that humans are highly consistent in their responses when given a pair of vertices on a 3D model and asked to select the vertex that they perceive to be more soft. This motivates us to take a crowdsourcing and machine learning framework. We collect crowdsourced data for such pairs of vertices. We then combine a learning-to-rank approach and a multi-layer neural network to learn a non-linear softness measure mapping any vertex to a softness value. For a new 3D shape, we can use the learned measure to compute the relative softness of every vertex on its surface. We demonstrate the robustness of our framework with a variety of 3D shapes and compare our non-linear learning approach with a linear method from previous work. Finally, we demonstrate the accuracy of our learned measure with user studies comparing our measure with the human-perceived softness of both virtual and real objects, and we show the usefulness of our measure with some applications.

We propose a simple yet efficient steganalytic algorithm for watermarks embedded by two state-of-the-art 3D watermarking algorithms by Cho et al. The main observation is that while in a clean model the means/variances of Cho et al.'s normalized histogram bins are expected to follow a Gaussian distribution, in a marked model their distribution will be bimodal. The proposed algorithm estimates the number of bins through an exhaustive search and then the presence of a watermark is decided by a tailor made normality test or a t-test. We also propose a modification of Cho et al.'s watermarking algorithms with the watermark embedded by changing the histogram of the radial coordinates of the vertices. Rather than targeting a continuous statistics such as the mean or variance of the values in a bin, the proposed watermarking modifies a discrete statistic, which here is the height of the histogram bin, to achieve watermark embedding. Experimental results demonstrate that the modified algorithm offers not only better resistance against the steganalytic attack we developed, but also an improved robustness/capacity trade-off.

We introduce the problem of computing a human-perceived softness measure for virtual 3D objects. As the virtual objects do not exist in the real world, we do not directly consider their physical properties but instead compute the human-perceived softness of the geometric shapes. We collect crowdsourced data where humans rank their perception of the softness of vertex pairs on virtual 3D models. We then compute shape descriptors and use a learning-to-rank approach to learn a softness measure mapping any vertex to a softness value. Finally, we demonstrate our framework with a variety of 3D shapes.

Abstract We present a review of recent techniques for performing geometric analysis in cultural heritage (CH) applications. The survey is aimed at researchers in the areas of computer graphics, computer vision and CH computing, as well as to scholars and practitioners in the CH field. The problems considered include shape perception enhancement, restoration and preservation support, monitoring over time, object interpretation and collection analysis. All of these problems typically rely on an understanding of the structure of the shapes in question at both a local and global level. In this survey, we discuss the different problem forms and review the main solution methods, aided by classification criteria based on the geometric scale at which the analysis is performed and the cardinality of the relationships among object parts exploited during the analysis. We finalize the report by discussing open problems and future perspectives.

While the concept of visual saliency has been previously explored in the areas of mesh and image processing, saliency detection also applies to other sensory stimuli. In this paper, we explore the problem of tactile mesh saliency, where we define salient points on a virtual mesh as those that a human is more likely to grasp, press, or touch if the mesh were a real-world object. We solve the problem of taking as input a 3D mesh and computing the relative tactile saliency of every mesh vertex. Since it is difficult to manually define a tactile saliency measure, we introduce a crowdsourcing and learning framework. It is typically easy for humans to provide relative rankings of saliency between vertices rather than absolute values. We thereby collect crowdsourced data of such relative rankings and take a learning-to-rank approach. We develop a new formulation to combine deep learning and learning-to-rank methods to compute a tactile saliency measure. We demonstrate our framework with a variety of 3D meshes and various applications including material suggestion for rendering and fabrication.

Texture synthesis is a well-established area, with many important applications in computer graphics and vision. However, despite their success, synthesis techniques are not used widely in practice because the creation of good exemplars remains challenging and extremely tedious. In this paper, we introduce an unsupervised method for analyzing texture content across multiple scales that automatically extracts good exemplars from natural images. Unlike existing methods, which require extensive manual tuning, our method is fully automatic. This allows the user to focus on using texture palettes derived from their own images, rather than on manual interactions dictated by the needs of an underlying algorithm. Most natural textures exhibit patterns at multiple scales that may vary according to the location (non-stationarity). To handle such textures many synthesis algorithms rely on an analysis of the input and a guidance of the synthesis. Our new analysis is based on a labeling of texture patterns that is both (i) multi-scale and (ii) unsupervised -- that is, patterns are labeled at multiple scales, and the scales and the number of labeled clusters are selected automatically. Our method works in two stages. The first builds a hierarchical extension of superpixels and the second labels the superpixels based on random walk in a graph of similarity between superpixels and a nonnegative matrix factorization. Our label-maps provide descriptors for pixels and regions that benefit state-of-the-art texture synthesis algorithms. We show several applications including guidance of non-stationary synthesis, content selection and texture painting. Our method is designed to treat large inputs and can scale to many megapixels. In addition to traditional exemplar inputs, our method can also handle natural images containing different textured regions.

It is known that humans can be insensitive to large changes in illumination. For example, if an object of interest is extracted from one digital photograph and inserted into another, we do not always notice the differences in illumination between the object and its new background. This inability to spot illumination inconsistencies is often the key to success in digital “doctoring” operations. We present a set of experiments in which we explore the perception of illumination in outdoor scenes. Our results can be used to predict when and why inconsistencies go unnoticed. Applications of the knowledge gained from our studies include smarter digital “cut-and-paste” and digital “fake” detection tools, and image-based composite scene backgrounds for layout and previsualization.

We propose a steganalytic algorithm for watermarks embedded by Cho et al.'s mean-based algorithm [1]. The main observation is that while in a clean model the means of Cho et al.'s normalized histogram bins are expected to follow a Gaussian distribution, in a marked model their distribution will be bimodal. The proposed algorithm estimates the number of bins through an exhaustive search and then the presence of a watermark is decided by a tailor made normality test. We also propose a modification of Cho et al.'s algorithm which is more resistant to the steganalytic attack and offers an improved robustness/capacity trade-off.

A 3D modeling system that provides all-inclusive functionality is generally too demanding for a casual 3D modeler to learn. In recent years, there has been a shift towards developing more approachable systems, with easy-to-learn, intuitive interfaces. However, most modeling systems still employ mouse and keyboard interfaces, despite the ubiquity of tablet devices, and the benefits of multi-touch interfaces applied to 3D modeling. In this paper, we introduce an alternative 3D modeling paradigm for creating developable surfaces, inspired by traditional papercrafting, and implemented as a system designed from the start for a multi-touch tablet. We demonstrate the process of assembling complex 3D scenes from a collection of simpler models, in turn shaped through operations applied to sheets of virtual paper. The modeling and assembling operations mimic familiar, real-world operations performed on paper, allowing users to quickly learn our system with very little guidance. We outline key design decisions made throughout the development process, based on feedback obtained through collaboration with target users. Finally, we include a range of models created in our system.

The BTF data structure was a breakthrough for appearance modeling in computer graphics. More research is needed though to make BTFs practical in rendering applications. We present the first systematic study of the effects of Approximate filtering on the appearance of BTFs, by exploring the spatial, angular and temporal domains over a varied set of stimuli. We perform our initial experiments on simple geometry and lighting, and verify our observations on more complex settings. We consider multi-dimensional filtering versus conventional mipmapping, and find that multi-dimensional filtering produces superior results. We examine the tradeoff between under- and oversampling, and find that different filtering strategies can be applied in each domain, while maintaining visual equivalence with respect to a ground truth. For example, we find that preserving contrast is more important in static than dynamic images, indicating greater levels of spatial filtering are possible for animations. We find that filtering can be performed more aggressively in the angular domain than in the spatial. Additionally, we find that high-level visual descriptors of the BTF are linked to the perceptual performance of pre-filtered approximations. In turn, some of these high-level descriptors correlate with low level statistics of the BTF. We show six different practical applications of applying our findings to improving filtering, rendering and compression strategies.

Texture is an essential feature in modeling the appearance of objects and is instrumental in making virtual objects appear interesting and/or realistic. Unfortunately, obtaining textures is a labor intensive task requiring parameter tuning for procedural methods or careful photography and post-processing for natural images. Many texture synthesis techniques have been developed to generate textures of arbitrary spatial extent, but these techniques require the user to first produce an exemplar consisting solely of the desired texture. We present a fast method using diffusion manifolds to locate textures in unconstrained photographs, and extract exemplar tiles. The method requires the user to only specify a single point within the image containing the desired texture and the scale of the desired texture. The user may tune the result using simple interactions. The method is non-local, in the sense that the desired texture does not have to appear in a single contiguous region in the source image. This document is a summary of our work and system. A full account is available online.

Abstract Directors employ a process called “color grading” to add color styles to feature films. Color grading is used for a number of reasons, such as accentuating a certain emotion or expressing the signature look of a director. We collect a database of feature film clips and label them with tags such as director, emotion, and genre. We then learn a model that maps from the low‐level color and tone properties of film clips to the associated labels. This model allows us to examine a number of common hypotheses on the use of color to achieve goals, such as specific emotions. We also describe a method to apply our learned color styles to new images and videos. Along with our analysis of color grading techniques, we demonstrate a number of images and videos that are automatically filtered to resemble certain film styles.

One major shortcoming of existing bi-scale material design systems is the lack of support for inverse design: there is no way to directly edit the large-scale appearance and then rapidly solve for the small-scale details that approximate that look. Prior work is either too slow to provide quick feedback, or limited in the types of small-scale details that can be handled. We present a novel computational framework for inverse bi-scale material design. The key idea is to convert the challenging inverse appearance computation into efficient search in two precomputed large libraries: one including a wide range of measured and analytical materials, and the other procedurally generated and height-map-based geometries. We demonstrate a variety of editing operations, including finding visually equivalent details that produce similar large-scale appearance, which can be useful in applications such as physical fabrication of materials.

Abstract A common weathering effect is the appearance of cracks due to material fractures. Previous exemplar‐based aging and weathering methods have either reused images or sought to replicate observed patterns exactly. We introduce a new approach to exemplar‐based modeling that creates weathered patterns on synthetic objects by matching thestatisticsof fracture patterns in a photograph. We present a user study to determine which statistics are correlated to visual similarity and how they are perceived by the user. We then describe a revised physically‐based fracture model capable of producing a wide range of crack patterns at interactive rates. We demonstrate how a Bayesian optimization method can determine the parameters of this model so it can produce a pattern with the same key statistics as an exemplar. Finally, we present results using our approach and various exemplars to produce a variety of fracture effects in synthetic renderings of complex environments. The speed of the fracture simulation allows interactive previews of the fractured results and its application on large scale environments.

Sophisticated methods for true spectral rendering have been developed in computer graphics to produce highly accurate images. In addition to traditional applications in visualizing appearance, such methods have potential applications in many areas of scientific study. In particular, we are motivated by the application of studying avian vision and appearance. An obstacle to using graphics in this application is the lack of reliable input data. We introduce an end-to-end measurement system for capturing spectral data on 3D objects. We present the modification of a recently developed hyperspectral imager to make it suitable for acquiring such data in a wide spectral range at high spectral and spatial resolution. We capture four megapixel images, with data at each pixel from the near-ultraviolet (359 nm) to near-infrared (1,003 nm) at 12 nm spectral resolution. We fully characterize the imaging system, and document its accuracy. This imager is integrated into a 3D scanning system to enable the measurement of the diffuse spectral reflectance and fluorescence of specimens. We demonstrate the use of this measurement system in the study of the interplay between the visual capabilities and appearance of birds. We show further the use of the system in gaining insight into artifacts from geology and cultural heritage.

Compositing is one of the most commonly performed operations in computer graphics. A realistic composite requires adjusting the appearance of the foreground and background so that they appear compatible; unfortunately, this task is challenging and poorly understood. We use statistical and visual perception experiments to study the realism of image composites. First, we evaluate a number of standard 2D image statistical measures, and identify those that are most significant in determining the realism of a composite. Then, we perform a human subjects experiment to determine how the changes in these key statistics influence human judgements of composite realism. Finally, we describe a data-driven algorithm that automatically adjusts these statistical measures in a foreground to make it more compatible with its background in a composite. We show a number of compositing results, and evaluate the performance of both our algorithm and previous work with a human subjects study.

Abstract Bidirectional texture functions (BTFs) represent the appearance of complex materials. Three major shortcomings with BTFs are the bulky storage, the difficulty in editing and the lack of efficient rendering methods. To reduce storage, many compression techniques have been applied to BTFs, but the results are difficult to edit. To facilitate editing, analytical models have been fit, but at the cost of accuracy of representation for many materials. It becomes even more challenging if efficient rendering is also needed. We introduce a high‐quality general representation that is, at once, compact, easily editable, and can be efficiently rendered. The representation is computed by adopting the stagewise Lasso algorithm to search for a sparse set of analytical functions, whose weighted sum approximates the input appearance data. We achieve compression rates comparable to a state‐of‐the‐art BTF compression method. We also demonstrate results in BTF editing and rendering.

Abstract The appearance of weathering effects on stone is important for creating outdoor scenes in computer graphics. To achieve them, previous research has built upon physical simulation, which, while yielding a degree of realism, is computationally expensive and inapplicable to the situation when the object geometry is unknown. Also, physical simulation requires specific knowledge of the stone properties and environmental processes. In this paper, we present a simple visual simulation pipeline for creating weathering effects on stone within a single image. Two primary effects of stone weathering, i.e., smoothing and roughening, are considered. In addition, erosion on the object silhouette is treated. These challenging effects involve significant geometry changes, which are intractable for previous image‐based editing techniques. The effectiveness of our technique is illustrated on a variety of scenes and types of stone. While it can be fully automatic, it also allows easy user interaction.

The simulation of weathered appearance is essential in the realistic modeling of urban environments. A representative and particularly difficult effect to produce on a large scale is the effect of fluid flow. Changes in appearance due to flow are the result of both the global effect of large-scale shape, and local effects, such as the detailed roughness of a surface. With digital photography and Internet image collections, visual examples of flow effects are readily available. These images, however, mix the appearance of flows with the specific local context. We present a methodology to extract parameters and detail maps from existing imagery in a form that allows new target-specific flow effects to be produced, with natural variations in the effects as they are applied in different locations in a new scene. In this article, we focus on producing a library of parameters and detail maps for generating flow patterns; and this methodology can be used to extend the library with additional image exemplars. To illustrate our methodology, we show a rich collection of patterns applied to urban models.

Architecture is design in spatial context. The only current methods for representing context involve designing in a heavyweight computer-aided design system, using a full model of existing buildings and landscape, or sketching on a panoramic photo. The former is too cumbersome; the latter is too restrictive in viewpoint and in the handling of occlusions and topography. We introduce a novel approach to presenting context such that it is an integral component in a lightweight conceptual design system. We represent sites through a fusion of data available from different sources. We derive a site model from geographic elevation data, on-site point-to-point distance measurements, and images of the site. To acquire and process the data, we use publicly available data sources, multidimensional scaling techniques and refinements of recent bundle adjustment techniques. We offer a suite of interactive tools to acquire, process, and combine the data into a lightweight stroke and image-billboard representation. We create multiple and linked pop-ups derived from images, forming a lightweight representation of a three-dimensional environment. We implemented our techniques in a stroke-based conceptual design system we call Insitu . We developed our work through continuous interaction with professional designers. We present designs created with our new techniques integrated in a conceptual design system.

We present the first physically-based interactive system to facilitate the appearance design at different scales consistently, through manipulations of both small-scale geometry and materials. The core of our system is a novel reflectance filtering algorithm, which rapidly computes the large-scale appearance from small-scale details, by exploiting the low-rank structures of the Bidirectional Visible Normal Distribution Function and pre-rotated BRDFs in the matrix formulation of our rendering problem. Our algorithm is three orders of magnitude faster than a ground-truth method. We demonstrate various editing results of different small-scale geometry with analytical and measured BRDFs. In addition, we show the applications of our system to physical realization of appearance, as well as modeling of real-world materials using very sparse measurements.

Abstract Papercraft models can serve as inexpensive prototypes in shape design applications. However, in making the models some geometric detail is necessarily lost, and artificial creases may be visible, thereby limiting the utility of these models. To compensate for these practical limitations, we introduce the use of printed patterns on papercraft models to enhance the perception of the shape they are intended to represent. We propose pattern generation schemes that modulate the sizes, directions, and densities of glyphs of patterns based on geometric attributes. We present a psychophysical experiment designed to explore the effect that printed patterns have on the perception of the papercraft model shapes. We find that models with printed patterns are perceived to represent the intended shape more accurately, and, further, that the type of printed pattern has an impact on the perceived shape.

Abstract Texture synthesis techniques require nearly uniform texture samples, however identifying suitable texture samples in an image requires significant data preprocessing. To eliminate this work, we introduce a fully automatic pipeline to detect dominant texture samples based on a manifold generated using the diffusion distance. We define the characteristics of dominant texture and three different types of outliers that allow us to efficiently identify dominant texture in feature space. We demonstrate how this method enables the analysis/synthesis of a wide range of natural textures. We compare textures synthesized from a sample image, with and without dominant texture detection. We also compare our approach to that of using a texture segmentation technique alone, and to using Euclidean, rather than diffusion, distances between texture features.

Abstract Extremely dense spatial sampling is often needed to prevent aliasing when rendering objects with high frequency variations in geometry and reflectance. To accelerate the rendering process, we introduce characteristic point maps (CPMs), a hierarchy of view‐independent points, which are chosen to preserve the appearance of the original model across different scales. In preprocessing, randomized matrix column sampling is used to reduce an initial dense sampling to a minimum number of characteristic points with associated weights. In rendering, the reflected radiance is computed using a weighted average of reflectances from characteristic points. Unlike existing techniques, our approach requires no restrictions on the original geometry or reflectance functions.

In modern computer graphics applications, textures play an important role in conveying the appearance of real-world materials. But while surface appearance can often be effectively captured with a photograph, it is difficult to use example imagery to synthesize fully three-dimensional (3D) solid textures that are perceptually similar to their inputs. Specifically, this research focuses on human perception of 3D solid textures composed of aggregate particles in a binding matrix. Holding constant an established algorithm for approximating particledistributions, we examine the problem of estimating particleshape. We consider four methods for approximating plausible particle shapes—including two methods of our own contribution. We compare the performance of these methods under a variety of input conditions using automated, perceptually motivated metrics, as well as a psychophysical experiment. In the course of assessing the relative performance of the four algorithms, we also evaluate the reliability of the automated metrics in predicting the results of the experiment.

We describe a computer graphics system that supports conceptual architectural design and analysis. We use as a starting point the traditional sketchbook drawings that architects use to experiment with various views, sections, and details. Rather than interpret or infer 3D structure from drawings, our system is designed to allow the designer to organize concept drawings in 3D, and gradually fuse a series of possibly geometrically-inconsistent sketches into a set of 3D strokes. Our system uses strokes and planar "canvases" as basic primitives; the basic mode of input is traditional 2D drawing. We introduce methods for the user to control stroke visibility and transfer strokes between canvases. We also introduce methods for the user to position and orient the canvases that have infinite extent. We demonstrate the use of the system to analyze existing structures and conceive new designs.

Interesting textures form on the surfaces of objects as the result of external chemical, mechanical, and biological agents. Simulating these textures is necessary to generate models for realistic image synthesis. The textures formed are progressively variant, with the variations depending on the global and local geometric context. We present a method for capturing progressively varying textures and the relevant context parameters that control them. By relating textures and context parameters, we are able to transfer the textures to novel synthetic objects. We present examples of capturing chemical effects, such as rusting; mechanical effects, such as paint cracking; and biological effects, such as the growth of mold on a surface. We demonstrate a user interface that provides a method for specifying where an object is exposed to external agents. We show the results of complex, geometry-dependent textures evolving on synthetic objects.

HIS special section contains extended versions of nine papers published at the IEEE Visualization 2005 (VIS 2005) conference. The program committee cochairs selected these papers by taking into account the detailed reviews from external referees and program committee members. Authors were invited to submit a substantially revised and extended version of their manuscripts. Each submitted paper went through the complete IEEE Transactions on Visualization and Computer Graphics (TVCG) peer review process, including multiple rounds of reviews. For VIS 2005, the application and research papers were folded into a single track, and all papers were published together, without explicit classification information. At the conference, the talks were organized by topic, with both application and research papers presented in the same session, when appropriate. This special section contains revised versions of both application and research papers. The first paper, “Views on Visualization,” by Jarke J. van Wijk, is based on his “best research paper award winning” paper. It provides a thought-provoking look at the state of visualization research, with an emphasis on the effectiveness of visualization techniques on a number of particular applications. It builds on the author’s extensive experience and stature as one of the leading figures in the field, having published extensively in the IEEE Visualization and Information Visualization conferences. We note that this is not a typical research paper, however, it is important reading for anyone who has interest in visualization as a scientific discipline. The goal of visualization is not necessarily to generate beautiful pictures, but to create insightful visual representations that faithfully represent the scientific truth. Issues of precision are extremely important. Xiaoru Yuan, Minh. X. Nguyen, Baoquan Chen, and David H. Porter present a set of techniques for faithfully rendering high-dynamic volume data in their paper, “HDR VolVis: High Dynamic Range Volume Visualization.” They received the best application paper award for their groundbreaking work in this area. Their paper provides convincing evidence that properly handling precision and dynamic range issues are key to maintaining the integrity of the scientific data, and are bound to be ever more important as the field expands. In modern applications, many numerical solutions of simulations of experiments are done using high-order basis functions.Thistrendislikelytoincreaseasweunderstandthe numericalmethodsbetter,andareabletodevelopmorestable and efficient techniques for handling high-order finite elements, which often lead to a substantially smaller number of elementsfor agiven accuracy of the solution. The paper by

No abstract available.

We describe the use of traditional stereological methods to synthesize 3D solid textures from 2D images of existing materials. We first illustrate our approach for aggregate materials of spherical particles, and then extend the technique to apply to particles of arbitrary shapes. We demonstrate the effectiveness of the approach with side-by-side comparisons of a real material and a synthetic model with its appearance parameters derived from its physical counterpart. Unlike ad hoc methods for texture synthesis, stereology provides a disciplined, systematic basis for predicting material structure with well-defined assumptions.

Three‐dimensional (3D) image acquisition systems are rapidly becoming more affordable, especially systems based on commodity electronic cameras. At the same time, personal computers with graphics hardware capable of displaying complex 3D models are also becoming inexpensive enough to be available to a large population. As a result, there is potentially an opportunity to consider new virtual reality applications as diverse as cultural heritage and retail sales that will allow people to view realistic 3D objects on home computers. Although there are many physical techniques for acquiring 3D data—including laser scanners, structured light and time‐of‐flight—there is a basic pipeline of operations for taking the acquired data and producing a usable numerical model. We look at the fundamental problems of range image registration, line‐of‐sight errors, mesh integration, surface detail and color, and texture mapping. In the area of registration we consider both the problems of finding an initial global alignment using manual and automatic means, and refining this alignment with variations of the Iterative Closest Point methods. To account for scanner line‐of‐sight errors we compare several averaging approaches. In the area of mesh integration, that is finding a single mesh joining the data from all scans, we compare various methods for computing interpolating and approximating surfaces. We then look at various ways in which surface properties such as color (more properly, spectral reflectance) can be extracted from acquired imagery. Finally, we examine techniques for producing a final model representation that can be efficiently rendered using graphics hardware.

Rendering is the process of creating an image from numerical input data. In the past few years our ideas about methods for acquiring the input data and the form of the output have expanded. The availability of inexpensive cameras and scanners has influenced how we can obtain data needed for rendering. Input for rendering ranges from sets of images to complex geometric descriptions with detailed BRDF data. The images that are rendered may be simply arrays of RGB images, or they may be arrays with vectors or matrices of data defined for each pixel. The rendered images may not be intended for direct display, but may be textures for geometries that are to be transmitted to be rendered on another system. A broader range of parameters now need to be taken into account to render images that are perceptually consistent across displays that range from CAVEs to personal digital assistants. This presentation will give an overview of how new hardware and new applications have changed traditional ideas of rendering input and output.

We present a method for computing horizon maps from captured images of a bumpy surface. 1Horizon maps encode surface self-shadowing effects, and can be used with bump or normals maps to realistically render surfaces with small height perturbations. The method does not rely on complete surface reconstruction, and requires only eight captured images as input. In this paper we discuss how shadow information is extrapolated from the eight captured images to compute the horizon map. Our implementation accounts for the noise and uncertainties in physically acquired data.

The creation of three-dimensional digital content by scanning real objects has become common practice in graphics applications for which visual quality is paramount, such as animation, e-commerce, and virtual museums. While a lot of attention has been devoted recently to the problem of accurately capturing the geometry of scanned objects, the acquisition of high-quality textures is equally important, but not as widely studied. In this paper, we focus on methods to construct accurate digital models of scanned objects by integrating high-quality texture and normal maps with geometric data. These methods are designed for use with inexpensive, electronic camera-based systems in which low-resolution range images and high-resolution intensity images are acquired. The resulting models are well-suited for interactive rendering on the latest-generation graphics hardware with support for bump mapping. Our contributions include new techniques for processing range, reflectance, and surface normal data, for image-based registration of scans, and for reconstructing high-quality textures for the output digital object.

The Ball-Pivoting Algorithm (BPA) computes a triangle mesh interpolating a given point cloud. Typically, the points are surface samples acquired with multiple range scans of an object. The principle of the BPA is very simple: Three points form a triangle if a ball of a user-specified radius p touches them without containing any other point. Starting with a seed triangle, the ball pivots around an edge (i.e., it revolves around the edge while keeping in contact with the edge's endpoints) until it touches another point, forming another triangle. The process continues until all reachable edges have been tried, and then starts from another seed triangle, until all points have been considered. The process can then be repeated with a ball of larger radius to handle uneven sampling densities. We applied the BPA to datasets of millions of points representing actual scans of complex 3D objects. The relatively small amount of memory required by the BPA, its time efficiency, and the quality of the results obtained compare favorably with existing techniques.

N this issue, we present outstanding papers from the I IEEE Visualization '98 Conference held in Research Triangle Park, North Carolina. The papers have been substantially extended from the form presented at the conference. The extended versions have undergone a thorough review process and have been revised according to thoughtful comments by expert referees. The papers included here are a sampling of the diverse topics represented at the conference. We only have room in this issue to highlight some of the major trends in current visualization research. Two of the papers selected deal with the management and display of fundamental graphics primitives used in visualization-polygonal meshes and voxels. The other two papers deal with constructing the appropriate primitives for a visualization-feature extraction and the mapping of visual attributes to data. Many visualization problems involve the display of large polygonal meshes. In the past few years, a number of geometric simplification algorithms have been developed to facilitate the interactive examination of large meshes. Lindstrom and Turk present a thorough examination of their memoryless simplification approach. An important aspect of their paper is a comparison of memoryless simplification to several other recently published simplification algorithms. Volume visualization remains the approach of choice for many problems. Mueller et al. consider the splatting algorithm for volume visualization. They introduce new variations of this approach that increase accuracy and efficiency. The new variations eliminate the popping artifact that has been a problem when splatting is used to generate animated sequences. In data visualization, it is not adequate to find just a way to map data to graphics primitives for display. Brute force visualizations of extensive simulations can leave the user with large quantities of imagery to search through looking for phenomena of interest. Feature extraction techniques have emerged to distill the quantity of data to be mapped to images. Kenwright et al. present a feature extraction algorithm for indentifying key characteristics of 3D vector fields. While their specific application is fluid flow over aircraft, the technique can generally be applied to gain insight into 3D vector fields that arise in various disciplines. Constructing a visualization requires mapping data attributes to visual attributes. Not all possible mappings are

We describe a set of visualization programs developed for understanding segmentations of customer records produced by a self organizing map (SOM) algorithm. A SOM produces segments of similar customer records that can then be used as the basis of a marketing campaign. Since the characteristics that each segment will have in common are not specified a priori, visualization is essential to understanding the segment to design specific marketing strategies. Two different styles of visualizations were found to be useful for the two types of observers of the data. Abstract overviews of the entire segmentation were designed for analysts applying the SOM algorithm. Detailed scatterplots of individual records were designed for communicating the results to decision makers specifying marketing strategy.

We present a tone reproduction operator that preserves visibility in high dynamic range scenes. Our method introduces a new histogram adjustment technique, based on the population of local adaptation luminances in a scene. To match subjective viewing experience, the method incorporates models for human contrast sensitivity, glare, spatial acuity, and color sensitivity. We compare our results to previous work and present examples of our techniques applied to lighting simulation and electronic photography.

The authors describe a graphical system developed for researchers in materials science for extracting information from data obtained by atomic force microscopy. In particular, they consider the problem of computing surface orientations from data obtained from ceramic materials. The visualization problems they consider in designing this system include finding useful mechanisms for the researcher to interact with the data, presenting results in forms familiar to the scientist, and enhancing traditional display techniques.

We describe a new framework for efficiently computing and storing global illumination effects for complex, animated environments. The new framework allows the rapid generation of sequences representing any arbitrary path in a "view space" within an environment in which both the viewer and objects move. The global illumination is stored as time sequences of range-images at base locations that span the view space. We present algorithms for determining locations for these base images, and the time steps required to adequately capture the effects of object motion. We also present algorithms for computing the global illumination in the base images that exploit spatial and temporal coherence by considering direct and indirect illumination separately. We discuss an initial implementation using the new framework. Results and analysis of our implementation demonstrate the effectiveness of the individual phases of the approach; we conclude with an application of the complete framework to a complex environment that includes object motion.

Monte Carlo techniques for image synthesis are simple and powerful, but they are prone to noise from inadequate sampling. This paper describes a class of non-linear filters that remove sampling noise in synthetic images without removing salient features. This is achieved by spreading real input sample values into the output image via variable-width filter kernels, rather than gathering samples into each output pixel via a constant-width kernel. The technique is nonlinear because kernel widths are based on sample magnitudes, and this local redistribution of values cannot generally be mapped to a linear function. Nevertheless, the technique preserves energy because the kernels are normalized, and all input samples have the same average influence on the output. To demonstrate its effectiveness, the new filtering method is applied to two rendering techniques. The first is a Monte Carlo path tracing technique with the conflicting goals of keeping pixel variance below a specified limit and finishing in a finite amount of time; this application shows how the filter may be used to “clean up” areas where it is not practical to sample adequately. The second is a hybrid deterministic and Monte Carlo ray-tracing program; this application shows how the filter can be effective even when the pixel variance is not known.

We describe how techniques from computer graphics are used to visualize pool fire data and compute radiative effects from pool fires. The basic tools are ray casting and accurate line integration using the RADCAL program. Example images in the visible and infrared band are shown which are given of irradiation calculations and novel methods to visualize the results of irradiation calculations.>

A new progressive global illumination method is presented which produces approximate images quickly, and then continues to systematically produce more accurate images. The method combines the existing methods of progressive refinement radiosity, Monte Carlo path tracing and light ray tracing. The method does not place any limitation on surface properties such as ideal Lambertian or mirror-like. To increase efficiency and accuracy, the new concepts of light source reclassification, caustics reconstruction, Monte Carlo path tracing with a radiosity preprocess and an interruptible radiosity solution are introduced. The method presents the user with most useful information about the scene as early as possible by reorganizing the method into a radiosity pass, a high frequency refinement pass and a low frequency refinement pass. The implementation of the method is demonstrated, and sample images are presented.

An extension of the radiosity method is presented that rigorously accounts for the presence of a small number of specularly reflecting surfaces in an otherwise diffuse scene, and for the presence of a small number of specular or ideal diffuse transmitters. The relationship between the extended method and earlier radiosity and ray-tracing methods is outlined. It is shown that all three methods are based on the same general equation of radiative transfer. A simple superposition of the earlier radiosity and ray-tracing methods in order to account for specular behavior is shown to be physically inconsistent, as the methods are based on different assumptions. Specular behavior is correctly included in the present method. The extended radiosity method and example images are presented.

The zonal method for calculating radiative transfer in the presence of a participating medium is applied to the generation of realistic synthetic images. The method generalizes the radiosity method and allows for emission, scattering, and absorption by a participating medium. The zonal method accounts for volume/surface interactions which have not been previously included, as well as volume/volume and surface/surface interactions. In addition, new algorithms, based on the hemi-cube formulation, are introduced for calculating the geometric factors required by the zonal method.

Accurate simulation of light propagation within an environment and perceptually based imaging techniques are necessary for the creation of realistic images. A physical experiment that verifies the simulation of reflected light intensities for diffuse environments was conducted. Measurements of radiant energy flux densities are compared with predictions using the radiosity method for those physical environments. By using color science procedures the results of the light model simulation are then transformed to produce a color television image. The final image compares favorably with the original physical model. The experiment indicates that, when the physical model and the simulation were viewed through a view camera, subjects could not distinguish between them. The results and comparison of both test procedures are presented within this paper.