Abstract We review recent methods in 2D creative pattern generation and their control mechanisms, focusing on procedural methods. The review is motivated by an artist's perspective and investigates interactive pattern generation as a complex design problem. While the repetitive nature of patterns is well‐suited to algorithmic creation and automation, an artist needs more flexible control mechanisms for adaptable and inventive designs. We organize the state of the art around pattern design features, such as repetition, frames, curves, directionality, and single visual accents. Within those areas, we summarize and discuss the techniques' control mechanisms for enabling artist intent. The discussion includes questions of how input is given by the artist, what type of content the artist inputs, where the input affects the canvas spatially, and when input can be given in the timeline of the creation process. We categorize the available control mechanisms on an algorithmic level and categorize their input modes based on exemplars, parameterization, handling, filling, guiding, and placing interactions. To better understand the potential of the current techniques for creative design and to make such an investigation more manageable, we motivate our discussion with how navigation, transparency, variation, and stimulation enable creativity. We conclude our review by identifying possible new directions that can inspire innovation for artist‐centered creation processes and algorithms.

Light fields are a powerful concept in computational imaging and a mainstay in image-based rendering; however, so far their acquisition required either carefully designed and calibrated optical systems (micro-lens arrays), or multi-camera/multi-shot settings. Here, we show that fully calibrated light field data can be obtained from a single ordinary photograph taken through a partially wetted window. Each drop of water produces a distorted view on the scene, and the challenge of recovering the unknown mapping from pixel coordinates to refracted rays in space is a severely underconstrained problem. The key idea behind our solution is to combine ray tracing and low-level image analysis techniques (extraction of 2D drop contours and locations of scene features seen through drops) with state-of-the-art drop shape simulation and an iterative refinement scheme to enforce photo-consistency across features that are seen in multiple views. This novel approach not only recovers a dense pixel-to-ray mapping, but also the refractive geometry through which the scene is observed, to high accuracy. We therefore anticipate that our inherently self-calibrating scheme might also find applications in other fields, for instance in materials science where the wetting properties of liquids on surfaces are investigated.

Abstract The choice of parameters for procedural textures to achieve a desired appearance poses a challenging problem even for experienced artists. We propose a method to automatically determine such parameters to reproduce the appearance of input images. Addressing two‐tone textures, we separate the estimation of color and structure information and interpret the problem as image retrieval task from the space of procedural outputs. Applying a perceptually motivated image metric based on a texture descriptor enables us to precompute a comprehensive collection of possible parameter sets and yet achieve interactive retrieval performance. Our method supports a large variety of procedural texture models with a unified approach.

Abstract The high resolution of digital cameras has made single‐shot, single‐sensor acquisition of light fields feasible, though considerable design effort is still necessary in order to construct the necessary collection of optical elements for particular acquisition scenarios. This paper explores a pipeline for designing, fabricating and utilizing faceted mirror arrays which simplifies this task. The foundation of the pipeline is an interactive tool that automatically optimizes for mirror designs while exposing to the user a set of intuitive parameters for light field quality and manufacturing constraints. We investigate two manufacturing processes for automatic fabrication of the resulting designs: one is based on CNC milling, polishing, and plating of one solid work piece, while the other involves assembly of CNC‐cut mirror facets. We demonstrate results for refocusing in a macro photography scenario. In addition, we observe that traditional photographic parameters take novel roles in the faceted mirror array setup and discuss their influence.

We investigate a complete pipeline for measuring, modeling, and fabricating objects with specified subsurface scattering behaviors. The process starts with measuring the scattering properties of a given set of base materials, determining their radial reflection and transmission profiles. We describe a mathematical model that predicts the profiles of different stackings of base materials, at arbitrary thicknesses. In an inverse process, we can then specify a desired reflection profile and compute a layered composite material that best approximates it. Our algorithm efficiently searches the space of possible combinations of base materials, pruning unsatisfactory states imposed by physical constraints. We validate our process by producing both homogeneous and heterogeneous composites fabricated using a multi-material 3D printer. We demonstrate reproductions that have scattering properties approximating complex materials.

Traditional flat screen displays present 2D images. 3D and 4D displays have been proposed making use of lenslet arrays to shape a fixed outgoing light field for horizontal or bidirectional parallax. In this article, we present different designs of multi-dimensional displays which passively react to the light of the environment behind. The prototypes physically implement a reflectance field and generate different light fields depending on the incident illumination, for example light falling through a window. We discretize the incident light field using an optical system, and modulate it with a 2D pattern, creating a flat display which is view and illumination-dependent. It is free from electronic components. For distant light and a fixed observer position, we demonstrate a passive optical configuration which directly renders a 4D reflectance field in the real-world illumination behind it. We further propose an optical setup that allows for projecting out different angular distributions depending on the incident light direction. Combining multiple of these devices we build a display that renders a 6D experience, where the incident 2D illumination influences the outgoing light field, both in the spatial and in the angular domain. Possible applications of this technology are time-dependent displays driven by sunlight, object virtualization and programmable light benders / ray blockers without moving parts.

The quality of a 3D range scan should not depend on the surface properties of the object. Most active range scanning techniques, however, assume a diffuse reflector to allow for a robust detection of incident light patterns. In our approach we embed the object into a fluorescent liquid. By analyzing the light rays that become visible due to fluorescence rather than analyzing their reflections off the surface, we can detect the intersection points between the projected laser sheet and the object surface for a wide range of different materials. For transparent objects we can even directly depict a slice through the object in just one image by matching its refractive index to the one of the embedding liquid. This enables a direct sampling of the object geometry without the need for computational reconstruction. This way, a high-resolution 3D volume can be assembled simply by sweeping a laser plane through the object. We demonstrate the effectiveness of our light sheet range scanning approach on a set of objects manufactured from a variety of materials and material mixes, including dark, translucent and transparent objects.

Abstract Captured reflectance fields tend to provide a relatively coarse sampling of the incident light directions. As a result, sharp illumination features, such as highlights or shadow boundaries, are poorly reconstructed during relighting; highlights are disconnected, and shadows show banding artefacts. In this paper, we propose a novel interpolation technique for 4D reflectance fields that reconstructs plausible images even for non‐observed light directions. Given a sparsely sampled reflectance field, we can effectively synthesize images as they would have been obtained from denser sampling. The processing pipeline consists of three steps: (1) segmentation of regions where intermediate lighting cannot be obtained by blending, (2) appropriate flow algorithms for highlights and shadows, plus (3) a final reconstruction technique that uses image‐based priors to faithfully correct errors that might be introduced by the segmentation or flow step. The algorithm reliably reproduces scenes that contain specular highlights, interreflections, shadows or caustics.

Image-based relighting achieves high quality in rendering, but it requires a large number of measurements of the reflectance field. This article discusses sampling techniques that improve on the trade-offs between measurement effort and reconstruction quality. Specifically, we (i) demonstrate that sampling with point lights and from a sparse set of incoming light directions creates artifacts which can be reduced significantly by employing extended light sources for sampling, (ii) propose a sampling algorithm which incrementally chooses light directions adapted to the properties of the reflectance field being measured, thus capturing significant features faster than fixed-pattern sampling, and (iii) combine reflectance fields from two different light domain resolutions. We present an automated measurement setup for well-defined angular distributions of the incident, indirect illumination. It is based on programmable spotlights with controlled aperture that illuminate the walls around the scene.

In this paper, we present an image-based framework that acquires the reflectance properties of a human face. A range scan of the face is not required. Based on a morphable face model, the system estimates the 3D shape and establishes point-to-point correspondence across images taken from different viewpoints and across different individuals' faces. This provides a common parameterization of all reconstructed surfaces that can be used to compare and transfer BRDF data between different faces. Shape estimation from images compensates deformations of the face during the measurement process, such as facial expressions. In the common parameterization, regions of homogeneous materials on the face surface can be defined a priori. We apply analytical BRDF models to express the reflectance properties of each region and we estimate their parameters in a least-squares fit from the image data. For each of the surface points, the diffuse component of the BRDF is locally refined, which provides high detail. We present results for multiple analytical BRDF models, rendered at novel orientations and lighting conditions.