One augmented reality approach is to use digital projectors to alter the appearance of a physical scene, avoiding the need for head-mounted displays or special goggles. Instead, spatial augmented reality (SAR) systems depend on having sufficient light radiance to compensate the surface's colors to those of a target visualization. However, standard SAR systems in dark room settings may suffer from insufficient light radiance causing bright colors to exhibit unexpected color shifts, resulting in a misleading visualization. We introduce a SAR framework which focuses on minimally altering the appearance of arbitrarily shaped and colored objects to exploit the presence of environment/room light as an additional light source to achieve compliancy for bright colors. While previous approaches have compensated for environment light, none have explicitly exploited the environment light to achieve bright, previously incompliant colors. We implement a full working system and compared our results to solutions achievable with standard SAR systems.

We present a system that superimposes multiple projections onto an object of arbitrary shape and color to produce high-resolution appearance changes. Our system produces appearances at an improved resolution compared to prior works and can change appearances at near interactive rates. Three main components are central to our system. First, the problem of computing compensation images is formulated as a constrained optimization which yields high-resolution appearances. Second, decomposition of the target appearance into base and scale images enables fast swapping of appearances on the object by requiring the constrained optimization to be computed only once per object. Finally, to make high-quality appearance edits practical, an elliptical Gaussian is used to model projector pixels and their interaction between projectors. To the best of our knowledge, we build the first system that achieves high-resolution and high-quality appearance edits using multiple superimposed projectors on complex nonplanar colored objects. We demonstrate several appearance edits including specular lighting, subsurface scattering, inter-reflections, and color, texture, and geometry changes on objects with different shapes and colors.

Abstract Projection‐based appearances are used in a variety of computer graphics applications to impart different appearances onto physical surfaces using digitally controlled projector light. To achieve a compliant appearance, all points on the physical surface must be altered to the colours of the desired target appearance; otherwise, an incompliant appearance results in a misleading visualization. Previous systems typically assume to operate with compliant appearances or restrict themselves to the simpler case of white surfaces. To achieve compliancy, one may change the physical surface's albedo, increase the amount of projector light radiance available or modify the target appearance's colours. This paper presents an approach to modify a target appearance to achieve compliant appearance editing without altering the physical surface or the projector setup. Our system minimally alters the target appearance's colours while maintaining cues important for perceptual similarity (e.g. colour constancy). First, we discuss how to measure colour compliancy. Next, we describe our approach to partition the physical surface into patches based on the surface's colours and the target appearance's colours. Finally, we describe our appearance optimization process, which computes a compliant appearance that is as perceptually similar as possible to the target appearance's colours. We perform several real‐world projection‐based appearances and compare our results to naïve approaches, which either ignore compliancy or simply reduce the appearance's overall brightness.

Many visualization applications benefit from displaying content on real-world objects rather than on a traditional display (e.g., a monitor). This type of visualization display is achieved by projecting precisely controlled illumination from multiple projectors onto the real-world colored objects. For such a task, the placement of the projectors is critical in assuring that the desired visualization is possible. Using ad hoc projector placement may cause some appearances to suffer from color shifting due to insufficient projector light radiance being exposed onto the physical surface. This leads to an incorrect appearance and ultimately to a false and potentially misleading visualization. In this paper, we present a framework to discover the optimal position and orientation of the projectors for such projection-based visualization displays. An optimal projector placement should be able to achieve the desired visualization with minimal projector light radiance. When determining optimal projector placement, object visibility, surface reflectance properties, and projector-surface distance and orientation need to be considered. We first formalize a theory for appearance editing image formation and construct a constrained linear system of equations that express when a desired novel appearance or visualization is possible given a geometric and surface reflectance model of the physical surface. Then, we show how to apply this constrained system in an adaptive search to efficiently discover the optimal projector placement which achieves the desired appearance. Constraints can be imposed on the maximum radiance allowed by the projectors and the projectors' placement to support specific goals of various visualization applications. We perform several real-world and simulated appearance edits and visualizations to demonstrate the improvement obtained by our discovered projector placement over ad hoc projector placement.

In this paper, we introduce a system to virtually restore damaged or historically significant objects without needing to physically change the object in any way. Our work addresses both creating a restored synthetic version of the object as viewed from a camera and projecting the necessary light, using digital projectors, to give the illusion of the object being restored. The restoration algorithm uses an energy minimization method to enforce a set of criteria over the surface of the object and provides an interactive tool to the user which can compute a restoration in a few minutes. The visual compensation method develops a formulation that is particularly concerned with obtaining bright compensations under a specified maximum amount of light. The bound on the amount of light is of crucial importance when viewing and restoring old and potentially fragile objects. Finally, we demonstrate our system by restoring several deteriorated and old objects enabling the observer to view the original or restored object at will.