We describe an augmented reality, optical see-through display based on a DMD chip with an extremely fast (16 kHz) binary update rate. We combine the techniques of post-rendering 2-D offsets and just-in-time tracking updates with a novel modulation technique for turning binary pixels into perceived gray scale. These processing elements, implemented in an FPGA, are physically mounted along with the optical display elements in a head tracked rig through which users view synthetic imagery superimposed on their real environment. The combination of mechanical tracking at near-zero latency with reconfigurable display processing has given us a measured average of 80 µs of end-to-end latency (from head motion to change in photons from the display) and also a versatile test platform for extremely-low-latency display systems. We have used it to examine the trade-offs between image quality and cost (i.e. power and logical complexity) and have found that quality can be maintained with a fairly simple display modulation scheme.

We present initial results from a new image generation approach for low-latency displays such as those needed in head-worn AR devices. Avoiding the usual video interfaces, such as HDMI, we favor direct control of the internal display technology. We illustrate our new approach with a bench-top optical see-through AR proof-of-concept prototype that uses a Digital Light Processing (DLPTM) projector whose Digital Micromirror Device (DMD) imaging chip is directly controlled by a computer, similar to the way random access memory is controlled. We show that a perceptually-continuous-tone dynamic gray-scale image can be efficiently composed from a very rapid succession of binary (partial) images, each calculated from the continuous-tone image generated with the most recent tracking data. As the DMD projects only a binary image at any moment, it cannot instantly display this latest continuous-tone image, and conventional decomposition of a continuous-tone image into binary time-division-multiplexed values would induce just the latency we seek to avoid. Instead, our approach maintains an estimate of the image the user currently perceives, and at every opportunity allowed by the control circuitry, sets each binary DMD pixel to the value that will reduce the difference between that user-perceived image and the newly generated image from the latest tracking data. The resulting displayed binary image is “neither here nor there,” but always approaches the moving target that is the constantly changing desired image, even when that image changes every 50μs. We compare our experimental results with imagery from a conventional DLP projector with similar internal speed, and demonstrate that AR overlays on a moving object are more effective with this kind of low-latency display device than with displays of similar speed that use a conventional video interface.

In this work, we explore the lossless compression of 32-bit floating-point buffers on graphics hardware. We first adapt a state-of-the-art 16-bit floating-point color and depth buffer compression scheme for operation on 32-bit data and propose two specific enhancements: dynamic bucket selection and a Fibonacci encoder. Next, we describe a unified codec for any type of floating-point buffer: color, depth, geometry, and GPGPU data. We also propose a method to further compress variable-precision data. Finally, we test our techniques on color, depth, and geometry buffers from existing applications. Using our enhancements to an existing technique, we have improved bandwidth savings by an average of 1.26x. Our unified codec achieved average bandwidth savings of 1.5x, 7.9x, and 2.9x for color (including buffers incompressible by past work), depth, and geometry buffers. Even higher savings were achieved when combined with our variable-precision technique, though specific ratios will depend on the tolerance of the application to reducing its precision.

We present an energy model for a graphics processing unit (GPU) that is based on the amount and type of work performed in various parts of the unit. By designing and running directed tests on a GPU, we measure the energy consumed when performing different arithmetic and memory operations, allowing us to accurately predict the energy that any arbitrary mix of operations will take. With some knowledge of how data travels through and is transformed by the graphics pipeline, we can predict how many of each operation will occur for a given scene, leading to an estimate of the energy usage. We validate our model against different types of existing graphical applications. With an average difference of 3% from measured energy under typical workloads, our model can be used for various purposes. In this work, we explore and present two use cases: 1) predicting the energy performance of applications on a different architecture, and 2) exploring the energy efficiency of different algorithms to achieve the same graphical effect.

In mobile devices, limiting the Graphics Processing Unitpsilas (GPUpsilas) energy usage is of great importance to extending battery life. This paper focuses on the first stage of the graphics processor pipeline - the vertex transformation stage - and introduces an approach to lowering its switching activity by reducing the precision of arithmetic operations. As a result, the approach enables a tradeoff between energy efficiency and the quality of the rendered image. This paper makes the following specific contributions: 1) a transition-based energy model for quantifying energy consumed as a function of arithmetic precision, and 2) detailed simulation results on several real-world graphics applications to evaluate the tradeoff between energy and precision. In most examples, over 23% of the energy can be saved by lowering arithmetic precision while still maintaining a faithful reproduction of the full-precision image. Pushing the idea further, over 36% energy can be saved by further lowering the precision while preserving acceptable result accuracy. We assert that this represents a significant energy savings that warrants further investigation and extension of our approach to the remaining stages of the graphics processor pipeline.

Photon mapping places an enormous burden on the memory hierarchy. Rendering a 512 x 512 image of a simple scene can require more than 196 Gbytes of raw bandwidth to the photon map data structure. This bandwidth is a major obstacle to real time photon mapping. This paper investigates two approaches for reducing the required bandwidth: 1) reordering the kNN searches and 2) cache conscious data structures. Using a Hilbert curve reordering, we demonstrate an experimental lower bound of 15 Mbytes of bandwidth for the same scene. Unfortunately, this improvement of four orders of magnitude requires a prohibitive amount of intermediate storage. We introduce two novel cost-effective algorithms that reduce the bandwidth by one order of magnitude. Scenes of different complexities are shown to exhibit similar reductions in bandwidth. We explain why the choice of data structure does not achieve similar reductions. We also examine the interaction of query reordering with two photon map acceleration techniques, importance sampling and the irradiance cache. Query reordering exploits the additional coherence that arises from the use of importance sampling in scenes with glossy surfaces. Irradiance caching also benefits from query reordering, even when complex surface geometry reduces the effectiveness of the irradiance cache.

We introduce a technique to rapidly generate summed-area tables using graphics hardware. Summed area tables, originally introduced by Crow, provide a way to filter arbitrarily large rectangular regions of an image in a constant amount of time. Our algorithm for generating summed-area tables, similar to a technique used in scientific computing called recursive doubling, allows the generation of a summed-area table in O(log n) time. We also describe a technique to mitigate the precision requirements of summed-area tables. The ability to calculate and use summed-area tables at interactive rates enables numerous interesting rendering effects. We present several possible applications. First, the use of summed-area tables allows real-time rendering of interactive, glossy environmental reflections. Second, we present glossy planar reflections with varying blurriness dependent on a reflected object’s distance to the reflector. Third, we show a technique that uses a summed-area table to render glossy transparent objects. The final application demonstrates an interactive depth-of-field effect using summedarea tables. Categories and Subject Descriptors (according to ACM CCS): I.3.7 [Computer Graphics]: Three-Dimensional

The recent explosion in the number of handheld multimedia devices has created a need for energy-efficient computation due to limited battery lifetimes. We focus on multiplication, which is needed in several application domains, e.g., 3D graphics, signal processing, and cryptography. We introduce an asynchronous implementation of a plain Booth multiplier (i.e., radix-2), which is both area- and energy-efficient, and therefore suitable for mobile applications. This paper makes the following contributions. First, a novel counterflow organization is introduced, in which the data bits flow in one direction, and the Booth commands piggyback on the acknowledgments flowing in the opposite direction. Second, the arithmetic and shifter units are merged together to obtain significant improvement in area, energy as well as speed. Third, our design performs overlapped execution of multiple iterations of the Booth algorithm. Finally, the design is quite modular, which allows scaling to arbitrary operand widths, without gate resizing or cycle time overheads. Spice simulations in a 0.18 /spl mu/m TSMC process at 1.8 V, indicate promising performance: the multiplier takes 1.08 ns per Booth iteration, regardless of the operand widths, thereby demonstrating the scalability of our approach. In addition, the multiplier is fully functional at reduced supply voltages (e.g., 1.0 V), and thus capable of dynamically trading off performance for energy efficiency.

Abstract: This sketch reports the development of an acquisition methodology to acquire high-resolution surface light fields of an office-size environment. 1

We introduce and present preliminary results for a hybrid display system combining head-mounted and projector-based displays. Our work is motivated by a surgical training application, where it is necessary to simultaneously provide both a high-fidelity view of a central close-up task (the surgery) and visual awareness of objects and events in the surrounding environment In particular, for trauma surgeons it would be valuable to learn to work in an environment that is realistically filled with both necessary and distracting objects and events. In this paper, we motivate the use of a hybrid display system, discuss previous work, describe a prototype along with methods for geometric calibration, and present results from a controlled human subject experiment.

Article The vacuum buffer Share on Authors: Voicu Popescu Univ. of North Carolina, Chapel Hill Univ. of North Carolina, Chapel HillView Profile , Anselmo Lastra Univ. of North Carolina, Chapel Hill Univ. of North Carolina, Chapel HillView Profile Authors Info & Claims I3D '01: Proceedings of the 2001 symposium on Interactive 3D graphicsMarch 2001 Pages 73–76https://doi.org/10.1145/364338.364367Online:01 March 2001Publication History 5citation285DownloadsMetricsTotal Citations5Total Downloads285Last 12 Months1Last 6 weeks1 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

We introduce an idea and some preliminary results for a new projector-based approach to re-creating real and imagined sites. Our goal is to achieve re-creations that are both visually and spatially realistic, providing a small number of relatively unencumbered users with a strong sense of immersion as they jointly walkaround the virtual site.Rather than using head-mounted or general-purpose projector-based displays, our idea builds on previous projector-based work on spatially-augmented realityand shader lamps. Using simple white building blocks we construct a static physical model that approximates the size, shape, and spatial arrangementof the site. We then project dynamic imagery onto the blocks, transforming the lifeless physical model into a visually faithful reproduction of the actual site. Some advantages of this approach include wide field-of-view imagery, real walking around the site, reduced sensitivity to tracking errors, reduced sensitivity to system latency, auto-stereoscopic vision, the natural addition of augmented virtualityand the provision of haptics.In addition to describing the major challenges to (and limitations of) this vision, in this paper we describe some short-term solutions and practical methods, and we present some proof-of-concept results.

This paper presents a method for realistic real-time rendering of clouds suitable for flight simulation and games. It provides a cloud shading algorithm that approximates multiple forward scattering in a preprocess, and first order anisotropic scattering at runtime. Impostors are used to accelerate cloud rendering by exploiting frame-to-frame coherence in an interactive flight simulation. Impostors are shown to be particularly well suited to clouds, even in circumstances under which they cannot be applied to the rendering of polygonal geometry. The method allows hundreds of clouds and hundreds of thousands of particles to be rendered at high frame rates, and improves interaction with clouds by reducing artifacts introduced by direct particle rendering techniques.

We present the WarpEngine, an architecture designed for real-time imaged-based rendering of natural scenes from arbitrary viewpoints. The modeling primitives are real-world images with per-pixel depth. Currently they are acquired and stored off-line; in the near future real-time depth-image acquisition will be possible, the WarpEngine is designed to render in immediate mode from such data sources.The depth-image resolution is locally adapted by interpolation to match the resolution of the output image. 3D warping can occur either before or after the interpolation; the resulting warped/interpolated samples are forward-mapped into a warp buffer, with the precise locations recorded using an offset. Warping processors are integrated on-chip with the warp buffer, allowing efficient, scalable implementation of very high performance systems. Each chip will be able to process 100 million samples per second and provide 4.8GigaBytes per second of bandwidth to the warp buffer.The WarpEngine is significantly less complex than our previous efforts, incorporating only a single ASIC design. Small configurations can be packaged as a PC add-in card, while larger deskside configurations will provide HDTV resolutions at 50 Hz, enabling radical new applications such as 3D television.WarpEngine will be highly programmable, facilitating use as a test-bed for experimental IBR algorithms.

This paper presents a new method for interactive rendering of globally illuminated static scenes. Global illumination is decomposed into view-independent (diffuse) and view-dependent (non-diffuse) components. The two are recombined during rendering using a hybrid geometry- and image-based approach along with multi-pass blending techniques. This approach allows the preprocessing of both components and the fast rendering of globally illuminated scenes. The view-independent component uses a traditional precomputed geometry-based radiosity solution that is rendered using standard graphics hardware. The view-dependent component is decomposed into “what is reflected ” (radiance with depth) and “how it is reflected ” (BRDF), and precomputed and rendered using image-based approaches. Radiance is stored as images with depth, and rendered using perspective reprojection; the BRDF is decomposed into an integration of incoming radiance and a directional modulation. The radiance integration term is approximated by convolving the reflected image with precomputed kernel textures based on material properties. The directional modulation is stored as a reflectance modulation texture based on material properties and is rendered using spheremapping during a blending pass.

We present a preprocessing algorithm and run-time system for rendering 3D geometric models at a guaranteed frame rate.Our approach trades off space for frame rate by using images to replace distant geometry.The preprocessing algorithm automatically chooses a subset of the model to display as an image so as to render no more than a specified number of geometric primitives.We also summarize an optimized layered-depth-image warper to display images surrounded by geometry at run time.Furthermore, we show the results of applying our method to accelerate the interactive walkthrough of several complex models.

Article LDI tree: a hierarchical representation for image-based rendering Share on Authors: Chun-Fa Chang University of North Carolina at Chapel Hill University of North Carolina at Chapel HillView Profile , Gary Bishop University of North Carolina at Chapel Hill University of North Carolina at Chapel HillView Profile , Anselmo Lastra University of North Carolina at Chapel Hill University of North Carolina at Chapel HillView Profile Authors Info & Claims SIGGRAPH '99: Proceedings of the 26th annual conference on Computer graphics and interactive techniquesJuly 1999 Pages 291–298https://doi.org/10.1145/311535.311571Online:01 July 1999Publication History 95citation1,161DownloadsMetricsTotal Citations95Total Downloads1,161Last 12 Months22Last 6 weeks2 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

Over the years, there have been two main branches of computer graphics image-synthesis research; one focused on interactivity, the other on image quality.Procedural shading is a powerful tool, commonly used for creating high-quality images and production animation.A key aspect of most procedural shading is the use of a shading language, which allows a high-level description of the color and shading of each surface.However, shading languages have been beyond the capabilities of the interactive graphics hardware community.We have created a parallel graphics multicomputer, PixelFlow, that can render images at 30 frames per second using a shading language.This is the first system to be able to support a shading language in real-time.In this paper, we describe some of the techniques that make this possible.

This paper presents efficient image-based rendering techniques used in the context of an architectural walkthrough system. Portals (doors and windows) are rendered by warping layered depth images (LDIs). In a preprocessing phase, for every portal, a number of pre-rendered images are combined into an LDI. The resulting LDI stores, exactly once, all surfaces visible in at least one of the images used in the construction, so most of the exposure errors are efficiently eliminated. The LDI can be warped in the McMillan occlusion compatible ordering. A substantial increase in performance is obtained by warping in parallel. Our parallelization scheme achieves good load balancing, scales with the number of processors, and preserves the occlusion compatible ordering. A fast, conservative reference-image-space clipping algorithm also reduces the warping effort.

This paper outlines a method to dynamically replace portals with textures in a cell-partitioned model. The rendering complexity is reduced to the geometry of the current cell thus increasing interactive performance. A portal is a generalization of windows and doors. It connects two adjacent cells (or rooms). Each portal of the current cell that is some distance away from the viewpoint is rendered as a texture. The portal texture (smoothly) returns to geometry when the viewpoint gets close to the portal. This way all portal sequences (not too close to the viewpoint) have a depth complexity of one. The size of each texture and distance at which the transition occurs is configurable for each portal.

Abstract We present a methodfor simplifying the meshes produced as solutions to global illumination problems, reducing geometric complexity while retaining the perceived imagefidelity. The method has been applied to produce meshes of linearly, quadratically and cubically colour‐interpolated triangles. The goal of our work is to permit interactive rendering of more complex global illumination solutions through the application of simplification algorithms as well as the use of more powerful rendering primitives.

We present algorithms for interactive rendering of large-scale NURBS models. The algorithms convert the NURBS surfaces to Bezier surfaces, tessellate each Bezier surface into triangles, and render them using the triangle-rendering capabilities common to current graphics systems. We present algorithms for computing tight bounds on surface properties in order to generate high quality tessellation of Bezier surfaces. We introduce enhanced visibility determination techniques and present methods to make efficient use of coherence between successive frames. In addition, we also discuss issues in parallelization of these techniques. The algorithm also avoids polygonization anomalies like cracks. Our algorithms work well in practice and, on high-end graphics systems, are able to display models described using thousands of Bezier surfaces at interactive frame rates.

We present serial and parallel algorithms for interactive rendering of large scale and complex NURBS models on current graphics systems. The algorithms tessellate the NURBS surfaces into triangles and render them using triangle rendering engines. The main characteristics of the algorithms are improved polygonization algorithms, exploitation of spatial and temporal coherence and back-patch culling. Polygonization anomalies like cracks and angularities are avoided. We analyze a number of issues in parallelization of these techniques, as well. The algorithms work well in practice and are able to display models consisting of thousands of surfaces at interactive frame rates on the highly parallel graphics system, Pixel-Planes 5.

One of the main techniques used by software renderers to produce stunningly realistic images is programmable shading—executing an arbitrarily complex program to compute the color at each pixel. Thus far, programmable shading has only been available on software rendering systems that run on general-purpose computers. Rendering each image can take from minutes to hours.