Holographic displays generate Three-Dimensional (3D) images by displaying single-color holograms time-sequentially, each lit by a single-color light source. However, representing each color one by one limits brightness in holographic displays. This paper introduces a new driving scheme for realizing brighter images in holographic displays. Unlike the conventional driving scheme, our method utilizes three light sources to illuminate each displayed hologram simultaneously at various intensity levels. In this way, our method reconstructs a multiplanar three-dimensional target scene using consecutive multi-color holograms and persistence of vision. We co-optimize multi-color holograms and required intensity levels from each light source using a gradient descent-based optimizer with a combination of application-specific loss terms. We experimentally demonstrate that our method can increase the intensity levels in holographic displays up to three times, reaching a broader range and unlocking new potentials for perceptual realism in holographic displays.

This paper introduces a new multiplane CGH computation method to reconstruct artifact-free high-quality holograms with natural-looking defocus blur. Our method introduces a new targeting scheme and a new loss function. While the targeting scheme accounts for defocused parts of the scene at each depth plane, the new loss function analyzes focused and defocused parts separately in reconstructed images. Our method support phase-only CGH calculations using various iterative (e.g., Gerchberg-Saxton, Gradient Descent) and non-iterative (e.g., Double Phase) CGH techniques. We achieve our best image quality using a modified gradient descent-based optimization recipe where we introduce a constraint inspired by the double phase method. We validate our method experimentally using our proof-of-concept holographic display, comparing various algorithms, including multi-depth scenes with sparse and dense contents.

Computer-Generated Holography (CGH) offers the potential for genuine, high-quality three-dimensional visuals. However, fulfilling this potential remains a practical challenge due to computational complexity and visual quality issues. We propose a new CGH method that exploits gaze-contingency and perceptual graphics to accelerate the development of practical holographic display systems. Firstly, our method infers the user’s focal depth and generates images only at their focus plane without using any moving parts. Second, the images displayed are metamers; in the user’s peripheral vision, they need only be statistically correct and blend with the fovea seamlessly. Unlike previous methods, our method prioritises and improves foveal visual quality without causing perceptually visible distortions at the periphery. To enable our method, we introduce a novel metameric loss function that robustly compares the statistics of two given images for a known gaze location. In parallel, we implement a model representing the relation between holograms and their image reconstructions. We couple our differentiable loss function and model to metameric varifocal holograms using a stochastic gradient descent solver. We evaluate our method with an actual proof-of-concept holographic display, and we show that our CGH method leads to practical and perceptually three-dimensional image reconstructions.

Head-mounted holographic displays (HMHD) are projected to be the first commercial realization of holographic video display systems. HMHDs use liquid crystal on silicon (LCoS) spatial light modulators (SLM), which are best suited to display phase-only holograms (POH). The performance/watt requirement of a monochrome, 60 fps Full HD, 2-eye, POH HMHD system is about 10 TFLOPS/W, which is orders of magnitude higher than that is achievable by commercially available mobile processors. To mitigate this compute power constraint, display-ready POHs shall be generated on a nearby server and sent to the HMHD in compressed form over a wireless link. This paper discusses design of a feasible HMHD-based augmented reality system, focusing on compression requirements and per-pixel rate-distortion trade-off for transmission of display-ready POH from the server to HMHD. Since the decoder in the HMHD needs to operate on low power, only coding methods that have low-power decoder implementation are considered. Effects of 2D phase unwrapping and flat quantization on compression performance are also reported. We next propose a versatile PCM-POH codec with progressive quantization that can adapt to SLM-dynamic-range and available bitrate, and features per-pixel rate-distortion control to achieve acceptable POH quality at target rates of 60-200 Mbit/s that can be reliably achieved by current wireless technologies. Our results demonstrate feasibility of realizing a low-power, quality-ensured, multi-user, interactive HMHD augmented reality system with commercially available components using the proposed adaptive compression of display-ready POH with light-weight decoding.

Recent developmental studies state that nonsymbolic number representation (i.e., more-less comparisons) is important for math development, and children's judgment about such non-numerical magnitudes can be affected by sensory properties (i.e., volume, space). Yet, to our knowledge, there are no tangible based systems for training this math concept. Building on theories of cognition and learning, we developed MaR-T, a projector-camera setup. This paper is a step towards investigating the effects of projection-based mixed-reality (MR) system with tangibles on nonsymbolic number representation of 3- to 5-year-old children. We present our user studies with a total of 14 participants, conducted to observe their interaction with the setup and the possible effects of our design on learning. The results indicate that MaR-T can provide active, engaging, and social learning, and our insights can inspire other interaction design and education studies.

A child's early math development can stem from interactions with the physical world. Accordingly, current tangible interaction studies focus on preschool children's formal (symbolic) mathematics, i.e. number knowledge. However, recent developmental studies stress the importance of nonsymbolic number representation in math learning, i.e. understanding quantity relations without counting(more/less). To our knowledge, there are no tangible systems based on this math concept. We developed an initial tangible based mixed-reality(MR) setup with a small tabletop projector and depth camera. Our goal was observing children's interaction with the setup to guide our further design process towards developing nonsymbolic math training. In this paper, we present our observations from sessions with four 3-to-5 year old children and discuss their meaning for future work. Initial clues show that our MR setup leads to exploratory and mindful interactions, which might be generalizable to other tangible MR systems for child education and could inspire interaction design studies.

Head-mounted displays and augmented reality headsets are emerging as the future of human-computer interaction. Such devices can display high resolution 3D images and use on-board cameras to capture the surroundings of the user. However, capturing the user who is wearing the device to facilitate 3D telepresence is not possible with such headsets. Here we propose and demonstrate a new integrated platform to provide mobile 3D telepresence experience using a head-worn device and a dual-purpose passive screen. At the core of this telepresence architecture, we use a portable multi-layered passive screen which facilitates the stereoscopic 3D display using a pair of head-worn projectors and at the same time, captures the multi-perspective views of the user on a head-worn camera through reflections of the screen. The screen contains retroreflective material for stereo image display and an array of convex mirrors for 3D capture. The 3D telepresence is demonstrated using an experimental setup where a local-user wearing the developed head-worn device perceives the 3D images on the dual-purpose screen, while the captured perspective views of user-1 are rendered as stereo viewpoints and showed to the user-2 on a virtual reality headset.

The Head Mounted Projection Display (HMPD) is a growing interest area in HCI. Although various aspects of HMPDs have been investigated, there is not enough information regarding the effect of HMPDs (i.e., head referenced static and dynamic displays while a user is in motion and standing) on visual attentional performance. For this purpose, we conducted a user study (N=18) with three experimental conditions (control, standing, walking) and two visual perceptual tasks (with dynamic and static displays). Significant differences between conditions were only found for the task with dynamic display; accuracy was lower in walking condition compared to the other two conditions. Our work contributes an empirical investigation of the effect of HMPDs on visual attentional performance by providing data-driven benchmarks for developing graphical user interface design guidelines for HMPDs.

Motion capture shoots involve a wide range of technology and entertainment production systems such as motion capture cameras, tracking software and digital environments to create entertainment applications. However, acting in this high-tech environment is still traditional and brings its own challenges to the actors. Good acting and imagination skills are highly needed for many motion capture shoots to deliver satisfying results.

The main goal of this research is to develop a mixed reality (MR) application to support motion capture actors. This application allows seeing and exploring a digital environment without occluding the actor's visual field. A prototype is built by combining a retro-reflective screen covering surrounding walls and a headband consisting of a laser scanning projector with a smartphone. Built-in sensors of a smartphone provide navigation capabilities in the digital world. The integrated system has some unique advantages, which are collectively demonstrated for the first time: (i) providing fixed field-of-view (50° in diagonal), fixed retinal images at full-resolution, and distortion-free images that are independent of the screen distance and shape; (ii) presenting different perspectives to the users as they move around or tilt their heads, (iii) allowing a focus-free and calibration-free display even on non-flat surfaces using laser scanning technology, (iv) enabling multiple users to share the same screen without crosstalk due to the use of retro-reflectors, and (v) producing high brightness pictures with a projector of only 15 lm; due to a high-gain retro-reflective screen. We demonstrated a lightweight, comfortable to wear and low cost head-mounted projection display (HMPD) which acts as a stand-a-lone mobile system. Initial informal functionality tests have been successfully performed. The prototype can also be used as a 3D stereo system using the same hardware by additionally mounting polarized glasses and an active polarization rotator, while maintaining all of the advantages listed above.

Underlying principles of stereoscopic direct-view displays, binocular head-mounted displays, and autostereoscopic direct-view displays are explained and some early work as well as the state of the art in those technologies are reviewed. Stereoscopic displays require eyewear and can be categorized based on the multiplexing scheme as: 1) color multiplexed (old technology but there are some recent developments; low-quality due to color reproduction and crosstalk issues; simple and does not require additional electronics hardware); 2) polarization multiplexed (requires polarized light output and polarization-based passive eyewear; high-resolution and high-quality displays available); and 3) time multiplexed (requires faster display hardware and active glasses synchronized with the display; high-resolution commercial products available). Binocular head-mounted displays can readily provide 3-D, virtual images, immersive experience, and more possibilities for interactive displays. However, the bulk of the optics, matching of the left and right ocular images and obtaining a large field of view make the designs quite challenging. Some of the recent developments using unconventional optical relays allow for thin form factors and open up new possibilities. Autostereoscopic displays are very attractive as they do not require any eyewear. There are many possibilities in this category including: two-view (the simplest implementations are with a parallax barrier or a lenticular screen), multiview, head tracked (requires active optics to redirect the rays to a moving viewer), and super multiview (potentially can solve the accommodation–convergence mismatch problem). Earlier 3-D booms did not last long mainly due to the unavailability of enabling technologies and the content. Current developments in the hardware technologies provide a renewed interest in 3-D displays both from the consumers and the display manufacturers, which is evidenced by the recent commercial products and new research results in this field.

The display is the last component in a chain of activity from image acquisition, compression, coding transmission and reproduction of 3-D images through to the display itself. There are various schemes for 3-D display taxonomy; the basic categories adopted for this paper are: holography where the image is produced by wavefront reconstruction, volumetric where the image is produced within a volume of space and multiple image displays where two or more images are seen across the viewing field. In an ideal world a stereoscopic display would produce images in real time that exhibit all the characteristics of the original scene. This would require the wavefront to be reproduced accurately, but currently this can only be achieved using holographic techniques. Volumetric displays provide both vertical and horizontal parallax so that several viewers can see 3-D images that exhibit no accommodation/convergence rivalry. Multiple image displays fall within three fundamental types: holoform in which a large number of views give smooth motion parallax and hence a hologram-like appearance, multiview where a series of discrete views are presented across viewing field and binocular where only two views are presented in regions that may occupy fixed positions or follow viewers' eye positions by employing head tracking. Holography enables 3-D scenes to be encoded into an interference pattern, however, this places constraints on the display resolution necessary to reconstruct a scene. Although holography may ultimately offer the solution for 3DTV, the problem of capturing naturally lit scenes will first have to be solved and holography is unlikely to provide a short-term solution due to limitations in current enabling technologies. Liquid crystal, digital micromirror, optically addressed liquid crystal and acoustooptic spatial light modulators (SLMs) have been employed as suitable spatial light modulation devices in holography. Liquid crystal SLMs are generally favored owing to the commercial availability of high fill factor, high resolution addressable devices. Volumetric displays provide both vertical and horizontal parallax and several viewers are able to see a 3-D image that exhibits no accommodation/convergence rivalry. However, the principal disadvantages of these displays are: the images are generally transparent, the hardware tends to be complex and non-Lambertian intensity distribution cannot be displayed. Multiple image displays take many forms and it is likely that one or more of these will provide the solution(s) for the first generation of 3DTV displays.

An optoelectronic processor that should allow the implementation of multiresolution Gabor filtering at TV frame rates is described. The system takes the form of a TV camera with modified optics and some simple processing incorporated in the focal-plane detector array. The output of the camera is an array of images, each produced by putting the input image through an angularly-oriented Gabor-type spatial frequency filter and converting the output to baseband. Key elements of the system include: (a) bandpass spatial filtering by pupil modification of an incoherent imaging system, (b) separation of spatial lowpass and spatial bandpass structures by temporal modulation of the bandpass distribution, (c) detection of the temporally modulated distribution by VLSI circuitry in the focal-plane detector array, and (d) multiplexing of multiple spatial bandpass channels in a single smart pixel array.