Spatial perception in virtual environments has been a topic of intense research. Arguably, the majority of this work has focused on distance perception. However, orientation perception is also an important factor. In this paper, we systematically investigate allocentric orientation judgments in both real and virtual contexts over the course of four experiments. A pattern of sinusoidal judgment errors known to exist in 2D perspective displays is found to persist in immersive virtual environments. This pattern also manifests itself in a real world setting using two differing judgment methods. The findings suggest the presence of a radial anisotropy that persists across viewing contexts. Additionally, there is some evidence to suggest that observers have multiple strategies for processing orientations but further investigation is needed to fully describe this phenomenon. We also offer design suggestions for 3D user interfaces where users may perform orientation judgments.

The 2018 Virtual Reality Career Award goes to Mark Bolas in recognition of his lifetime contributions to the design and commercialization of virtual reality displays, systems, and applications. Professor Bolas has been working in Virtual and Augmented Reality for 30 years. He co-founded Fakespace Labs Inc. with Ian McDowall and Eric Lorimer, a company that created such trailblazing VR systems as the BOOM (Binocular Omni-Orientation Monitor), the Pinch glove, the RAVE, the PUSH, and VLIB software. His teams at the USC School of Cinematic Arts and the Institute for Creative Technologies championed low-cost designs that were open-sourced and helped fuel the resurgence in consumer VR as evidenced by the Rift, Vive, and Cardboard. He was awarded the 2005 IEEE VGTC Virtual Reality Technical Achievement Award. The IEEE VGTC is pleased to award Mark Bolas the 2018 Virtual Reality Career Award.

High-fidelity virtual content is essential for the creation of compelling and effective virtual reality (VR) experiences. However, creating photorealistic content is not easy, and handcrafting detailed 3D models can be time and labor intensive. Structured camera arrays, such as light-stages, can scan and reconstruct high-fidelity virtual models, but the expense makes this technology impractical for most users. In this paper, we present a complete end-to-end pipeline for the capture, processing, and rendering of view-dependent 3D models in virtual reality from a single consumer-grade depth camera. The geometry model and the camera trajectories are automatically reconstructed and optimized from a RGB-D image sequence captured offline. Based on the head-mounted display (HMD) position, the three closest images are selected for real-time rendering and fused together to smooth the transition between viewpoints. The specular reflections and light-burst effects can also be preserved and reproduced. We confirmed that our method does not require technical background knowledge by testing our system with data captured by non-expert operators.

Visualization is a critical component of neuroimaging, and how to best view data that is naturally three dimensional is a long standing question in neuroscience. Many approaches, programs, and techniques have been developed specifically for neuroimaging. However, exploration of 3D information through a 2D screen is inherently limited. Many neuroscientific researchers hope that with the recent commercialization and popularization of VR, it can offer the next-step in data visualization and exploration. Neuro Imaging in Virtual Reality (NIVR), is a visualization suite that employs various immersive visualizations to represent neuroimaging information in VR. Some established techniques, such as raymarching volume visualization, are paired with newer techniques, such as near-field rendering, to provide a broad basis of how we can leverage VR to improve visualization and navigation of neuroimaging data. Several of the neuroscientific visualization approaches presented are, to our knowledge, the first of their kind. NIVR offers not only an exploration of neuroscientific data visualization, but also a tool to expose and educate the public regarding recent advancements in the field of neuroimaging. By providing an engaging experience to explore new techniques and discoveries in neuroimaging, we hope to spark scientific interest through a broad audience. Furthermore, neuroimaging offers deep and expansive datasets; a single scan can involve several gigabytes of information. Visualization and exploration of this type of information can be challenging, and real-time exploration of this information in VR even more so. NIVR explores pathways which make this possible, and offers preliminary stereo visualizations of these types of massive data.

Virtual objects are essential for building environments in virtual reality (VR) applications. However, creating photorealistic 3D models is not easy, and handcrafting the detailed 3D model from a real object can be time and labor intensive. An alternative way is to build a structured camera array such as a light-stage to reconstruct the model from a real object. However, these technologies are very expensive and not practical for most users. In this work, we demonstrate a complete end-to-end pipeline for the capture, processing, and rendering of view-dependent 3D models in virtual reality from a single consumer-grade RGB-D camera. The geometry model and the camera trajectories are automatically reconstructed from a RGB-D image sequence captured offline. Based on the HMD position, selected images are used for real-time model rendering. The result of this pipeline is a 3D mesh with view-dependent textures suitable for real-time rendering in virtual reality. Specular reflections and light-burst effects are especially noticeable when users view the objects from different perspectives in a head-tracked environment.

In this article, we detail a series of experiments that examines the effect of vertical field-of-view extension and the addition of non-specific peripheral visual stimulation on gait characteristics and distance judgments in a head-worn virtual environment. Specifically, we examined four field-of-view configurations: a common 60° diagonal field of view (48° × 40°), a 60° diagonal field of view with the addition of a luminous white frame in the far periphery, a field of view with an extended upper edge, and a field of view with an extended lower edge. We found that extension of the field of view, either with spatially congruent or spatially non-informative visuals, resulted in improved distance judgments and changes in observed posture. However, these effects were not equal across all field-of-view configurations, suggesting that some configurations may be more appropriate than others when balancing performance, cost, and ergonomics.

Redirected walking enables the exploration of large virtual environments while requiring only a finite amount of physical space. Unfortunately, in living room sized tracked areas the effectiveness of common redirection algorithms such as Steer-to-Center is very limited. A potential solution is to increase redirection effectiveness by applying two types of perceptual manipulations (curvature and translation gains) simultaneously. This paper investigates how such combination may affect detection thresholds for curvature gain. To this end we analyze the estimation methodology and discuss selection process for a suitable estimation method. We then compare curvature detection thresholds obtained under different levels of translation gain using two different estimation methods: method of constant stimuli and Green's maximum likelihood procedure. The data from both experiments shows no evidence that curvature gain detection thresholds were affected by the presence of translation gain (with test levels spanning previously estimated interval of undetectable translation gain levels). This suggests that in practice currently used levels of translation and curvature gains can be safely applied simultaneously. Furthermore, we present some evidence that curvature detection thresholds may be lower that previously reported. Our estimates indicate that users can be redirected on a circular arc with radius of either 11.6m or 6.4m depending on the estimation method vs. the previously reported value of 22m. These results highlight that the detection threshold estimates vary significantly with the estimation method and suggest the need for further studies to define efficient and reliable estimation methodology.

Projected displays can present life-sized imagery of a virtual human character that can be seen by multiple observers. However, typical projected displays can only render that virtual human from a single viewpoint, regardless of whether head tracking is employed. This results in the virtual human being rendered from an incorrect perspective for most individuals. This could cause perceptual miscues, such as the “Mona Lisa” effect, causing the virtual human to appear as if it is simultaneously gazing and pointing at all observers regardless of their location. This may be detrimental to training scenarios in which all trainees must accurately assess where the virtual human is looking or pointing. We discuss our investigations into the presentation of eye gaze using REFLCT, a previously introduced head mounted projective display. REFLCT uses head tracked, head mounted projectors and retroreflective screens to present personalized, perspective correct imagery to multiple users without the occlusion of a traditional head mounted display. We examined how head mounted projection for enhanced presentation of eye gaze might facilitate or otherwise affect social interactions during a multi-person guessing game of “Twenty Questions”.

Redirection techniques allow users to explore large virtual environments on foot while remaining within a limited physical space. However, research has primarily focused on redirection through manipulation of the visuals used to represent the virtual environment. We describe a within-subjects study (n=31) exploring if participants' ability to detect differences between real and virtual rotations is influenced by the addition of sound that is spatially aligned with its virtual source. The results revealed similar detection thresholds for conditions involving moving audio, static audio, and no audio. This may be viewed as an indication of visual dominance during scenarios such as the one used for the current study.

We believe that virtual humans presented over video chat services, such as Skype via smartphones, can be an effective way to deliver innovative applications where social interactions are important, such as counseling and coaching. We hypothesize that the context of a smartphone communication channel, i.e. how a virtual human is presented within a smartphone app, and indeed, the nature of that app, can profoundly affect how a real human perceives the virtual human. We have built an apparatus that allows virtual humans to initiate, receive, and interact over video calls using Skype or any similar service. With this platform, we are examining effective designs and social implications of virtual humans that interact over mobile video. The current study examines a relationship involving repeated counseling-style interactions with a virtual human, leveraging the virtual human's ability to call and interact with a real human on multiple occasions over a period of time. The results and implications of this preliminary study suggest that repeated interactions may improve perceived social characteristics of the virtual human.

Mixed Reality can be a valuable tool for research and development in robotics. In this work, we refine the definition of Mixed Reality to accommodate seamless interaction between physical and virtual objects in any number of physical or virtual environments. In particular, we show that Mixed Reality can reduce the gap between simulation and implementation by enabling the prototyping of algorithms on a combination of physical and virtual objects, including robots, sensors, and humans. Robots can be enhanced with additional virtual capabilities, or can interact with humans without sharing physical space. We demonstrate Mixed Reality with three representative experiments, each of which highlights the advantages of our approach. We also provide a testbed for Mixed Reality with three different virtual robotics environments in combination with the Crazyflie 2.0 quadcopter.

Redirected walking techniques have been introduced to overcome physical limitations for natural locomotion in virtual reality. Although subtle perceptual manipulations are helpful to keep users within relatively small tracked spaces, it is inevitable that users will approach critical boundary limits. Current solutions to this problem involve breaks in presence by introducing distractors, or freezing the virtual world relative to the user's perspective. We propose an approach that integrates into the virtual world narrative to draw users' attention and to cause them to temporarily alter their course to avoid going off bounds. This method ties together unnoticeable translation, rotation, and curvature gains, efficiently reorienting the user while maintaining the user's sense of immersion. We also discuss how this new method can be effectively used in conjunction with other reorientation techniques.

Redirected Walking is technique that leverages human perception characteristics to allow locomotion in virtual environments larger than the tracking area. Among the many redirection techniques, some strictly depend on the user's current position and orientation, while more recent algorithms also depend on the user's predicted behavior. This prediction serves as an input to a computationally expensive search to determine an optimal path. The search output is formulated as a series of gains to be applied at different stages along the path. An example prediction could be if a user is walking down a corridor, a natural prediction would be that the user will walk along a straight line down the corridor, and she will choose one of the possible directions with equal probability. In practice, deviations from the expected virtual path are inevitable, and as a result, the real world path traversed will differ from the original prediction. These deviations can not only force the search to select a less optimal path in the next iteration, but also in cases cause the users to go off bounds, requiring resets, causing a jarring experience for the user. We propose a method to account for these deviations by modifying the redirection gains per update frame, aiming to keep the user on the intended predicted physical path.

In the good old days, the human was here, the computer there, and a good living was to be made by designing ways to interface between the two. Now we find ourselves unthinkingly pinching to zoom in on a picture in a paper magazine. User interfaces are changing instinctual human behavior and instinctual human behavior is changing user interfaces. We point or look left in the "virtual" world just as we point or look left in the physical.

Redirected walking techniques enable natural locomotion through immersive virtual environments that are considerably larger than the available real world walking space. However, the most effective strategy for steering the user remains an open question, as most previously presented algorithms simply redirect toward the center of the physical space. In this work, we present a theoretical framework that plans a walking path through a virtual environment and calculates the parameters for combining translation, rotation, and curvature gains such that the user can traverse a series of defined waypoints efficiently based on a utility function. This function minimizes the number of overt reorientations to avoid introducing potential breaks in presence. A notable advantage of this approach is that it leverages knowledge of the layout of both the physical and virtual environments to enhance the steering strategy.

In this document we present a method for calibrating head-mounted displays and other display surfaces using an automated, low-cost camera system. A unique aspect of this method is that the calibration of geometric distortions, field of view, and chromatic aberration are achieved without the need for a priori knowledge of the display system's intrinsic parameters. This method operates by capturing and storing the pixel space locations of a series of real world control points. These control points are then used as ground truth references by which virtual space transformations can be automatically generated for a display system.

In this document we discuss a study that investigates the effect of eye position on the apparent location of imagery presented in an off-the-shelf head worn display. We test a range of reasonable eye positions that may result from person-to-person variations in display placement and interpupillary distances. It was observed that the pattern of geometric distortions introduced by the display's optical system changes substantially as the eye moves from one position to the next. These visual displacements can be on the order of several degrees and increase in magnitude towards the peripheral edges of the field of view. Though many systems calibrate for interpupillary distance and optical distortions separately, this may be insufficient as eye position influences distortion characteristics.

With the current widespread interest in head mounted displays, we perceived a need for devices that support expressive and adaptive interaction in a low-cost, eyes-free manner. Leveraging rapid prototyping techniques for fabrication, we have designed and manufactured a variety of panels that can be overlaid on multi-touch tablets and smartphones. The panels are coupled with an app running on the multi-touch device that exchanges commands and state information over a wireless network with the virtual reality application. Sculpted features of the panels provide tactile disambiguation of control widgets and an onscreen heads-up display provides interaction state information. A variety of interaction mappings can be provided through software to support several classes of interaction techniques in virtual environments. We foresee additional uses for applications where eyes-free use and adaptable interaction interfaces can be beneficial.

Our demo deals with the need in immersive virtual reality for devices that support expressive and adaptive interaction in a low-cost, eyes-free manner. Leveraging rapid prototyping techniques for fabrication, we have developed a variety of panels that can be overlaid on multi-touch tablets and smartphones. The panels are coupled with an app running on the multi-touch device that exchanges commands and state information over a wireless network with the virtual reality application. Sculpted features of the panels provide tactile disambiguation of control widgets and an onscreen heads-up display provides interaction state information. A variety of interaction mappings can be provided through software to support several classes of interaction techniques in virtual environments. We foresee additional uses for applications where eyes-free use and adaptable interaction interfaces can be beneficial.

The USC Institute for Creative Technologies will demonstrate a pipline for automatic reconstruction and animation of lifelike 3D avatars acquired by rotating the user's body in front of a single Microsoft Kinect sensor. Based on a fusion of state-of-the-art techniques in computer vision, graphics, and animation, this approach can produce a fully rigged character model suitable for real-time virtual environments in less than four minutes.

ABSTRACT We demonstrate a method of acquiring a 3D model of a human using commodity scanning hardware and then controlling that 3D figure in a simulated environment in only a few minutes. The model acquisition requires four static poses taken at 90° angles relative to each other. The 3D model is then given a skeleton and smooth binding information necessary for control and simulation. The 3D models that are captured are suitable for use in applications where recognition and distinction among characters by shape, form, or clothing is important, such as small group or crowd simulations or other socially oriented applications. Because of the speed at which a human figure can be captured and the low hardware requirements, this method can be used to capture, track, and model human figures as their appearances change over time. Copyright © 2014 John Wiley & Sons, Ltd.

The ICT Mixed Reality Lab is leveraging an open source philosophy to influence and disrupt industry. Projects spun out of the lab's efforts include the VR2GO smartphone based viewer, the inVerse tablet based viewer, the Socket HMD reference design, the Oculus Rift and the Project Holodeck gaming platforms, a repurposed FOV2GO design with Nokia Lumia phones for a 3D user interface course at Columbia University, and the EventLab's Socket based HMD at the University of Barcelona. A subset of these will be demonstrated. This open approach is providing low cost yet surprisingly compelling immersive experiences.

Summary form only given. The ICT Mixed Reality Lab will demonstrate a pipeline for rapidly generating personalized avatars from multiple depth and RGB scans of a user with a consumer level sensor such as a Microsoft Kinect. Based on a fusion of state-of-the-art techniques in graphics, surface reconstruction, and animation, our semi-automatic method can produce a fully rigged, skinned, and textured character model suitable for real-time virtual environments in less than 15 minutes. First, a 3D point cloud is collected from the sensor using a simultaneous localization and mapping (SLAM) approach to track the device's movements over time (see Figure 1.a). Next, surface reconstruction techniques are employed to generate a watertight 3D mesh from the raw 3D points (see Figure 1.b). The resulting model is then analyzed to determine the human joint locations, and if a skeleton can be successfully generated for the model, the mesh is then rigged and skinned using weights that are calculated automatically [2]. Finally, photos captured periodically during the scanning process using the sensor's RGB camera are used to texture the final model. The resulting avatar is suitable for real-time animation using a virtual environment or video game engine (see Figure 1.c). We will demonstrate our avatar generation pipeline at IEEE Virtual Reality 2013. Conference attendees may opt to be scanned, and their generated avatar will be provided to them either on a USB stick or through email. A video of this demo can be found at the MxR Lab website [1].

In this paper we introduce the concept of flexible spaces - a novel redirection technique that generalizes the use of overlapping (impossible) spaces and change blindness in an algorithm for dynamic layout generation. Flexible spaces is an impossible environment that violates the real world constancy in favor of providing the experience of seamless, unrestricted natural walking over a large-scale virtual environment (VE).

The following series of experiments explore the effect of static peripheral stimulation on the perception of distance and spatial scale in a typical head-mounted virtual environment. It was found that applying constant white light in an observer's far periphery enabled the observer to more accurately judge distances using blind walking. An effect of similar magnitude was also found when observers estimated the size of a virtual space using a visual scale task. The presence of the effect across multiple psychophysical tasks provided confidence that a perceptual change was, in fact, being invoked by the addition of the peripheral stimulation. These results were also compared to observer performance in a very large field of view virtual environment and in the real world. The subsequent findings raise the possibility that distance judgments in virtual environments might be considerably more similar to those in the real world than previous work has suggested.

As wider field-of-view displays become more common, the question arises as to whether or not data collected on these displays are comparable to those collected with smaller field-of-view displays. This document describes a pilot study that aimed to address these concerns by comparing medium-field distance judgments in a 60° FOV display, a 150° FOV display, and a simulated 60° FOV within the 150° FOV display. The results indicate that participants performed similarly in both the actual and simulated 60° FOV displays. On average, participants in the 150° FOV display improved distance judgments by 13% over the 60° FOV displays.

We describe two approaches for unobtrusively sensing subtle nonverbal behaviors using a consumer-level depth sensing camera. The first signal, respiratory rate, is estimated by measuring the visual expansion and contraction of the user's chest cavity during inhalation and exhalation. Additionally, we detect a specific type of fidgeting behavior, known as “leg jiggling,” by measuring high-frequency vertical oscillations of the user's knees. Both of these techniques rely on the combination of skeletal tracking information with raw depth readings from the sensor to identify the cyclical patterns in jittery, low-resolution data. Such subtle nonverbal signals may be useful for informing models of users' psychological states during communication with virtual human agents, thereby improving interactions that address important societal challenges in domains including education, training, and medicine.

Thin computing clients, such as smartphones and tablets, have exhibited recent growth in display resolutions, processing power, and graphical rendering speeds. In this poster, we show how we leveraged these trends to create virtual reality (VR) training games which run entirely on a commodity mobile computing platform. This platform consists of a commercial off-the-shelf game engine, commodity smartphones, and mass produced optics. The games utilize the strengths of this platform to provide immersive features like 360 degree photo panoramas and interactive 3D virtual scenes. By sharing information about building such applications, we hope to enable others to develop new types of mobile VR applications. In particular, we feel this system is ideally suited for casual “pick up and use” VR applications for collaborative classroom learning, design reviews, and other multi-user immersive experiences.

Our demonstration will exhibit a number of low cost virtual reality systems built using smartphones, inexpensive optics, and game engine software. These systems will demonstrate how VR researchers and developers can leverage current trends in commodity hardware and software to bring virtual reality to a wider audience. This demonstration dovetails with the Workshop on Off-the-Shelf Virtual Reality (OTSVR), which we will also be organizing at the VR conference. The Mixed Reality Lab at the USC Institute for Creative Technologies works to advance the design and practice of mixed reality and virtual reality. The lab is particularly focused on the research and development of new technologies and techniques to enhance immersion and interaction for learning and training.

Using video games in rehabilitation settings has the potential to provide patients with fun and motivating exercise tools. Within the Medical VR and MxR groups at the USC Institute for Creative Technologies, we have been leveraging the technology of the Microsoft Kinect 3D depth-sensing camera. Our Kinect-based rehabilitation game “JewelMine” consists of a set of static balance training exercises which encourage the players to reach out of their base of support. We plan to demonstrate a sophisticated post-session analysis tool and several content themes which can be changed dynamically during a therapy session.

Natural walking can provide a compelling experience in immersive virtual environments, but it remains an implementation challenge due to the physical space constraints imposed on the size of the virtual world. The use of redirection techniques is a promising approach that relaxes the space requirements of natural walking by manipulating the user's route in the virtual environment, causing the real world path to remain within the boundaries of the physical workspace. In this paper, we present and apply a novel taxonomy that separates redirection techniques according to their geometric flexibility versus the likelihood that they will be noticed by users. Additionally, we conducted a user study of three reorientation techniques, which confirmed that participants were less likely to experience a break in presence when reoriented using the techniques classified as subtle in our taxonomy. Our results also suggest that reorientation with change blindness illusions may give the impression of exploring a more expansive environment than continuous rotation techniques, but at the cost of negatively impacting spatial knowledge acquisition.

Walking is only possible within immersive virtual environments that fit inside the boundaries of the user's physical workspace. To reduce the severity of the restrictions imposed by limited physical area, we introduce "impossible spaces," a new design mechanic for virtual environments that wish to maximize the size of the virtual environment that can be explored with natural locomotion. Such environments make use of self-overlapping architectural layouts, effectively compressing comparatively large interior environments into smaller physical areas. We conducted two formal user studies to explore the perception and experience of impossible spaces. In the first experiment, we showed that reasonably small virtual rooms may overlap by as much as 56% before users begin to detect that they are in an impossible space, and that the larger virtual rooms that expanded to maximally fill our available 9.14 m x 9.14 m workspace may overlap by up to 31%. Our results also demonstrate that users perceive distances to objects in adjacent overlapping rooms as if the overall space was uncompressed, even at overlap levels that were overtly noticeable. In our second experiment, we combined several well-known redirection techniques to string together a chain of impossible spaces in an expansive outdoor scene. We then conducted an exploratory analysis of users' verbal feedback during exploration, which indicated that impossible spaces provide an even more powerful illusion when users are naive to the manipulation.

Mixed and augmented reality is crossing the chasm from research to widespread adoption. This requires us to think beyond registered planes of pixels in space, and confront the unregistered mess of users, culture, and application.

Thin computing clients, such as smartphones and tablets, have experienced recent growth in display resolutions and graphics processing power. In this poster, we show how to leverage these trends to create an experimental wide field of view, 3D stereoscopic head mounted display (HMD), based on two high resolution smart-phones. This HMD prototype is unique in that the graphics system is entirely onboard, allowing it to be lightweight, wireless, and convenient to use.

We present change blindness redirection, a novel technique for allowing the user to walk through an immersive virtual environment that is considerably larger than the available physical workspace. In contrast to previous redirection techniques, this approach, based on a dynamic environment model, does not introduce any visual-vestibular conflicts from manipulating the mapping between physical and virtual motions, nor does it require breaking presence to stop and explicitly reorient the user. We conducted two user studies to evaluate the effectiveness of the change blindness illusion when exploring a virtual environment that was an order of magnitude larger than the physical walking space. Despite the dynamically changing environment, participants were able to draw coherent sketch maps of the environment structure, and pointing task results indicated that they were able to maintain their spatial orientation within the virtual world. Only one out of 77 participants across both both studies definitively noticed that a scene change had occurred, suggesting that change blindness redirection provides a remarkably compelling illusion. Secondary findings revealed that a wide field-of-view increases pointing accuracy and that experienced gamers reported greater sense of presence than those with little or no experience with 3D video games.

The Flexible Action and Articulated Skeleton Toolkit (FAAST) is middleware to facilitate integration of full-body control with virtual reality applications and video games using OpenNI-compliant depth sensors (currently the PrimeSensor and the Microsoft Kinect). FAAST incorporates a VRPN server for streaming the user's skeleton joints over a network, which provides a convenient interface for custom virtual reality applications and games. This body pose information can be used for goals such as realistically puppeting a virtual avatar or controlling an on-screen mouse cursor. Additionally, the toolkit also provides a configurable input emulator that detects human actions and binds them to virtual mouse and keyboard commands, which are sent to the actively selected window. Thus, FAAST can enable natural interaction for existing off-the-shelf video games that were not explicitly developed to support input from motion sensors. The actions and input bindings are configurable at run-time, allowing the user to customize the controls and sensitivity to adjust for individual body types and preferences. In the future, we plan to substantially expand FAAST's action lexicon, provide support for recording and training custom gestures, and incorporate real-time head tracking using computer vision techniques.

Audio systems for virtual reality and augmented reality training environments commonly focus on high-quality audio reproduction. Yet many trainees may face real-world situations where hearing is compromised. In these cases, the hindrance caused by impaired or lost hearing is a significant stressor that may affect performance. Because this phenomenon is hard to simulate without actually causing hearing damage, trainees are largely unpracticed at operating with diminished hearing. To improve the match between training scenarios and the real-world situation, this effort aims to add simulated hearing loss or impairment as a training variable. Stated briefly, the goal is to effect everything users hear - including non-simulated sounds such as their own and each other's voices - without damaging their hearing, being overtly noticeable, or requiring the donning of headphones.

We present a set of algorithms and an associated display system capable of producing correctly rendered eye contact between a three-dimensionally transmitted remote participant and a group of observers in a 3D teleconferencing system. The participant's face is scanned in 3D at 30Hz and transmitted in real time to an autostereoscopic horizontal-parallax 3D display, displaying him or her over more than a 180° field of view observable to multiple observers. To render the geometry with correct perspective, we create a fast vertex shader based on a 6D lookup table for projecting 3D scene vertices to a range of subject angles, heights, and distances. We generalize the projection mathematics to arbitrarily shaped display surfaces, which allows us to employ a curved concave display surface to focus the high speed imagery to individual observers. To achieve two-way eye contact, we capture 2D video from a cross-polarized camera reflected to the position of the virtual participant's eyes, and display this 2D video feed on a large screen in front of the real participant, replicating the viewpoint of their virtual self. To achieve correct vertical perspective, we further leverage this image to track the position of each audience member's eyes, allowing the 3D display to render correct vertical perspective for each of the viewers around the device. The result is a one-to-many 3D teleconferencing system able to reproduce the effects of gaze, attention, and eye contact generally missing in traditional teleconferencing systems.

We describe a set of rendering techniques for an autostereoscopic light field display able to present interactive 3D graphics to multiple simultaneous viewers 360 degrees around the display. The display consists of a high-speed video projector, a spinning mirror covered by a holographic diffuser, and FPGA circuitry to decode specially rendered DVI video signals. The display uses a standard programmable graphics card to render over 5,000 images per second of interactive 3D graphics, projecting 360-degree views with 1.25 degree separation up to 20 updates per second. We describe the system's projection geometry and its calibration process, and we present a multiple-center-of-projection rendering technique for creating perspective-correct images from arbitrary viewpoints around the display. Our projection technique allows correct vertical perspective and parallax to be rendered for any height and distance when these parameters are known, and we demonstrate this effect with interactive raster graphics using a tracking system to measure the viewer's height and distance. We further apply our projection technique to the display of photographed light fields with accurate horizontal and vertical parallax. We conclude with a discussion of the display's visual accommodation performance and discuss techniques for displaying color imagery.

S.313

Confocal microscopy is a family of imaging techniques that employ focused patterned illumination and synchronized imaging to create cross-sectional views of 3D biological specimens. In this paper, we adapt confocal imaging to large-scale scenes by replacing the optical apertures used in microscopy with arrays of real or virtual video projectors and cameras. Our prototype implementation uses a video projector, a camera, and an array of mirrors. Using this implementation, we explore confocal imaging of partially occluded environments, such as foliage, and weakly scattering environments, such as murky water. We demonstrate the ability to selectively image any plane in a partially occluded environment, and to see further through murky water than is otherwise possible. By thresholding the confocal images, we extract mattes that can be used to selectively illuminate any plane in the scene.

Article The two-user Responsive Workbench: support for collaboration through individual views of a shared space Share on Authors: Maneesh Agrawala Stanford University, Stanford, CA Stanford University, Stanford, CAView Profile , Andrew C. Beers Stanford University, Stanford, CA Stanford University, Stanford, CAView Profile , Ian McDowall Fakespace, Inc., Mountain View, CA Fakespace, Inc., Mountain View, CAView Profile , Bernd Fröhlich Stanford University, Stanford, CA Stanford University, Stanford, CAView Profile , Mark Bolas Fakespace, Inc., Mountain View, CA Fakespace, Inc., Mountain View, CAView Profile , Pat Hanrahan Stanford University, Stanford, CA Stanford University, Stanford, CAView Profile Authors Info & Claims SIGGRAPH '97: Proceedings of the 24th annual conference on Computer graphics and interactive techniquesAugust 1997 Pages 327–332https://doi.org/10.1145/258734.258875Online:03 August 1997Publication History 168citation1,305DownloadsMetricsTotal Citations168Total Downloads1,305Last 12 Months58Last 6 weeks7 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access