Freehand pointing is a fundamental gesture commonly used for cursorless interactions. Prior work in HCI often elicits the same pointing behaviour—facing the target with an outstretched dominant arm and index finger. However, freehand pointing outside of HCI shows more variability across hand pose, usage, and coordination with gaze. To understand what variability exists and how it affects pointing performance, we collected data (N = 23) using a hybrid motion capture system. To elicit a wide variety of pointing behaviours, we included different levels of user effort and attention, as well as the widest range of target placements studied. We systematically characterised and described three distinct pointing behaviours, each with three different traits, ranging from accurate stereotypical pointing observed in prior works to more casual hip fire-style pointing. Our analysis demonstrates how different pointing behaviours affect pointing performance and highlights their importance when designing interactive systems for more naturalistic freehand pointing.

Immersive technologies are capable of transporting people to distant or inaccessible environments that they might not otherwise visit. Practitioners and researchers alike are discovering new ways to replicate and enhance existing tourism experiences using virtual reality, yet few controlled experiments have studied how users perceive virtual tours of real-world locations. In this paper we present an initial exploration of a new system for virtual tourism, measuring the effects of real-time experiences and storytelling on presence, place attachment, and user memories of the destination. Our results suggest that narrative plays an important role in inducing presence within and attachment to the destination, while livestreaming can further increase place attachment while providing flexible, tailored experiences. We discuss the design and evaluation of our system, including feedback from our tourism partners, and provide insights into current limitations and further opportunities for virtual tourism.

Pervasive Augmented Reality, as a context-aware and ubiquitous technology, captures and scans a user's environment to create and tailor augmentations for the user. We assume that Pervasive AR will not only be commonplace in recognising the general environment of a user but also include the identification of people in its context by way of face recognition. While there is a body of research addressing privacy issues with Pervasive AR technology and face recognition individually, less is known about the implications of the asymmetry of information availability and agency on users' perceptions, acceptability, and ethical concerns. In addition, little is known about potential social behaviour changes due to those aspects. We exposed 50 participants to a purpose-developed Pervasive AR technology probe and explored the ethical and social implications of the experience. Our findings show that Pervasive AR, in combination with face recognition and asymmetric information delivery, can lead to skewed social interactions with consequences that affect users' sense of control, agency, and identity. Furthermore, this exploration raises questions about technology acceptability in general when bringing together emerging technologies, like AR and face recognition, as queried here.

Advancements in 3D rendering like Gaussian Splatting (GS) allow novel view synthesis and real-time rendering in virtual reality (VR). However, GS-created 3D environments are often difficult to edit. For scene enhancement or to incorporate 3D assets, segmenting Gaussians by class is essential. Existing segmentation approaches are typically limited to certain types of scenes, e.g., "circular" scenes, to determine clear object boundaries. However, this method is ineffective when removing large objects in non-"circling" scenes such as large outdoor scenes.We propose Semantics-Controlled GS (SCGS), a segmentation-driven GS approach, enabling the separation of large scene parts in uncontrolled, natural environments. SCGS allows scene editing and the extraction of scene parts for VR. Additionally, we introduce a challenging outdoor dataset, overcoming the "circling" setup. We outperform the state-of-the-art in visual quality on our dataset and in segmentation quality on the 3D-OVS dataset. We conducted an exploratory user study, comparing a 360-video, plain GS, and SCGS in VR with a fixed viewpoint. In our subsequent main study, users were allowed to move freely, evaluating plain GS and SCGS. Our main study results show that participants clearly prefer SCGS over plain GS. We overall present an innovative approach that surpasses the state-of-the-art both technically and in user experience.

Image-based scene understanding allows Augmented Reality (AR) systems to provide contextual visual guidance in unprepared, real-world environments. While effective on video see-through (VST) head-mounted displays (HMDs), such methods suffer on optical see-through (OST) HMDs due to misregistration between the world-facing camera and the user's eye perspective. To approximate the user's true eye view, we implement and evaluate three software-based eye-perspective rendering (EPR) techniques on a commercially available, untethered OST HMD (Microsoft HoloLens 2): (1) Plane-Proxy EPR, projecting onto a fixed-distance plane; (2) Mesh-Proxy EPR, using SLAM-based reconstruction for projection; and (3) Gaze-Proxy EPR, a novel eye-tracking-based method that aligns the projection with the user's gaze depth. A user study on real-world tasks underscores the importance of accurate EPR and demonstrates gaze-proxy as a lightweight alternative to geometry-based methods. We release our EPR framework as open source.

Head-mounted displays were originally introduced to directly present computer-generated information to the human eye. More recently, the potential to use this kind of technology to support human vision and augment human perception has become actively pursued with applications such as compensating for visual impairments or aiding unimpaired vision. Unfortunately, a systematic analysis of the field is missing. Within this work, we close that gap by presenting a design space for vision augmentations that allows research to systematically explore the field of digital eyewear for vision aid and how it can augment the human visual system. We test our design space against currently available solutions and conceptually develop new solutions. The design space and findings can guide future development and can lead to a consistent categorisation of the many existing approaches.

Task switching can occur frequently in daily routines with physical activity. In this paper, we introduce Spatial Gaze Markers, an augmented reality tool to support users in immediately returning to the last point of interest after an attention shift. The tool is task-agnostic, using only eye-tracking information to infer distinct points of visual attention and to mark the corresponding area in the physical environment. We present a user study that evaluates the effectiveness of Spatial Gaze Markers in simulated physical repair and inspection tasks against a no-marker baseline. The results give insights into how Spatial Gaze Markers affect user performance, task load, and experience of users with varying levels of task type and distractions. Our work is relevant to assist physical workers with simple AR techniques and render task switching faster with less effort.

Providing attention guidance, such as assisting in search tasks, is a prominent use for Augmented Reality. Typically, this is achieved by graphically overlaying geometrical shapes such as arrows. However, providing visual guidance can cause side effects such as attention tunnelling or scene occlusions, and introduce additional visual clutter. Alternatively, visual guidance can adjust saliency but this comes with different challenges such as hardware requirements and environment dependent parameters. In this work we advocate for using flicker as an alternative for real-world guidance using Augmented Reality. We provide evidence for the effectiveness of flicker from two user studies. The first compared flicker against alternative approaches in a highly controlled setting, demonstrating efficacy (N = 28). The second investigated flicker in a practical task, demonstrating feasibility with higher ecological validity (N = 20). Finally, our discussion highlights the opportunities and challenges when using flicker to provide real-world visual guidance using Augmented Reality.

Presence is a defining element of virtual reality (VR), but it is also increasingly used when assessing mixed reality (MR) experiences. The increased interest in measuring presence in MR and recent works underpinning the specific nature of presence in MR raise the question of the current state and practice of assessing presence in MR. To address this question, we present an analysis of more than 320 studies that report on presence measurements in MR. Our analysis showed that questionnaires are the dominant measurement but also identify problematic trends that stem from the lack of a generally agreed-upon concept or measurement for presence in MR. More specifically, we show that using measurements that are not validated in MR or custom questionnaires limiting the comparability of results is commonplace and could contribute to a looming replication crisis in an increasingly relevant field.

We present Neural Bokeh, a deep learning approach for synthesizing convincing out-of-focus effects with applications in Mixed Reality (MR) image and video compositing. Unlike existing approaches that solely learn the amount of blur for out-of-focus areas, our approach captures the overall characteristic of the bokeh to enable the seamless integration of rendered scene content into real images, ensuring a consistent lens blur over the resulting MR composition. Our method learns spatially varying blur shapes, i.e., bokeh, from a dataset of real images acquired using the physical camera that is used to capture the photograph or video of the MR composition. Accordingly, those learned blur shapes mimic the characteristics of the physical lens. As the run-time and the resulting quality of Neural Bokeh increase with the resolution of input images, we employ low-resolution images for the MR view finding at runtime and high-resolution renderings for compositing with high-resolution photographs or videos in an offline process. We envision a variety of applications, including visual enhancement of image and video compositing containing creative utilization of out-of-focus effects.

Acoustic noise control or cancellation (ANC) is a commonplace component of modern audio headphones. ANC aims to actively mitigate disturbing environmental noise for a quieter and improved listening experience. ANC is digitally controlling frequency and amplitude characteristics of sound. Much less explored is visual noise and active visual noise control, which we address here. We first explore visual noise and scenarios in which visual noise arises based on findings from four workshops we conducted. We then introduce the concept of visual noise cancellation (VNC) and how it can be used to reduce identified effects of visual noise. In addition, we developed head-worn demonstration prototypes to practically explore the concept of active VNC with selected scenarios in a user study. Finally, we discuss the application of VNC, including vision augmentations that moderate the user’s view of the environment to address perceptual needs and to provide augmented reality content.

Presence, or the experience of being present in a computer-generated environment, is a defining element of virtual reality. While there are different methodologies to measure presence, questionnaires remain the most popular, particularly the Igroup Presence Questionnaire (IPQ). In this paper, we analyse the results of over 20 years of IPQ usage to develop a new comparative means of reporting presence scores and comparing them across existing and future work. We additionally report on correct and problematic usage of the questionnaire and, through this, present guidelines on how to administer the IPQ in future to aid further analysis. Finally, we present a new web-based tool to streamline the analysis and reporting of IPQ results, which we hope will facilitate more standardised usage of the questionnaire in future research.

Abstract In augmented reality applications it is essential to know the position and orientation of the user to correctly register virtual 3D content in the user’s field of view. For this purpose, visual tracking through simultaneous localization and mapping (SLAM) is often used. However, when applied to the commonly occurring situation where the users are mostly stationary, many methods presented in previous research have two key limitations. First, SLAM techniques alone do not address the problem of global localization with respect to prior models of the environment. Global localization is essential in many applications where multiple users are expected to track within a shared space, such as spectators at a sporting event. Secondly, these methods often assume significant translational movement to accurately reconstruct and track from a local model of the environment, causing challenges for many stationary applications. In this paper, we extend recent research on Spherical Localization and Tracking to support relocalization after tracking failure, as well as global localization in large shared environments, and optimize the method for operation on mobile hardware. We also evaluate various state-of-the-art localization approaches, the robustness of our visual tracking method, and demonstrate the effectiveness of our system in real-life scenarios.

Optical see-through (OST) head-mounted displays (HMDs) enable users to experience Augmented Reality (AR) support in the form of helpful real-world annotations. Unfortunately, the blend of the environment with virtual augmentations due to semitransparent OST displays often deteriorates the contrast and legibility of annotations. View management algorithms adapt the annotations’ layout to improve legibility based on real-world information, typically captured by built-in HMD cameras. However, the camera views are different from the user’s view through the OST display which decreases the final layout quality. We present eye-perspective view management that synthesizes high-fidelity renderings of the user’s view to optimize annotation placement. Our method significantly improves over traditional camera-based view management in terms of annotation placement and legibility. Eye-perspective optimizations open up opportunities for further research on use cases relying on the user’s true view through OST HMDs.

This work introduces off-axis layered displays, the first approach to stereoscopic direct-view displays with support for focus cues. Off-axis layered displays combine a head-mounted display with a traditional direct-view display for encoding a focal stack and thus, for providing focus cues. To explore the novel display architecture, we present a complete processing pipeline for the real-time computation and post-render warping of off-axis display patterns. In addition, we build two prototypes using a head-mounted display in combination with a stereoscopic direct-view display, and a more widely available monoscopic direct-view display. In addition we show how extending off-axis layered displays with an attenuation layer and with eye-tracking can improve image quality. We thoroughly analyze each component in a technical evaluation and present examples captured through our prototypes.

Augmented Reality has traditionally been used to display digital overlays in real environments. Many AR applications such as remote collaboration, picking tasks, or navigation require highlighting physical objects for selection or guidance. These highlights use graphical cues such as outlines and arrows. Whilst effective, they greatly contribute to visual clutter, possibly occlude scene elements, and can be problematic for long-term use. Substituting those overlays, we explore saliency modulation to accentuate objects in the real environment to guide the user’s gaze. Instead of manipulating video streams, like done in perception and cognition research, we investigate saliency modulation of the real world using optical-see-through head-mounted displays. This is a new challenge, since we do not have full control over the view of the real environment. In this work we provide our specific solution to this challenge, including built prototypes and their evaluation.

Colour vision deficiency is a common visual impairment that cannot be compensated for using optical lenses in traditional glasses, and currently remains untreatable. In our work, we report on research on Computational Glasses for compensating colour vision deficiency. While existing research only showed corrected images within the periphery or as an indirect aid, Computational Glasses build on modified standard optical see-through head-mounted displays and directly modulate the user’s vision, consequently adapting their perception of colours. In this work, we present an exhaustive literature review of colour vision deficiency compensation and subsequent findings; several prototypes with varying advantages—from well-controlled bench prototypes to less controlled but higher application portable prototypes; and a series of studies evaluating our approach starting with proving its efficacy, comparing to the state-of-the-art, and extending beyond static lab prototypes looking at real world applicability. Finally, we evaluated directions for future compensation methods for computational glasses.

Coherent rendering is important for generating plausible Mixed Reality presentations of virtual objects within a user’s real-world environment. Besides photo-realistic rendering and correct lighting, visual coherence requires simulating the imaging system that is used to capture the real environment. While existing approaches either focus on a specific camera or a specific component of the imaging system, we introduce Neural Cameras, the first approach that jointly simulates all major components of an arbitrary modern camera using neural networks. Our system allows for adding new cameras to the framework by learning the visual properties from a database of images that has been captured using the physical camera. We present qualitative and quantitative results and discuss future direction for research that emerge from using Neural Cameras.

We present computational phase-modulated eyeglasses, a see-through optical system that modulates the view of the user using phase-only spatial light modulators (PSLM). A PSLM is a programmable reflective device that can selectively retardate, or delay, the incoming light rays. As a result, a PSLM works as a computational dynamic lens device. We demonstrate our computational phase-modulated eyeglasses with either a single PSLM or dual PSLMs and show that the concept can realize various optical operations including focus correction, bi-focus, image shift, and field of view manipulation, namely optical zoom. Compared to other programmable optics, computational phase-modulated eyeglasses have the advantage in terms of its versatility. In addition, we also presents some prototypical focus-loop applications where the lens is dynamically optimized based on distances of objects observed by a scene camera. We further discuss the implementation, applications but also discuss limitations of the current prototypes and remaining issues that need to be addressed in future research.

In recent years, the development of Augmented Reality (AR) frameworks made AR application development widely accessible to developers without AR expert background. With this development, new application fields for AR are on the rise. This comes with an increased need for visualization techniques that are suitable for a wide range of application areas. It becomes more important for a wider audience to gain a better understanding of existing AR visualization techniques. In this article we provide a taxonomy of existing works on visualization techniques in AR. The taxonomy aims to give researchers and developers without an in-depth background in Augmented Reality the information to successively apply visualization techniques in Augmented Reality environments. We also describe required components and methods and analyze common patterns.

Remote assistance represents an important use case for mixed reality. With the rise of handheld and wearable devices, remote assistance has become practical in the wild. However, spontaneous provisioning of remote assistance requires an easy, fast and robust approach for capturing and sharing of unprepared environments. In this work, we make a case for utilizing interactive light fields for remote assistance. We demonstrate the advantages of object representation using light fields over conventional geometric reconstruction. Moreover, we introduce an interaction method for quickly annotating light fields in 3D space without requiring surface geometry to anchor annotations. We present results from a user study demonstrating the effectiveness of our interaction techniques, and we provide feedback on the usability of our overall system.

When implementing an Augmented Reality (AR) interface, it is essential to track camera motion in order to precisely register the virtual overlay in the view of the user. However, unlike most indoor AR scenarios, in many outdoor scenarios the user maintains a static position performing mostly rotational movements. Simultaneous Localization and Mapping (SLAM) methods typically used to solve the tracking problem require significant translational camera motion to perform reliably. The magnitude of the required translation is proportional to the size of the scene, exacerbating this problem in large environments such as open places or stadiums. In this paper, we present an alternative SLAM method, which combines spherical Structure-from-Motion and a robust 3D tracking method. We compare our method to ORB SLAM2 in synthetic and real tests, and show that our method can track more reliably in large spaces, with simpler calculation due to the spherical motion constraint. We discuss this issue in the context of implementing an AR interface for live sport events in stadiums or other open environments, but possible application scenarios for our technique go beyond and can be applied to handheld AR in many outdoor environments.

Target disambiguation is a common problem in gaze interfaces, as eye tracking has accuracy and precision limitations. In 3D environments this is compounded by objects overlapping in the field of view, as a result of their positioning at different depth with partial occlusion. We introduce VOR depth estimation, a method based on the Vestibulo-ocular reflex of the eyes in compensation of head movement, and explore its application to resolve target ambiguity. The method estimates gaze depth by comparing the rotations of the eye and the head when the users look at a target and deliberately rotate their head. We show that VOR eye movement presents an alternative to vergence for gaze depth estimation, that is feasible also with monocular tracking. In an evaluation of its use for target disambiguation, our method outperforms vergence for targets presented at greater depth.

The latest generation of consumer market Head-mounted displays (HMD) now include self-contained inside-out tracking of head motions, which makes them suitable for mobile applications. However, 3D tracking of input devices is either not included at all or requires to keep the device in sight, so that it can be observed from a sensor mounted on the HMD. Both approaches make natural interactions cumbersome in mobile applications. TrackCap, a novel approach for 3D tracking of input devices, turns a conventional smartphone into a precise 6DOF input device for an HMD user. The device can be conveniently operated both inside and outside the HMD's field of view, while it provides additional 2D input and output capabilities.

Recent improvements of Virtual Reality (VR) technology have enabled researchers to investigate the benefits VR may provide for various domains such as health, entertainment, training, and education. A significant proportion of VR system evaluations rely on perception-based measures such as user pre-and post-questionnaires and interviews. While these self-reports provide valuable insights into users' perceptions of VR environments, recent developments in digital sensors and data collection techniques afford researchers access to measures of physiological response. This work explores the merits of physiological measures in the evaluation of emotional responses in virtual environments (ERVE). We include and place at the center of our ERVE methodology emotional response data by way of electrodermal activity and heart-rate detection which are analyzed in conjunction with event-driven data to derive further measures. In this paper, we present our ERVE methodology together with a case study within the context of VR-based learning in which we derive measures of cognitive load and moments of insight. We discuss our methodology, and its potential for use in many other application and research domains to provide more in-depth and objective analyses of experiences within VR.

Calibrating multiple depth cameras to a common coordinate space can be a laborious and time-consuming task, and often relies on expensive motion capture tracking systems. However, if accuracy constraints can be relaxed then a consumer-grade virtual reality tracking system is sufficient to support the calibration process. In this paper, we present a fast and convenient camera calibration method aimed at such systems. The calibration process can be carried out in about ten minutes and the resulting calibration is sufficiently accurate to achieve spatial coherence when targeting reconstructions on an 8 mm grid over a ≈ 2.2m3capture volume.

We present a display for optical see-through near-eye displays based on light attenuation, a new paradigm that forms images by spatially subtracting colors of light. Existing optical see-through head-mounted displays (OST-HMDs) form virtual images in an additive manner-they optically combine the light from an embedded light source such as a microdisplay into the users' field of view (FoV). Instead, our light attenuation display filters the color of the real background light pixel-wise in the users' see-through view, resulting in an image as a spatial color filter. Our image formation is complementary to existing light-additive OST-HMDs. The core optical component in our system is a phase-only spatial light modulator (PSLM), a liquid crystal module that can control the phase of the light in each pixel. By combining PSLMs with polarization optics, our system realizes a spatially programmable color filter. In this paper, we introduce our optics design, evaluate the spatial color filter, consider applications including image rendering and FoV color control, and discuss the limitations of the current prototype.

The mobility and ubiquity of mobile head-mounted displays make them a promising platform for telepresence research as they allow for spontaneous and remote use cases not possible with stationary hardware. In this work we present a system that provides immersive telepresence and remote collaboration on mobile and wearable devices by building a live spherical panoramic representation of a user's environment that can be viewed in real time by a remote user who can independently choose the viewing direction. The remote user can then interact with this environment as if they were actually there through intuitive gesture-based interaction. Each user can obtain independent views within this environment by rotating their device, and their current field of view is shared to allow for simple coordination of viewpoints. We present several different approaches to create this shared live environment and discuss their implementation details, individual challenges, and performance on modern mobile hardware; by doing so we provide key insights into the design and implementation of next generation mobile telepresence systems, guiding future research in this domain. The results of a preliminary user study confirm the ability of our system to induce the desired sense of presence in its users.

Prescription glasses are used by many people as a simple, and even fashionable way, to correct refractive problems of the eye. However, there are other visual impairments that cannot be treated with an optical lens in conventional glasses. In this work we present ChromaGlasses, Computational Glasses using optical head-mounted displays for compensating colour vision deficiency. Unlike prior work that required users to look at a screen in their visual periphery rather than at the environment directly, ChromaGlasses allow users to directly see the environment using a novel head-mounted displays design that analyzes the environment in real-time and changes the appearance of the environment with pixel precision to compensate the impairment of the user. In this work, we present first prototypes for ChromaGlasses and report on the results from several studies showing that ChromaGlasses are an effective method for managing colour blindness.

Mixed Reality aims at combining virtual reality with the user's surrounding real environment in a way that they form one, coherent reality. A coherent visual quality is of utmost importance, expressed in measures of e.g. resolution, framerate, and latency for both the real and the virtual domains. For years, researchers have focused on maximizing the quality of the virtual visualization mimicking the real world to get closer to visual coherence. This however, makes Mixed Reality systems overly complex and requires high computational power. In this paper, we propose a different approach by decreasing the realism of one or both visual realms, real and virtual, to achieve visual coherence. Our system coarsely voxelizes the real and virtual environments, objects, and people to provide a believable, coherent mixed voxel reality. In this paper we present the general idea, the current implementation and demonstrate the effectiveness of our approach by technical and empirical evaluations. Our mixed voxel reality system serves as a platform for low-cost presence research and studies on human perception and cognition, a host of diagnostic and therapeutic applications, and for a variety of Mixed Reality applications where users' embodiment is important. Our findings challenge some commonplace assumptions on more is better approaches in mixed reality research and practice-sometimes less can be more.

User-reviews of mobile applications provide information that benefits other users and developers. Even though reviews contain feedback about an app's performance and problematic features, users and app developers need to spend considerable effort reading and analyzing the feedback provided. In this work, we introduce and evaluate QuickReview, an intelligent user interface for reporting problematic app features. Preliminary user evaluations show that QuickReview facilitates users to add reviews swiftly with ease, and also helps developers with quick interpretation of submitted reviews by presenting a ranked list of commonly reported features.

Augmented Reality is a technique that enables users to interact with their physical environment through the overlay of digital information. While being researched for decades, more recently, Augmented Reality moved out of the research labs and into the field. While most of the applications are used sporadically and for one particular task only, current and future scenarios will provide a continuous and multi-purpose user experience. Therefore, in this paper, we present the concept of Pervasive Augmented Reality, aiming to provide such an experience by sensing the user's current context and adapting the AR system based on the changing requirements and constraints. We present a taxonomy for Pervasive Augmented Reality and context-aware Augmented Reality, which classifies context sources and context targets relevant for implementing such a context-aware, continuous Augmented Reality experience. We further summarize existing approaches that contribute towards Pervasive Augmented Reality. Based our taxonomy and survey, we identify challenges for future research directions in Pervasive Augmented Reality.

We are presenting PanoVC - a mobile telepresence system based on continuously updated panoramic images. We are showing that the experience of telepresence, i.e. the sense of "being there together" at a distant location can be achieved with standard state-of-the-art mobile phones. Because mobile phones are always on hand users can share their environments with others in a pervasive way. Our approach is opening up the pathway for applications in a variety of domains such as the exploration of remote environments or novel forms of videoconferencing. We present implementation details, technical evaluation results, and the findings of a user study of an indoor-outdoor environments sharing task as proof of concept.

Optical see-through head-mounted displays are currently seeing a transition out of research labs towards the consumer-oriented market. However, whilst availability has improved and prices have decreased, the technology has not matured much. Most commercially available optical see-through head mounted displays follow a similar principle and use an optical combiner blending the physical environment with digital information. This approach yields problems as the colors for the overlaid digital information can not be correctly reproduced. The perceived pixel colors are always a result of the displayed pixel color and the color of the current physical environment seen through the head-mounted display. In this paper we present an initial approach for mitigating the effect of color-blending in optical see-through head-mounted displays by introducing a real-time radiometric compensation. Our approach is based on a novel prototype for an optical see-through head-mounted display that allows the capture of the current environment as seen by the user's eye. We present three different algorithms using this prototype to compensate color blending in real-time and with pixel-accuracy. We demonstrate the benefits and performance as well as the results of a user study. We see application for all common Augmented Reality scenarios but also for other areas such as Diminished Reality or supporting color-blind people.

Collaborative Virtual Environments (CVE) with co-located or remote video communication functionality require a continuous experience of social presence. If, at any stage during the experience the communication interrupts presence, then the CVE experience as a whole is affected - spatial presence is then decoupled from social presence. We present a solution to this problem by introducing the concept of a virtualized version of Google Glass™ called Virtual Glass. Virtual Glass is integrated into the CVE as a real-world metaphor for a communication device, one particularly suited for collaborative instructor-performer systems. In a study with 65 participants we demonstrated that the concept of Virtual Glass is effective, that it supports a high level of social presence and that the social presence is rated higher than a standard picture-in-picture videoconferencing approach for certain tasks.

In this work, we present an approach for optimizing targets for natural feature-based pose tracking such as used in Augmented Reality applications. Our contribution is an approach for locally optimizing a given tracking target instead of applying global optimizations, such as proposed in the literature. The local optimization together with visualized trackability rating leads to a tool to create high quality tracking targets.

Photorealistic Augmented Reality (AR) requires knowledge of the scene geometry and environment lighting to compute photometric registration. Recent work has introduced probeless photometric registration, where environment lighting is estimated directly from observations of reflections in the scene rather than through an invasive probe such as a reflective ball. However, computing the dense radiance transfer of a dynamically changing scene is computationally challenging. In this work, we present an improved radiance transfer sampling approach, which combines adaptive sampling in image and visibility space with robust caching of radiance transfer to yield real time framerates for photorealistic AR scenes with dynamically changing scene geometry and environment lighting.

As low-level hardware will soon allow us to visualize virtual content anywhere in the real world, managing it in a more structured manner still needs to be addressed. Augmented reality (AR) browser technology is the gateway to such structured software platform and an anywhere AR user experience. AR browsers are the substitute of Web browsers in the real world, permitting overlay of interactive multimedia content on the physical world or objects they refer to. As the current generation allows us to barely see floating virtual items in the physical world, a tighter coupling with our reality has not yet been explored. This paper presents our recent effort to create rich, seamless, and adaptive AR browsers. We discuss major challenges in the area and present an agenda on future research directions for an everyday augmented world.

Micro aerial vehicles equipped with high-resolution cameras can be used to create aerial reconstructions of an area of interest. In that context automatic flight path planning and autonomous flying is often applied but so far cannot fully replace the human in the loop, supervising the flight on-site to assure that there are no collisions with obstacles. Unfortunately, this workflow yields several issues, such as the need to mentally transfer the aerial vehicle’s position between 2D map positions and the physical environment, and the complicated depth perception of objects flying in the distance. Augmented Reality can address these issues by bringing the flight planning process on-site and visualizing the spatial relationship between the planned or current positions of the vehicle and the physical environment. In this paper, we present Augmented Reality supported navigation and flight planning of micro aerial vehicles by augmenting the user’s view with relevant information for flight planning and live feedback for flight supervision. Furthermore, we introduce additional depth hints supporting the user in understanding the spatial relationship of virtual waypoints in the physical world and investigate the effect of these visualization techniques on the spatial understanding.

The development of mobile Augmented Reality application became increasingly popular over the last few years. However, many of the existing solutions build on the reuse of available standard metaphors for visualization and interaction without considering the manifold contextual factors of their use. Within this workshop we want to discuss theoretical design approaches and practical tools which should help developers to make more informed choices when exploring the design space of Augmented Reality interfaces in mobile contexts.

The motivation of this workshop is to discuss future direction of content authoring in the field of Augmented Reality, as well as to discuss the current state of art on content creation and content authoring for augmented reality. The workshop will comprise of a paper session where authoring papers, late-breaking results and overviews over state-of-the-art are presented. In the afternoon, we will follow up with discussion sessions on different topics ranging from content creation and authoring to content distribution for AR and a short closing session.

In this paper, we introduce a novel view management technique for placing labels in Augmented Reality systems. A common issue in many Augmented Reality applications is the absence of knowledge of the real environment, limiting the efficient representation and optimal layout of the digital information augmented onto the real world. To overcome this problem, we introduce an image-based approach, which combines a visual saliency algorithm with edge analysis to identify potentially important image regions and geometric constraints for placing labels. Our proposed solution also includes adaptive rendering techniques that allow a designer to control the appearance of depth cues. We describe the results obtained from a user study considering different scenarios, which we performed for validating our approach. Our technique will provide special benefits to Augmented Reality browsers that usually lack scene knowledge, but also to many other applications in the domain of Augmented Reality such as cultural heritage and maintenance applications.

The motivation of this workshop is to discuss future directions of content authoring in the field of Augmented Reality, as well as to discuss the current state of art on content creation and asset assembly. The workshop will comprise of a paper session where papers, late-breaking results and overviews over state-of-the-art in content authoring for AR are presented. In the afternoon, we will follow up with a discussion on different topics ranging from AR asset creation to content distribution and a closing session. Our goal is to collect ideas and thoughts of research about desired approaches for authoring content for AR, as well as review current and future needs to achieve a high quality content and at the same time scalable approaches for AR.

A common goal of outdoor augmented reality (AR) is the presentation of annotations that are registered to anchor points in the real world. We present an enhanced approach for registering and tracking such anchor points, which is suitable for current generation mobile phones and can also successfully deal with the wide variety of viewing conditions encountered in real life outdoor use. The approach is based on on-the-fly generation of panoramic images by sweeping the camera over the scene. The panoramas are then used for stable orientation tracking, while the user is performing only rotational movements. This basic approach is improved by several new techniques for the re-detection and tracking of anchor points. For the re-detection, specifically after temporal variations, we first compute a panoramic image with extended dynamic range, which can better represent varying illumination conditions. The panorama is then searched for known anchor points, while orientation tracking continues uninterrupted. We then use information from an internal orientation sensor to prime an active search scheme for the anchor points, which improves matching results. Finally, global consistency is enhanced by statistical estimation of a global rotation that minimizes the overall position error of anchor points when transforming them from the source panorama in which they were created, to the current view represented by a new panorama. Once the anchor points are redetected, we track the user's movement using a novel 3-degree-of-freedom orientation tracking approach that combines vision tracking with the absolute orientation from inertial and magnetic sensors. We tested our system using an AR campus guide as an example application and provide detailed results for our approach using an off-the-shelf smartphone. Results show that the re-detection rate is improved by a factor of 2 compared to previous work and reaches almost 90% for a wide variety of test cases while still keeping the ability to run at interactive frame rates.

We present a novel method for the real-time creation and tracking of panoramic maps on mobile phones. The maps generated with this technique are visually appealing, very accurate and allow drift-free rotation tracking. This method runs on mobile phones at 30 Hz and has applications in the creation of panoramic images for offline browsing, for visual enhancements through environment mapping and for outdoor Augmented Reality on mobile phones.

Marker tracking has revolutionized augmented reality about a decade ago. However, this revolution came at the expense of visual clutter. In this paper, we propose several new marker techniques, which are less obtrusive than the usual black and white squares. Furthermore, we report methods that allow tracking beyond the visibility of these markers further improving robustness. All presented techniques are implemented in a single tracking library, are highly efficient in their memory and CPU usage and run at interactive frame rates on mobile phones.