Smart, pervasive Augmented Reality (AR) glasses are making their way out of the research labs. Many big tech companies are working on developing these promising next-generation interaction devices, and apps and services around them. When integrated with emerging face recognition technologies (FRT), Pervasive AR glasses can become powerful everyday tools. However, little is known about their acceptance, perceptual, and ethical ramifications. To address this, we developed a Pervasive AR technology probe with functional FRT and conducted an empirical study with 54 participants in a public environment. We collected interview data regarding perceived ethics about combining Pervasive AR with FRT. We developed five dominant themes informing the potential concerns and characteristics. Based on those findings, we propose to develop future Pervasive AR systems around principles of symmetry and consent—what we call a Kantian approach. We hope that our research will inform the design and development of near-future smart glasses.

Pervasive Augmented Reality, as a context-aware and ubiquitous technology, captures and scans a user's environment to create and tailor augmentations for the user. We assume that Pervasive AR will not only be commonplace in recognising the general environment of a user but also include the identification of people in its context by way of face recognition. While there is a body of research addressing privacy issues with Pervasive AR technology and face recognition individually, less is known about the implications of the asymmetry of information availability and agency on users' perceptions, acceptability, and ethical concerns. In addition, little is known about potential social behaviour changes due to those aspects. We exposed 50 participants to a purpose-developed Pervasive AR technology probe and explored the ethical and social implications of the experience. Our findings show that Pervasive AR, in combination with face recognition and asymmetric information delivery, can lead to skewed social interactions with consequences that affect users' sense of control, agency, and identity. Furthermore, this exploration raises questions about technology acceptability in general when bringing together emerging technologies, like AR and face recognition, as queried here.

Head-mounted displays were originally introduced to directly present computer-generated information to the human eye. More recently, the potential to use this kind of technology to support human vision and augment human perception has become actively pursued with applications such as compensating for visual impairments or aiding unimpaired vision. Unfortunately, a systematic analysis of the field is missing. Within this work, we close that gap by presenting a design space for vision augmentations that allows research to systematically explore the field of digital eyewear for vision aid and how it can augment the human visual system. We test our design space against currently available solutions and conceptually develop new solutions. The design space and findings can guide future development and can lead to a consistent categorisation of the many existing approaches.

Presence is a defining element of virtual reality (VR), but it is also increasingly used when assessing mixed reality (MR) experiences. The increased interest in measuring presence in MR and recent works underpinning the specific nature of presence in MR raise the question of the current state and practice of assessing presence in MR. To address this question, we present an analysis of more than 320 studies that report on presence measurements in MR. Our analysis showed that questionnaires are the dominant measurement but also identify problematic trends that stem from the lack of a generally agreed-upon concept or measurement for presence in MR. More specifically, we show that using measurements that are not validated in MR or custom questionnaires limiting the comparability of results is commonplace and could contribute to a looming replication crisis in an increasingly relevant field.

Acoustic noise control or cancellation (ANC) is a commonplace component of modern audio headphones. ANC aims to actively mitigate disturbing environmental noise for a quieter and improved listening experience. ANC is digitally controlling frequency and amplitude characteristics of sound. Much less explored is visual noise and active visual noise control, which we address here. We first explore visual noise and scenarios in which visual noise arises based on findings from four workshops we conducted. We then introduce the concept of visual noise cancellation (VNC) and how it can be used to reduce identified effects of visual noise. In addition, we developed head-worn demonstration prototypes to practically explore the concept of active VNC with selected scenarios in a user study. Finally, we discuss the application of VNC, including vision augmentations that moderate the user’s view of the environment to address perceptual needs and to provide augmented reality content.

Presence, or the experience of being present in a computer-generated environment, is a defining element of virtual reality. While there are different methodologies to measure presence, questionnaires remain the most popular, particularly the Igroup Presence Questionnaire (IPQ). In this paper, we analyse the results of over 20 years of IPQ usage to develop a new comparative means of reporting presence scores and comparing them across existing and future work. We additionally report on correct and problematic usage of the questionnaire and, through this, present guidelines on how to administer the IPQ in future to aid further analysis. Finally, we present a new web-based tool to streamline the analysis and reporting of IPQ results, which we hope will facilitate more standardised usage of the questionnaire in future research.

Augmented Reality has traditionally been used to display digital overlays in real environments. Many AR applications such as remote collaboration, picking tasks, or navigation require highlighting physical objects for selection or guidance. These highlights use graphical cues such as outlines and arrows. Whilst effective, they greatly contribute to visual clutter, possibly occlude scene elements, and can be problematic for long-term use. Substituting those overlays, we explore saliency modulation to accentuate objects in the real environment to guide the user’s gaze. Instead of manipulating video streams, like done in perception and cognition research, we investigate saliency modulation of the real world using optical-see-through head-mounted displays. This is a new challenge, since we do not have full control over the view of the real environment. In this work we provide our specific solution to this challenge, including built prototypes and their evaluation.

As a result of urban concentration, colonisation, increased physical distance from tribal homelands, and globalisation, many indigenous people, including Māori, are seeking digital solutions to connect to their culture and identity. Through co-design, close collaboration with our indigenous partners, and careful consideration of cultural context, we show that mixed reality experiences can be an effective mechanism for the growing diaspora of Māori to access and experience their language, genealogy, families, histories and knowledge. Inductive analysis of semi-structured interviews highlights the importance of cultural values and context in this experience, and confirms that this approach can support connections to community and culture. Our work is deeply embedded in a particular indigenous group’s context and culture, but we believe that it holds important lessons that can generalise to other groups. In particular, collaborative co-design and recognition of cultural values throughout the project are essential for producing experiences that meet the needs of specific communities, and that reflect and respect their culture and worldview.

The maturity of augmented reality (AR) technology and research now paves the way for dissemination of AR outside of the laboratory. However, it is still under-explored which factors are influencing the user experience of an AR application. In this poster, we describe some of the technical factors that could influence the user experience. We focus on a use-case in the field of on-site sports spectating with mobile AR. We present a study design which analyzes the influence of latency, registration accuracy, and jitter as factors on AR user experience.

In-stadium sports spectating delivers a unique social experience in a variety of sports. However, in contrast to broadcast delivery, it lacks the provision of real-time information augmentation, like game statistics overlaid on screen. In an earlier iteration, we developed ARSpectator, a prototypical, mobile system which can be brought to the stadium to experience both, the live sport action and situated infographics spatially augmented into the scene. In some situations it is difficult or often impossible to go to the stadium though, for instance because of limited stadium access during pandemics or when wanting to conduct controlled user studies. We address this by turning our ARSpectator system into an indirect augmented reality experience deployed to an immersive, virtual reality head-mounted display: The live stadium experience is delivered by way of a surrounding 360 video recording while maintaining and extending the provision of interactive, situated infographics. With our XRSpectator demo prototype presented here, users can have an ARSpectator experience of a rugby game in our local stadium.

In recent years, the development of Augmented Reality (AR) frameworks made AR application development widely accessible to developers without AR expert background. With this development, new application fields for AR are on the rise. This comes with an increased need for visualization techniques that are suitable for a wide range of application areas. It becomes more important for a wider audience to gain a better understanding of existing AR visualization techniques. In this article we provide a taxonomy of existing works on visualization techniques in AR. The taxonomy aims to give researchers and developers without an in-depth background in Augmented Reality the information to successively apply visualization techniques in Augmented Reality environments. We also describe required components and methods and analyze common patterns.

Augmented Reality (AR) sandtables facilitate the shaping of sand to form a surface that is transformed into a digital terrain map which is projected back onto the sand. Although a mature technology, there are still few instances of sandtables being used in surface analysis. Fundamentally there has not been any reported formal assessment of how well sandtables perform in an educational context compared to other conventional learning environments. We compared learning outcomes from using an AR sandtable versus a conventional 3D GIS to convey key concepts in terrain and hydrological analyses via usability and knowledge testing. Overall results from students at a research-intensive New Zealand university reveal a faster task performance and more learning satisfaction when using the sandtable to undertake experimental tasks. Effectiveness and knowledge quiz results revealed no significant difference between the technologies though there was a trend for more accurate answers with 3D GIS tasks. Student learning wise, the sandtable integrated core concepts (especially morphometry) more effectively though both technologies were otherwise similar. We conclude that sandtables have high potential in geospatial teaching, fostering accessible and engaging means of introducing terrain and hydrological concepts, prior to undertaking a more accurate and precise surface analysis with 3D GIS.

Recent improvements of Virtual Reality (VR) technology have enabled researchers to investigate the benefits VR may provide for various domains such as health, entertainment, training, and education. A significant proportion of VR system evaluations rely on perception-based measures such as user pre-and post-questionnaires and interviews. While these self-reports provide valuable insights into users' perceptions of VR environments, recent developments in digital sensors and data collection techniques afford researchers access to measures of physiological response. This work explores the merits of physiological measures in the evaluation of emotional responses in virtual environments (ERVE). We include and place at the center of our ERVE methodology emotional response data by way of electrodermal activity and heart-rate detection which are analyzed in conjunction with event-driven data to derive further measures. In this paper, we present our ERVE methodology together with a case study within the context of VR-based learning in which we derive measures of cognitive load and moments of insight. We discuss our methodology, and its potential for use in many other application and research domains to provide more in-depth and objective analyses of experiences within VR.

Calibrating multiple depth cameras to a common coordinate space can be a laborious and time-consuming task, and often relies on expensive motion capture tracking systems. However, if accuracy constraints can be relaxed then a consumer-grade virtual reality tracking system is sufficient to support the calibration process. In this paper, we present a fast and convenient camera calibration method aimed at such systems. The calibration process can be carried out in about ten minutes and the resulting calibration is sufficiently accurate to achieve spatial coherence when targeting reconstructions on an 8 mm grid over a ≈ 2.2m3capture volume.

The mobility and ubiquity of mobile head-mounted displays make them a promising platform for telepresence research as they allow for spontaneous and remote use cases not possible with stationary hardware. In this work we present a system that provides immersive telepresence and remote collaboration on mobile and wearable devices by building a live spherical panoramic representation of a user's environment that can be viewed in real time by a remote user who can independently choose the viewing direction. The remote user can then interact with this environment as if they were actually there through intuitive gesture-based interaction. Each user can obtain independent views within this environment by rotating their device, and their current field of view is shared to allow for simple coordination of viewpoints. We present several different approaches to create this shared live environment and discuss their implementation details, individual challenges, and performance on modern mobile hardware; by doing so we provide key insights into the design and implementation of next generation mobile telepresence systems, guiding future research in this domain. The results of a preliminary user study confirm the ability of our system to induce the desired sense of presence in its users.

Prescription glasses are used by many people as a simple, and even fashionable way, to correct refractive problems of the eye. However, there are other visual impairments that cannot be treated with an optical lens in conventional glasses. In this work we present ChromaGlasses, Computational Glasses using optical head-mounted displays for compensating colour vision deficiency. Unlike prior work that required users to look at a screen in their visual periphery rather than at the environment directly, ChromaGlasses allow users to directly see the environment using a novel head-mounted displays design that analyzes the environment in real-time and changes the appearance of the environment with pixel precision to compensate the impairment of the user. In this work, we present first prototypes for ChromaGlasses and report on the results from several studies showing that ChromaGlasses are an effective method for managing colour blindness.

Many categories of the illusion of self-motion have been widely studied with the potential support of virtual reality. However, the effects of directly and indirectly powering the movement on the possibility of perceiving changes in moving speed and their relationship with sensory feedback on users' speed change perception have not been investigated before. In this paper, we present the results of our user study on the difference in perceiving changes in moving speed between two different movement techniques: "pedaling" and "throttling". We also explore the effects of different velocity gains, accelerations and speeds of airflow, and their interactions with the movement techniques on users' perception of speed changes in addition to user performance and perception. We built a bike simulator that supports both of the movement techniques and provides sensory feedback. In general, "pedaling" gave users more possibility to perceive changes in moving velocity than "throttling".

Presents the introductory welcome message from the conference proceedings. May include the conference officers' congratulations to all involved with the conference event and publication of the proceedings record.

In this special issue ofIEEE Transactions on Visualization and Computer Graphics (TVCG), we are pleased to present theTVCGpapers from the 17th IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2018), held October 16–20 in Munich, Germany. ISMAR continues the 20-year long tradition of IWAR, ISMR, and ISAR, and is undoubtedly the premier conference for mixed and augmented reality in the world.

Mixed Reality aims at combining virtual reality with the user's surrounding real environment in a way that they form one, coherent reality. A coherent visual quality is of utmost importance, expressed in measures of e.g. resolution, framerate, and latency for both the real and the virtual domains. For years, researchers have focused on maximizing the quality of the virtual visualization mimicking the real world to get closer to visual coherence. This however, makes Mixed Reality systems overly complex and requires high computational power. In this paper, we propose a different approach by decreasing the realism of one or both visual realms, real and virtual, to achieve visual coherence. Our system coarsely voxelizes the real and virtual environments, objects, and people to provide a believable, coherent mixed voxel reality. In this paper we present the general idea, the current implementation and demonstrate the effectiveness of our approach by technical and empirical evaluations. Our mixed voxel reality system serves as a platform for low-cost presence research and studies on human perception and cognition, a host of diagnostic and therapeutic applications, and for a variety of Mixed Reality applications where users' embodiment is important. Our findings challenge some commonplace assumptions on more is better approaches in mixed reality research and practice-sometimes less can be more.

The experience of Virtual Reality (VR) can lead to unwanted or wanted psychological stress reactions. Highly immersive VR games for instance utilise extreme, life-threatening, or dangerous situations to achieve those responses from their players. There is also sufficient evidence that in clinical settings and specific situations, such as fear of heights or post-traumatic stress, virtual stimuli can lead to perceived stress for clients. However, there is a gap in research targeting everyday, mild emotional stimuli, which are neither extreme nor specific and which are not presented in an immersive system. To what extent can common stimuli in a non-immersive virtual environment elicit actual stress reactions for its users? We developed a desktop VR system and evaluated it in a study with 54 participants. We could show that virtual stimuli in a common, domestic family environment led to a significant increase in perceived stress as measured by quantitative (self-reports) and qualitative (semi-structured interviews analysed with a General Inductive Approach (GIA)) responses. The results also showed that the introduction of virtual stimuli induced significantly higher levels of perceived workload and sense of presence and led to different physiological reactions. These findings have implications for the design and implementation of non-immersive VR systems.

Augmented Reality is a technique that enables users to interact with their physical environment through the overlay of digital information. While being researched for decades, more recently, Augmented Reality moved out of the research labs and into the field. While most of the applications are used sporadically and for one particular task only, current and future scenarios will provide a continuous and multi-purpose user experience. Therefore, in this paper, we present the concept of Pervasive Augmented Reality, aiming to provide such an experience by sensing the user's current context and adapting the AR system based on the changing requirements and constraints. We present a taxonomy for Pervasive Augmented Reality and context-aware Augmented Reality, which classifies context sources and context targets relevant for implementing such a context-aware, continuous Augmented Reality experience. We further summarize existing approaches that contribute towards Pervasive Augmented Reality. Based our taxonomy and survey, we identify challenges for future research directions in Pervasive Augmented Reality.

We are presenting PanoVC - a mobile telepresence system based on continuously updated panoramic images. We are showing that the experience of telepresence, i.e. the sense of "being there together" at a distant location can be achieved with standard state-of-the-art mobile phones. Because mobile phones are always on hand users can share their environments with others in a pervasive way. Our approach is opening up the pathway for applications in a variety of domains such as the exploration of remote environments or novel forms of videoconferencing. We present implementation details, technical evaluation results, and the findings of a user study of an indoor-outdoor environments sharing task as proof of concept.

Optical see-through head-mounted displays are currently seeing a transition out of research labs towards the consumer-oriented market. However, whilst availability has improved and prices have decreased, the technology has not matured much. Most commercially available optical see-through head mounted displays follow a similar principle and use an optical combiner blending the physical environment with digital information. This approach yields problems as the colors for the overlaid digital information can not be correctly reproduced. The perceived pixel colors are always a result of the displayed pixel color and the color of the current physical environment seen through the head-mounted display. In this paper we present an initial approach for mitigating the effect of color-blending in optical see-through head-mounted displays by introducing a real-time radiometric compensation. Our approach is based on a novel prototype for an optical see-through head-mounted display that allows the capture of the current environment as seen by the user's eye. We present three different algorithms using this prototype to compensate color blending in real-time and with pixel-accuracy. We demonstrate the benefits and performance as well as the results of a user study. We see application for all common Augmented Reality scenarios but also for other areas such as Diminished Reality or supporting color-blind people.

Most interactive input devices for virtual reality-based simulators are proprietary and expensive. Can they be substituted with standard, inexpensive devices if the virtual representation of the input device looks and acts like the original? Visual dominance theory would appear to support such a possibility. According to this view the visual aspects of the displayed input device within the virtual environment should override the haptic aspects of the real device. We tested this visual dominance theory in a VR power wheelchair simulator scenario comparing standard gaming and proprietary wheelchair joysticks in combinations with their virtual counterparts and measured the effects on driving performance and reported experience. In a study with 48 participants we found significant support for visual dominance effects on driving performance. At the same time, users reported awareness of the presence and change of real joysticks and virtual representations.

Collaborative Virtual Environments (CVE) with co-located or remote video communication functionality require a continuous experience of social presence. If, at any stage during the experience the communication interrupts presence, then the CVE experience as a whole is affected - spatial presence is then decoupled from social presence. We present a solution to this problem by introducing the concept of a virtualized version of Google Glass™ called Virtual Glass. Virtual Glass is integrated into the CVE as a real-world metaphor for a communication device, one particularly suited for collaborative instructor-performer systems. In a study with 65 participants we demonstrated that the concept of Virtual Glass is effective, that it supports a high level of social presence and that the social presence is rated higher than a standard picture-in-picture videoconferencing approach for certain tasks.

Augmented reality (AR) has the potential to change the way therapy and rehabilitation is understood and administered. It can be used to manipulate the experience of reality, resulting in novel rehabilitative applications including but not limited to augmented mirror box (AMB) manipulations. We present a conceptual framework for the effective use of AR in a therapeutic context developed around the aspects of belief, interactivity, predictability, and decoupling. This framework is based on previous work in perception and emotion manipulation and is derived from and illustrated with a number of empirical studies. In particular, we describe how our augmented reflection technology (ART) system is able to manipulate the experience of reality in an effective way and how this demonstrates the potential of augmented environments to improve health and wellbeing.

Mixed reality rehabilitation systems and games are demonstrating potential as innovative adjunctive therapies for health professionals in their treatment of various hand and upper limb motor impairments. Unilateral motor deficits of the arm, for example, are commonly experienced post stroke. Our TheraMem system provides an augmented reality game environment that contributes to this increasingly rich area of research. We present a prototype system which “fools the brain” by visually amplifying users' hand movements — small actual hand movements lead to perceived larger movements. We validate the usability of our system in an empirical study with forty-five non-clinical participants. In addition, we present early qualitative evidence for the utility of our approach and system for stroke recovery and motor rehabilitation. Future uses of the system are considered by way of conclusion.

Virtual reality exposure therapy (VRET) is becoming an increasing commonplace technique for the treatment of a wide range of psychological disorders, such as phobias. Effective virtual reality systems are suggested to invoke presence, which in term elicits an emotional response, helping to lead a successful treatment outcome. However, a number of problems are apparent: (1) the expense of traditional virtual reality systems hampers their widespread adoption; (2) the depth of research into several disorders is still limited in depth; and (3) the understanding of presence and its relation to delivery mechanism and treatment outcome is still not entirely understood. We implemented and experimentally investigated an immersive VRET prototype system for the treatment of claustrophobia, a system that combines affordability, robustness and practicality while providing presence and effectiveness in treatment. The prototype system was heuristically evaluated and a controlled treatment scenario experiment using a non-clinical sample was performed. In the following, we describe the background, system concept and implementation, the tests and future directions.

Enhancing desk-based computer environments with virtual reality technology requires natural interaction support, in particular head and hand tracking. Todays motion capture systems instrument users with intrusive hardware like optical markers or data gloves which limit the perceived realism of interactions with a virtual environment and constrain the free moving space of operators. Our work therefore focuses on the development of fault-tolerant techniques for fast, non-contact 3D hand motion capture, targeted to the application in standard office environments. This paper presents a table-top setup which utilizes vision based volumetric reconstruction and tracking of skin colored objects to integrate the users hands and face into virtual environments. The system is based on off-the-shelf hardware components and satisfies real-time constraints.

In this paper, we explore ways to combine the video of a remote person with a shared tabletop display to best emulate face-to-face collaboration. Using a simple photo application we compare a variety of social and performance measures of collaboration of a standard non-spatial 2D interface with two approaches for adding spatial cues to videoconferencing: one based on simulated immersive 3D, the other based on video streams in a physically fixed arrangement around an interactive table. A face-to-face condition is included as a 'gold-standard' control. As expected, social presence and task measures were superior in the face-to-face condition, but there were also important differences between the 2D and spatial interfaces. In particular, the spatial interfaces positively influenced social presence and copresence measures in comparison to 2D, but the task measures favored the two-dimensional interface.

In addition to established methods, the treatment of mental disorders using Virtual Reality Exposure Therapy is becoming commonplace. We present an approach, which extends this concept in three ways: (1) the provision of maximum control for the therapist over the exposure environment, (2) the inclusion of different media and mediation elements in the virtuality continuum in Mixed Reality, and (3) support for co-located and remote collaboration between therapist and client. This paper gives a brief overview of related work in the field, describes our conceptual approach and a first implementation, and highlights initial findings and issues. Empirical results of our work in progress will be presented at the conference.

This paper describes the concept, prototypical implementation, and usability evaluation of an augmented virtuality (AV) based videoconferencing (VC) system: "cAR/PE!". We present a first solution which allows three participants at different locations to communicate over a network in an environment simulating a traditional face-to-face meeting. Integrated into the AV environment are live video streams of the participants spatially arranged around a virtual table, a large virtual presentation screen for 2D display and application sharing, and 3D geometry (models) within the room and on top of the table.

Industrial augmented reality (AR) applications require fast, robust, and precise tracking. In environments where conventional high-end tracking systems cannot be applied for certain reasons, marker-based tracking can be used with success as a substitute if care is taken about (1) calibration and (2) run-time tracking fidelity. In out-of-the-laboratory environments multi-marker tracking is needed because the pose estimated from a single marker is not stable enough. The overall pose estimation can be dramatically improved by fusing information from several markers fixed relative to each other compared to a single marker only. To achieve results applicable in an industrial context relative marker poses need to be properly calibrated. We propose a semiautomatic image-based calibration method requiring only minimal interaction within the workflow. Our method can be used off-line, or preferably incrementally online. When used online, our method shows reasonably good accuracy and convergence with workflow interruption of less than one second per incremental step. Thus, it can be interactively used. We illustrate our method with an industrial application scenario.

In this paper we describe the augmented reality (AR) authoring system "PowerSpace" which allows fast and comfortable generation of AR worlds. The system presented uses the functionality of a 2D presentation program (Microsoft PowerPoint) as the basis for the composition of 3D content. An MS PowerPoint export is used to generate an XML-based extensible description of a presentation. This description is enriched by 3D content with the help of an editor, which is also part of the PowerSpace system. The content of this presentation is finally converted into 3D scenes and used in an AR-viewer.

In its simplest form, reality is merely information that is presented or acquired. Mixed Reality (MR) is built around the integration of real world physical and computer generated virtual information. We do not use the term augmented reality (AR) because we view the merging of both worlds as a symbiosis, with desirable properties from each accentuated and complementing each other, rather than the enhancement of one with the other. Collaborative MR allows multiple participants to simultaneously share a physical space while being surrounded by a virtual space that is registered with the physical. Because the MR world inherits the properties of real and virtual worlds, it is rich with social context, spatial cues, and tangible objects from the real world as well as flexible digital information from virtual. We believe that Mixed Reality is a medium, largely unexplored, but very well suited for face-to-face collaboration.

This article describes a study in which a genuine effect of presence--the development of fear of virtual stimuli--was provoked. Using a self-report questionnaire, the sense of presence within this situation was measured. It was shown that fear increased with higher presence. The method, which involved 37 participants, was tested and validated with user tests at the Bauhaus University. A growing body of research in human-computer interface design for virtual environments (VE) concentrates on the problem of how to involve the user in the VE. This effect, usually called immersion or the sense of presence, has been the subject of much research activity. This research focuses on the influence of technical and technological parameters on the sense of presence. However, little work has been done on the effects of experienced sense of presence. One field in which a sense of presence is necessary for the successful application of VEs is the treatment of acrophobic patients. Our goals are to (a) create a theory-based self-report measurement for presence and (b) measure presence independently from specific effects to validate the measurement. The anxiety resulting from the confrontation with a virtual cliff is used to validate the measurement of presence.