In virtual reality (VR), portals serve as a common metaphor for relocation, allowing users to navigate large virtual environments. However, portal design is often ad hoc, leaving developers without clear principles for their intentional design and placement to support specific experiential or performance goals. We present a taxonomy for VR portals, deconstructing them into three main dimensions: World-related, Portal-related, and User-related. Based on the suggested taxonomy, we implemented eight unique portals varying across three categories: World Integration (Integrated vs. Non-Integrated), Nature Definition (Obvious vs. Subtle/Hidden), and Destination Visibility (Preview vs. Occluded). We assess the impact of these factors, focusing on spatial memory, presence, and experience in a user study with 32 participants. Our results show that World-Integrated portals improve spatial memory and realism, while the exposure of portal’s unusual Nature mainly affects perceived realism. Visibility of the portal’s Destination positively affected user comfort, acceptance, and presence. We discuss the insights for the intentional design of a portal-based relocation techniques that support comfortable and intuitive navigation. Such techniques are particularly suitable for small physical spaces, where portals enable space-efficient, self-overlapping virtual architectures.

Steering techniques are common for traveling in Virtual Reality (VR) applications. However, these techniques can generate Unintended Positional Drift (UPD), a phenomenon characterized by the user’s unconscious or unintentional physical movements of the users in the workspace. Although some research work has already investigated the influence of heading (i.e., how the virtual trajectory direction is updated) and cognitive load on travel performance using steering techniques, they did not investigate whether these factors could increase UPD. In this paper, we thus replicated and extended an existing user study to assess the influence of heading and cognitive task (CT) on UPD while navigating with steering techniques in VR. Participants had to perform a multidirectional travel task (based on the ISO 9241-9) while doing a CT (two-back task). We considered conditions with and without performing the CT, where we varied the steering heading (head, hand, torso). We compared our results with those of a previous study and found that the heading did not influence users’ travel and cognitive performance. However, we noticed the influence of workload on the user’s UPD. This work contributes to understanding UPD, which tends to be an overlooked topic, and the cognitive implications of navigation in VR. We discuss the design implications of these results for improving navigation in VR.

We present a novel shape-shifting haptic device, Shiftly, which renders plausible haptic feedback when touching virtual objects in Virtual Reality (VR). By changing its shape, different geometries of virtual objects can be approximated to provide haptic feedback for the user's hand. The device employs only three actuators and three curved origamis that can be programmatically folded and unfolded to create a variety of touch surfaces ranging from flat to curved. In this paper, we present the design of Shiftly, including its kinematic model and integration into VR setups for haptics. We also assessed Shiftly using two user studies. The first study evaluated how well Shiftly can approximate different shapes without visual representation. The second study investigated the realism of the haptic feedback with Shiftly for a user when touching a rendered virtual object. The results showed that our device can provide realistic haptic feedback for flat surfaces, convex shapes of different curvatures, and edge-shaped geometries. Shiftly can less realistically render concave surfaces and objects with small details.

Hydropower turbines are large-scale equipment essential to sustainable energy supply chains, and engineers have few opportunities to examine their internal structure. Our Immersive Analytics (IA) application is part of a research project that combines and compares simulated water turbine flows and sensor-measured data, looking for data-driven predictions of the lifetime of the mechanical parts of hydroelectric power plants. Our prototype combines spatial and abstract data in an immersive environment in which the user can navigate through a full-scale model of a water turbine, view simulated water flows of three different energy supply conditions, and visualize and interact with sensor-collected data situated at the reference position of the sensors in the actual turbine. In this paper, we detail our design process, which resulted from consultations with domain experts and a literature review, give an overview of our prototype, and present its evaluation, resulting from semi-structured interviews with experts and qualitative thematic analysis. Our findings confirm the current literature that IA applications add value to the presentation and analysis of situated data, as they show that we advance in the design directions for IA applications for domain experts that combine abstract and spatial data, with conclusions on how to avoid skepticism from such professionals.

This article presents a brain–computer interface (BCI) coupled with an augmented reality (AR) system to support human–robot interaction in controlling a robotic arm for pick-and-place tasks. BCIs can process steady-state visual evoked potentials (SSVEPs), which are signals generated through visual stimuli. The visual stimuli may be conveyed to the user with AR systems, expanding the range of possible applications. The proposed approach leverages the capabilities of the NextMind BCI to enable users to select objects in the range of the robotic arm. By displaying a visual anchor associated with each object in the scene with projected AR, the NextMind device can detect when users focus their eyesight on one of them, thus triggering the pick-up action of the robotic arm. The proposed system has been designed considering the needs and limitations of mobility-impaired people to support them when controlling a robotic arm for pick-and-place tasks. Two different approaches for positioning the visual anchors are proposed and analyzed. Experimental tests involving users show that both approaches are highly appreciated. The system performances are extremely robust, thus allowing the users to select objects in an easy, fast, and reliable way.

Sketching is a fundamental technique for early design and form finding. Digital 3D sketching can improve the early design phase by improving spatial understanding and enriching the design with additional information. However, the tools used for sketching should not hinder the expression and thought process inherent in form finding. Methods already exist for using 2D pen-on-tablet input for 3D sketching via stroke projection onto 3D drawing canvases. However, positioning the canvas and sketching lines are separate work steps. This breaks the flow of the designer’s thought process. We propose a novel technique for mixed reality 3D sketching that involves the use of viewport-attached drawing canvases, spatial meshing and intersection canvas visualisation. By combining the inside-out tracking capabilities of current portable consumer devices with stylus-on-tablet freehand drawing input, we transform 2D to 3D projective sketching into a more seamless experience. Results of a pilot user study with 16 participants show significant user preference for our technique, as well as increased sketching speed and immediacy.

We present CoboDeck - our proof-of-concept immersive virtual reality haptic system with free walking support. It provides prop-based encountered-type haptic feedback with a mobile robotic platform. Intended for use as a design tool for architects, it enables the user to directly and intuitively interact with virtual objects like walls, doors, or furniture. A collaborative robotic arm mounted on an omnidirectional mobile platform can present a physical prop that matches the position and orientation of a virtual counterpart anywhere in large virtual and real environments. We describe the concept, hardware, and software architecture of our system. Furthermore, we present the first behavioral algorithm tailored for the unique challenges of safe human-robot haptic interaction in VR, explicitly targeting availability and safety while the user is unaware of the robot and can change trajectory at any time. We explain our high-level state machine that controls the robot to follow a user closely and rapidly escape from him as required by the situation. We present our technical evaluation. The results suggest that our chasing approach saves time, decreases the travel distance and thus battery usage, compared to more traditional approaches for mobile platforms assuming a fixed parking position between interactions. We also show that the robot can escape from the user and prevent a possible collision within a mean time of 1.62 s. Finally, we confirm the validity of our approach in a practical validation and discuss the potential of the proposed system.

Microservices are widely used by large internet companies as they support scalable systems with high resilience and fault-tolerance, flexible and agile development, and continuous delivery enabling fast time to market. Hence, there is an increasing interest in adopting microservices in the field of industrial automation. This raises the question, if and to what degree this architectural style can also be applied for the development of industrial control systems (ICS). In this paper, we have systematically analyzed the applicability of microservices for ICS development. Together with domain experts from industry, we have developed a migration path from a monolithic ICS towards cloud-ready systems based on microservices. By studying the central principles for microservice development and operation, we found that microservices can be applied in the context of ICS and the use of microservices leads to increased flexibility with regard to frequent software releases and the development of new deployment variants. However, communication between real-time services is still an open research challenge that poses a potential technical risk in the migration towards adopting full microservice-based system architectures.

This paper presents a novel method for the visualization of 3D spatial sounds in Virtual Reality (VR) for Deaf and Hard-of-Hearing (DHH) people. Our method enhances traditional VR devices with additional haptic and visual feedback, which aids spatial sound localization. The proposed system automatically analyses 3D sound from VR application, and it indicates the direction of sound sources to a user by two Vibro-motors and two Light-Emitting Diodes (LEDs). The benefit of automatic sound analysis is that our method can be used in any VR application without modifying the application itself. We evaluated the proposed method for 3D spatial sound visualization in a user study. Additionally, the conducted user study investigated which condition (corresponding to different senses) leads to faster performance in 3D sound localization task. For this purpose, we compared three conditions: haptic feedback only, LED feedback only, combined haptic and LED feedback. Our study results suggest that DHH participants could complete sound-related VR tasks significantly faster using LED and haptic+LED conditions in comparison to only haptic feedback. The presented method for spatial sound visualization can be directly used to enhance VR applications for use by DHH persons, and the results of our user study can serve as guidelines for the future design of accessible VR systems.

Information visualization techniques play an important role in Virtual Reality (VR) because they improve task performance, support cognitive processes, and eventually increase the feeling of immersion. Deaf and Hard-of-Hearing (DHH) persons have special needs for information presentation because they feel and perceive VR environments differently. Therefore, it is necessary to pay attention to requirements about presenting information in VR for this group of users. Previous research showed that adding special features and using haptic methods helps DHH persons to do VR tasks better. In this paper, we propose a novel Omni-directional particle visualization method and also evaluate multi-modal presentation methods in VR for DHH persons, such as audio, visual, haptic, and a combination of them (AVH). Additionally, we compare the results with the results of persons without hearing problems. The methods for information presentation in our study focus on spatial object localization in VR. Our user studies show that both DHH persons and persons without hearing problems were able to do VR tasks significantly faster using AVH. Also, we found out that DHH persons can do visual-related VR tasks faster than persons without hearing problems by using our new proposed visualization method. Our results suggest that the benefits of using audio among persons without hearing problems and the benefits of using vision among DHH persons cause an interesting balance in the results of AVH between both groups. Finally, our qualitative and quantitative evaluation indicates that both groups of participants preferred and enjoyed AVH modality more than other modalities.

We introduce a novel framework for 3D scene reconstruction with simultaneous object annotation, using a pre-trained 2D convolutional neural network (CNN), incremental data streaming, and remote exploration, with a virtual reality setup. It enables versatile integration of any 2D box detection or segmentation network. We integrate new approaches to (i) asynchronously perform dense 3D-reconstruction and object annotation at interactive frame rates, (ii) efficiently optimize CNN results in terms of object prediction and spatial accuracy, and (iii) generate computationally-efficient colliders in large triangulated 3D-reconstructions at run-time for 3D scene interaction. Our method is novel in combining CNNs with long and varying inference time with live 3D-reconstruction from RGB-D camera input. We further propose a lightweight data structure to store the 3D-reconstruction data and object annotations to enable fast incremental data transmission for real-time exploration with a remote client, which has not been presented before. Our framework achieves update rates of 22 fps (SSD Mobile Net) and 19 fps (Mask RCNN) for indoor environments up to 800 m 3 . We evaluated the accuracy of 3D-object detection. Our work provides a versatile foundation for semantic scene understanding of large streamed 3D-reconstructions, while being independent from the CNN’s processing time. Source code is available for non-commercial use.

Virtual reality (VR) is limited in many ways and often is incomparable to real-world experience. Walkable smooth uneven surfaces are inherent to reality but extremely lacking in VR. At the same time, VR offers a lot of possibilities for manipulations. In this paper, we focus on human height and slant perception of the uneven surfaces with multi-sensory stimulation in VR. By employing viewport manipulations, haptic, and vibrotactile stimuli, we explore the possibility to simulate uneven surfaces different from the physical props used.Our results suggest that the use of a rounded prop helps to create a more convincing illusion of an uneven surface that is significantly higher than the physical one. The multi-sensory stimulation brings both height and slant estimations closer to the values suggested by the visual cues if there is no conflict with the haptic sensations. The use of a flat prop is less realistic and leads to massive height and slant underestimations as opposed to those suggested by visual cues. However, if the curved prop cannot be used, a flat surface might still be used to simulate small dents and bumps.

Virtual Reality (VR) has a great potential to improve skills of Deaf and Hard-of-Hearing (DHH) people. Most VR applications and devices are designed for persons without hearing problems. Therefore, DHH persons have many limitations when using VR. Adding special features in a VR environment, such as subtitles, or haptic devices will help them. Previously, it was necessary to design a special VR environment for DHH persons. We introduce and evaluate a new prototype called "EarVR" that can be mounted on any desktop or mobile VR Head-Mounted Display (HMD). EarVR analyzes 3D sounds in a VR environment and locates the direction of the sound source that is closest to a user. It notifies the user about the sound direction using two vibro-motors placed on the user's ears. EarVR helps DHH persons to complete sound-based VR tasks in any VR application with 3D audio and a mute option for background music. Therefore, DHH persons can use all VR applications with 3D audio, not only those applications designed for them. Our user study shows that DHH participants were able to complete a simple VR task significantly faster with EarVR than without. The completion time of DHH participants was very close to participants without hearing problems. Also, it shows that DHH participants were able to finish a complex VR task with EarVR, while without it, they could not finish the task even once. Finally, our qualitative and quantitative evaluation among DHH participants indicates that they preferred to use EarVR and it encouraged them to use VR technology more.

This paper investigates whether the body anticipation synergies in real environments (REs) are preserved during navigation in virtual environments (VEs). Experimental studies related to the control of human locomotion in REs during curved trajectories report a top-down reorientation strategy with the reorientation of the gaze anticipating the reorientation of head, the shoulders and finally the global body motion. This anticipation behavior provides a stable reference frame to the walker to control and reorient his/her body according to the future walking direction. To assess body anticipation during navigation in VEs, we conducted an experiment where participants, wearing a head-mounted display, performed a lemniscate trajectory in a virtual environment (VE) using five different navigation techniques, including walking, virtual steering (head, hand or torso steering) and passive navigation. For the purpose of this experiment, we designed a new control law based on the power-law relation between speed and curvature during human walking. Taken together our results showed a similar ordered top-down sequence of reorientation of the gaze, head and shoulders during curved trajectories between walking in REs and in VEs (for all the evaluated techniques). However, the anticipation mechanism was significantly higher for the walking condition compared to the others. The results presented in this paper pave the way to the better understanding of the underlying mechanisms of human navigation in VEs and to the design of navigation techniques more adapted to humans.

Visual information plays an important part in the perception of the world around us. Recently, head-mounted displays (HMD) came to the consumer market and became a part of everyday life of thousands of people. Like with the desktop screens or hand-held devices before, the public is concerned with the possible health consequences of the prolonged usage and question the adequacy of the default settings. It has been shown that the brightness and contrast of a display should be adjusted to match the external light to decrease eye strain and other symptoms. Currently, there is a noticeable mismatch in brightness between the screen and dark background of an HMD that might cause eye strain, insomnia, and other unpleasant symptoms. In this paper, we explore the possibility to significantly lower the screen brightness in the HMD and successfully compensate for the loss of the visual information on a dimmed screen. We designed a user study to explore the connection between the screen brightness in HMD and task performance, cybersickness, users' comfort, and preferences. We have tested three levels of brightness: the default Full Brightness, the optional Night Mode and a significantly lower brightness with original content and compensated content. Our results suggest that although users still prefer the brighter setting, the HMDs can be successfully used with significantly lower screen brightness, especially if the low screen brightness is compensated.

In this paper, we follow up on research dealing with motion learning in Virtual Reality (VR). We investigate the impact of VR motion learning on motion performance, motivation for motion learning and willingness to continue with the motion learning. In our research, we used three ball juggling as a subject of learning. We performed a user study with 30 participants. A VR application was used in our study which allows setting up lower gravity and thus slowing down the motion for learning purposes. The results were statistically evaluated and we comment on the positive influence of virtual reality on motivation and possibilities of using VR in the motion learning process.

This paper presents a novel method for illumination estimation from RGB-D images. The main focus of the proposed method is to enhance visual coherence in augmented reality applications by providing accurate and temporally coherent estimates of real illumination. For this purpose, we designed and trained a deep neural network which calculates a dominant light direction from a single RGB-D image. Additionally, we propose a novel method for real-time outlier detection to achieve temporally coherent estimates. Our method for light source estimation in augmented reality was evaluated on the set of real scenes. Our results demonstrate that the neural network can successfully estimate light sources even in scenes which were not seen by the network during training. Moreover, we compared our results with illumination estimates calculated by the state-of-the-art method for illumination estimation. Finally, we demonstrate the applicability of our method on numerous augmented reality scenes.

In walkable multi-user HMD-based VR immersed users have to avoid colliding with each other while performing tasks or exploring the virtual environment. This paper presents an experiment in which we compare mutual collision avoidance behaviour of pairs of users walking in the matched real and virtual environments. To generalize our study, we investigate two types of multi-user VR setup: one in which both users share a common tracking space (colocated condition) and one where they move in two separate tracking spaces while sharing a common virtual space (distributed condition). Our results indicated significant differences in locomotor trajectories of participants between the real and virtual conditions. Precisely, we observed an increase in the clearance distance and the median path curvature and a decrease in the walking speed when participants performed the test in VR in the colocated condition. Moreover, we found effects of pre-conditioning between the colocated and distributed scenarios.

In this paper, we present a novel method for fast generation of furniture arrangements in interior scenes. Our method exploits the benefits of optimization-based approaches for global aesthetic rules and the advantages of procedural approaches for local arrangement of small objects. We generate the furniture arrangements for a given room in two steps: We first optimize the selection and arrangement of furniture objects in a room with respect to aesthetic and functional rules. The infinite trans-dimensional space of furniture layouts is rapidly explored by greedy cost minimization. In the second step, the procedural methods are locally applied in a stochastic fashion to generate important scene details. We demonstrate that our method achieves comparable results to a recent method for automatic interior design in terms of user preferences and that local procedural design enhances the result of optimization-based interior design. Additionally, our method is one order of magnitude faster than the compared method. Finally, the execution times of up to one second show that our method is suitable for generating large-scale indoor virtual environments during runtime.

The purpose of the experiment presented in this paper is to investigate co-presence and locomotory patterns in a walkable shared virtual environment. In particular, trajectories of users that use a walkable tracking space alone are compared to those of users who use the tracking space in pairs. Co-presence, in a sense of perception of another person being present in the same virtual space is analyzed through subjective responses and behavioral markers. The results indicate that both perception and proxemics in relation to co-located and distributed players differ. The effect on the perception is however mitigated if participants do not collide with the avatars of distributed co-players.

We present the VROnSite platform that enables immersive training of first responder on-site squad leaders. Our training platform is fully immersive, entirely untethered to ease use and provides two means of navigation - abstract and natural walking - to simulate stress and exhaustion, two important factors for decision making. With the platform's capabilities, we close a gap in prior art for first responder training. Our research is closely interlocked with stakeholders from fire brigades and paramedics to gather early feedback in an iterative design process. In this paper, we present our first research results, which are the system's design rationale, the single user training prototype and results from a preliminary user study.

We present the design and development of a fully immersive virtual reality (VR) system that can provide prop-based haptic feedback in an infinite virtual environment. It is conceived as a research tool for studying topics related to haptics in VR and based on off-the-shelf components. A robotic arm moves physical props, dynamically matching pose and location of an object in the virtual world. When the user reaches for the virtual object, his or her hands also encounter it in the real physical space. The interaction is not limited to specific body parts and does not rely on an external structure like an exoskeleton. In combination with a locomotion platform for close-to-natural walking, this allows unrestricted haptic interaction in a natural way in virtual environments of unlimited size. We describe the concept, the hardware and software architecture in detail. We establish safety design guidelines for human-robot interaction in VR. Our technical evaluation shows good response times and accuracy. We report on a user study conducted with 34 participants indicating promising results, and discuss the potential of our system.

In this paper, we present a system that automatically populates indoor virtual scenes with furniture objects and optimizes their positions and orientations with respect to aesthetic, ergonomic and functional rules called interior design guidelines. These guidelines are represented as mathematical expressions which form the cost function. Our system optimizes the set of multiple interior designs by minimizing the cost function using a genetic algorithm. Moreover, we extend the optimization to transdimensional space by enabling automatic selection of furniture objects. Finally, we optimize the assignment of materials to the furniture objects to achieve a unified design and harmonious color distribution. We investigate the capability of our system to generate sensible and livable interior designs in a perceptual study.

Especially for young children measuring their physiological parameters to assess their health can be stressful, even when conducted at home by their parents. Therefore we present a concept that can relieve some of the anxiety correlated with an examination and implemented it in a test setup we call "MediCubes" to investigate how this approach is received. In this system cube shaped tangible objects are fitted with noninvasive sensors measuring pulse, temperature, blood oxygen saturation and lung capacity while interacting with them. Incorporation in a storytelling game allows guiding a child through a series of unperceived physiological measurements as an enjoyable experience. The acquired data is stored on a tablet computer and can be reviewed by parents or doctors. In this paper the design process and the developed hard- and software are presented. Furthermore we report on a usability study with 8 children and 12 adults indicating high acceptance and enjoyment of the system. These results as well as our "lessons learned" could have implications on the future development of home health monitoring toys.

Spatialized audio is the only output that players receive in audio games. In order to provide a realistic view of the environment, it has to be of superior quality in terms of immersion and realism. Complex sound models can be used to generate realistic sound effects, including reflections and reverb. An implementation of a hybrid sound model based on the ODEON approach is introduced and adapted for real-time sound calculations. This model is evaluated and compared to a baseline model usually used in audio games in a user study in a virtual reality environment. The results show that the implemented hybrid model allows players to adjust to the game faster and provides them more support in avoiding virtual obstacles in simple room geometries than the baseline model.

Fast and realistic synthesis of real videos with computer generated content has been a challenging problem in computer graphics. It involves computationally expensive light transport calculations. We present a novel and efficient algorithm for diffuse light transport calculation between virtual and real worlds called Differential Irradiance Caching. Our algorithm produces a high-quality result while preserving interactivity and allowing dynamic geometry, materials, lighting, and camera movement. The problem of expensive differential irradiance evaluation is solved by exploiting the spatial coherence in indirect illumination using irradiance caching. We enable multiple bounces of global illumination by using Monte Carlo integration in GPU ray-tracing to evaluate differential irradiance at irradiance cache records in one pass. The combination of ray-tracing and rasterization is used in an extended irradiance cache splatting algorithm to provide a fast GPU-based solution of indirect illumination. Limited information stored in the irradiance splat buffer causes errors for pixels on edges in case of depth of field rendering. We propose a solution to this problem using a reprojection technique to access the irradiance splat buffer. A novel cache miss detection technique is introduced which allows for a linear irradiance cache data structure. We demonstrate the integration of differential irradiance caching into a rendering framework for Mixed Reality applications capable of simulating complex global illumination effects.

In this paper, we present a low-weight and low-cost Unmanned Aerial Vehicle (UAV) for autonomous flight and navigation in GPS-denied environments using an off-the-shelf smartphone as its core on-board processing unit. Thereby, our approach is independent from additional ground hardware and the UAV core unit can be easily replaced with more powerful hardware that simplifies setup updates as well as maintenance. The UAV is able to map, locate and navigate in an unknown indoor environment fusing vision based tracking with inertial and attitude measurements. We choose an algorithmic approach for mapping and localization that does not require GPS coverage of the target area, therefore autonomous indoor navigation is made possible. We demonstrate the UAVs capabilities of mapping, localization and navigation in an unknown 2D marker environment. Our promising results enable future research on 3D self-localization and dense mapping using mobile hardware as the only on-board processing unit.

Precise localization for mobile Augmented Reality in large indoor environments without specific tracking infrastructure is challenging. This is especially true for rooms with changing properties, like lighting, seating and carpeting. With these constraints a map for a vision based tracking approach has to be continuously updated. The Parallel Tracking and Mapping (PTAM) algorithm is capable of generating and extending a map while tracking the camera pose in an unknown environment. However, it has originally been designed for small workspace environments and has therefore certain limitations. We have extended and modified the original implementation in order to ensure efficient and robust map generation and tracking in large rooms. Furthermore, we have tested a mobile setup with the system in Festsaal in Vienna's Hofburg, which is close to thousand square meters in size. The user's position and path was tracked while the environment was augmented with virtual objects and the system was successfully tested for robustness and occlusions.

Fire fighting remains a dangerous profession despite many recent technological and organizational measures. Sensors and technical systems can augment the performance of fire fighters and increase safety and efficiency during operation. An important aspect in that context is the awareness of location, structure and thermal properties of the environment. This work focuses on the design and development of a mobile system, which can reconstruct a 3d model of a building's interior structure in real-time and fuses the visualization with the image of a thermal camera. In addition, the position and viewing direction of the fire fighter within the model is determined and a thermal map can be generated from the gathered data. This helps an operational commander to provide accurate instructions to his men during a mission. First tests with our system in different situations showed good results, being able to reconstruct different larger scenes and create thermal maps thereof.

In this paper we introduce the concept of flexible spaces - a novel redirection technique that generalizes the use of overlapping (impossible) spaces and change blindness in an algorithm for dynamic layout generation. Flexible spaces is an impossible environment that violates the real world constancy in favor of providing the experience of seamless, unrestricted natural walking over a large-scale virtual environment (VE).

The ability to guide human motion through automatically generated feedback has significant potential for applications in areas, such as motor learning, human-computer interaction, telepresence, and augmented reality.

This paper presents an evaluation of spatio-temporal data generated by a dynamic stereo vision sensor in a highdimensional space (3D volume and time) for motion analysis and gesture recognition. In contrast to traditional frame-based (synchronous) stereo cameras, dynamic stereo vision sensors asynchronously generates events upon scene dynamics. Motion activities are intrinsically (on-chip) segmented by the sensor, such that activity, gesture recognition and tracking can be intuitively and efficiently performed. In this work, we investigated the applicability of this sensor for gesture recognition. We developed a machine learning method based on the Hidden Markow Model for training and automated classifications of gestures using the event data generated by the sensor. By training eight different activities (dance figures) with 15 persons we build up a library of 580 recorded activities. An average recognition rate of 97% has been reached.

In this paper we present a novel high-quality rendering system for Augmented Reality (AR). We study ray-tracing based rendering techniques in AR with the goal of achieving real-time performance and improving visual quality as well as visual coherence between real and virtual objects in a final composited image. A number of realistic and physically correct rendering effects are demonstrated, that have not been presented in real-time AR environments before. Examples are high-quality specular effects such as caustics, refraction, reflection, together with a depth of field effect and anti-aliasing. We present a new GPU implementation of photon mapping and its application for the calculation of caustics in environments where real and virtual objects are combined. The composited image is produced on-the-fly without the need of any preprocessing step. A main contribution of our work is the achievement of interactive rendering speed for high-quality ray-tracing algorithms in AR setups. Finally we performed an evaluation to study how users perceive visual quality and visual coherence with different realistic rendering effects. The results of our user study show that in 40.1% cases users mistakenly judged virtual objects as real ones. Moreover we show that high-quality rendering positively affects the perceived visual coherence.

The standard method for estimating the rigid-body motion of arbitrary interaction devices with an infrared-optical tracking system involves attaching pre-defined geometric constellations of retro-reflective or light-emitting markers, commonly referred to as "targets", to all tracked objects. Optical markers of the same type are typically indistinguishable from each other, requiring the tracking system to establish their identities through known spatial relationships. Consequently, the specific geometric arrangement of markers across multiple targets has a considerable impact on the system's overall performance and robustness.

This paper presents Construct3D, an application for geometry educationthat has been developed at the VR group at Vienna University of Technology throughout the past 6 years. Specific design aspects to allow distributed collaboration in Construct3D are presented. This concerns improvements of the user interface, usability and system performance including bandwidth efficiency.

Our work introduces immersive collaborative learning to geometry education. More specifically, we present a system that uses collaborative augmented reality as a medium for teaching, and uses 3D dynamic geometry to facilitate mathematics and geometry education. Both these aspects are novel to geometry education. We describe improvements in the user interface and visual design of such an application. We also report on practical experiences with using our system for actual teaching of high school students, and present initial quantitative data on the educational value of such an approach.

Published in: IEEE Proceedings. VR 2005. Virtual Reality, 2005

Construct3D is a three dimensional geometry construction tool specifically designed for mathematics and geometry education. It is based on the mobile collaborative augmented reality system "Studierstube". We describe our efforts in developing a system for the improvement of spatial abilities and maximization of transfer of learning. Means of application and integration in mathematics and geometry education at high school as well as university level are being discussed. Anecdotal evidence supports our claim that Construct3D is easy to learn, encourages experimentation with geometric constructions and improves spatial skills.