Navigation is a fundamental task supporting guided exploration and wayfinding as standalone or as part of other immersive applications. Previous research showed that navigational cues do not only impact wayfinding performance but can also affect perceptual and cognitive processes, e.g. divide attention and impair spatial memory. This study investigates whether varying the density of visual navigation cues can influence situational awareness. Additionally, we examine how cue density affects navigation usability and task performance. We compare three visual cue designs, ranging from high to low density: (1) PathLine, (2) ArrowTrail and (3) TurnMarker. We designed a user study augmenting a 3D scanned digital twin of a building with virtual machinery to simulate a factory floor maintenance task, where the cues guided participants to the next point of interest. A secondary task, the reporting of anomalies, was installed to assess situational awareness. Results showed that performance metrics remained unaffected by cue type, situational awareness and user experience results showed significant differences. Notably, the ArrowTrail cue, the medium-density design, was preferred by most participants and yielded the best overall results, e.g. in terms of anomaly detection and reaction time. These findings suggest that moderate cue density may offer an optimal balance between effective guidance and maintaining environmental awareness.

This paper introduces the Simultaneous Presence (SP) Continuum, a novel concept designed to enhance the use of portals in virtual reality (VR) by understanding users' experiences across multiple environments. Portals traditionally provide access to secondary worlds, but their limited field of view (FoV) can hinder full engagement with these environments. To overcome this, we introduce portal overlays, which expand the portal's FoV by superimposing the secondary world over the user's view of the primary world. We explore several overlay techniques-Contours, Blended (Opacity and Stencil), and Absolute-to adjust user experience across the SP Continuum. The Contours overlay, renders only the edges of objects, offering a minimalistic view and immersion of the secondary world. The Blended overlay allows for adjustable immersion between worlds through opacity or the distribution of pixels. The Absolute overlay virtually immerses users completely in the secondary world. Importantly, these overlays are context-activated instead of always on, to suit situational needs. Our study revealed high SP was possible in both worlds, and overlays significantly impacted users' experiences on the SP Continuum. The Blended overlay provided balanced SP, while Contours had the highest primary presence but lower secondary presence. In contrast, the Absolute overlay, alternating full immersion between worlds, had the highest secondary presence but resulted in reduced primary presence, slower navigation and selection times, higher effort and frustration, and lower usability.

Large transparent touch screens (LTTS) have recently become commercially available. These displays have the potential for engaging Augmented Reality (AR) applications, especially in public and shared spaces. However, the interaction with objects in the real environment behind the display remains challenging: Users must combine pointing and touch input if they want to select objects at varying distances. There is a lot of work on wearable or mobile AR displays, but little on how users interact with LTTS. Our goal is to contribute to a better understanding of natural user interaction for these AR displays. To this end, we developed a prototype and evaluated different pointing techniques for selecting 12 physical targets behind an LTTS, with distances ranging from 6 to 401 cm. We conducted a user study with 16 participants and measured user preferences, performance, and behavior. We analyzed the change in accuracy depending on the target position and the selection technique used. Our findings include: (a) Users naturally align the touch point with their line of sight for targets farther than 36 cm behind the LTTS. (b) This technique provides the lowest angular deviation compared to other techniques. (c) Some user close one eye to improve their performance. Our results help to improve future AR scenarios using LTTS systems.

This paper introduces foundational techniques for Adaptive Portals (AP); portals that adjust to users’ needs in Virtual Reality (VR). At the forefront of our contributions is the AP-Reach technique, with Entry and Exit strategies, designed to overcome the spatial limitations of portals and VR, enabling users to comfortably extend their reach to objects beyond their immediate vicinity. In parallel, we present PH-Reach as an alternative to the AP concept, also aimed at extending user reach inside portals. The effectiveness of these reach techniques is evaluated through a comparative user study, which compared Entry AP-Reach, Exit AP-Reach, and PHReach techniquesagainst traditional natural reach as a baseline. Results show Exit AP-Reach surpassed both its counterparts and the baseline to various degrees in terms of reducing task completion time, physical movement/demand, effort, and frustration. Additionally, this paper introduces AP-Bounds and AP-Restriction.

This research explores various Augmented Reality (AR) annotations for supporting performance and decision-making in a time-critical task. The task was designed in collaboration with defence contacts to mimic a command and control scenario, requiring rapid decisions under time pressure. A set of annotations was derived from previous literature, and modifications were made to suit the tasking requirements. The annotations chosen were based on their level of abstraction from least to most abstract: Animations, StickyNotes, Icons, and Highlights. The study compared five conditions (Animations, StickyNotes, Icons, Highlights, and no annotations) for their effectiveness in supporting performance and decision-making in a time-critical task. StickyNotes was found to be the most effective for supporting performance. The study’s results also identify that annotations for assisting with time-critical tasking should be easy and quick to understand, contain sufficient complexity, and not be too abstract in detail.

Virtual Reality (VR) is changing how we interact with complex spatial datasets. VR has the potential within forestry to help analyse remotely sensed spatial data for inventory management. However, challenges of scale and legibility of dense forest point clouds must be addressed. To do so, we present three VR Focus+Context (F+C) techniques: Wedgelight, Remote Lantern, and Ray Lantern. These techniques present point cloud data using a constrained highdetail focus within a broader abstracted context representation, manipulated using egocentric and exocentric interactions. We conducted two user studies. The first informed the design of the F+C techniques. The results from the first study indicate that VR significantly outperforms desktop displays in basic analysis of forestry point clouds, with improvements to both time and accuracy. The second study evaluated the performance of the resulting F+C techniques in a task analogous to real-world forestry operations. Our F+C techniques significantly outperformed current approaches, confirming their viability for facilitating detailed and effective forest assessments. These findings indicate that VR has significant advantages for spatial data analysis tasks in forestry, making a strong argument for its adoption in future analytical frameworks. The FF+CC visualisation techniques we introduced show considerable promise for analysis of high density point clouds, opening new avenues for development in immersive analytics. Supplemental materials (code, scans, results) are available at https://osf.io/hqc9u.

Parametric Design enables designers to formulate and explore new ideas through parameters, typically by manipulating numerical values. However, visualising and exploring the design space of an established parametric design solution is natively difficult through desktop displays for designers due to screen space constraints and requiring familiarity with visual-language programming interfaces. Thus, we sought to explore Virtual Reality (VR), inspired by Natural User Interfaces (NUI), to develop and explore new interfaces departing from traditional programming interfaces, that could complement the spatial and embodied affordances of contemporary VR devices. Informed by two industry-led focus groups with architects we developed and examined the usability of three different interfaces: 1) Paramaxes , an axes-based interface that allows designers to distribute and manipulate parameter visualisations around them in physical space; 2) ParamUtter , a Voice-based User Interface (VUI) that allows designers to manipulate parameter visualisations through natural languages; 3) Control Panel , which presents the parameters as sliders in a scrollable pane and acts as baseline comparison. We ran an exploratory study with experts and found that the Control Panel was ultimately the preferred interface for a design manipulation task. However, participants commented favorably towards qualities in the unconventional interfaces, with ParamUtter scoring highest in System Usability Scores (SUS), and participants valuing the potential of using physical space to explore design spaces with Paramaxes .

User interfaces in virtual reality enable diverse interactions within the virtual world, though they typically lack the haptic cues provided by physical interface controls. Haptic retargeting enables flexible mapping between dynamic virtual interfaces and physical controls to provide real haptic feedback. This investigation aims to extend these remapped interfaces to support more diverse control types. Many interfaces incorporate sliders, switches, and knobs. These controls hold fixed states between interactions creating potential conflicts where a virtual control has a different state from the physical control. This paper presents two methods, “manual” and “automatic”, for synchronizing physical and virtual control states and explores the effects of these methods on the usability of remapped interfaces. Results showed that interfaces without retargeting were the ideal configuration, but they lack the flexibility that remapped interfaces provide. Automatic synchronization was faster and more usable; however, manual synchronization is suitable for a broader range of physical interfaces.

Existing situated visualization models assume the user is able to directly interact with the objects and spaces to which the data refers (known as physical referents). We review a growing body of work exploring scenarios where the user interacts with a proxy representation of the physical referent rather than immediately with the object itself. This introduces a complex mixture of immediate situatedness and proxies of situatedness that goes beyond the expressiveness of current models. We propose an extended model of situated visualization that encompasses Immediate Situated Visualization and ProxSituated (Proxy of Situated) Visualization. Our model describes a set of key entities involved in proxSituated scenarios and important relationships between them. From this model, we derive design dimensions and apply them to existing situated visualization work. The resulting design space allows us to describe and evaluate existing scenarios, as well as to creatively generate new conceptual scenarios.

This paper introduces Point & Portal, a novel Virtual Reality (VR) interaction technique, inspired by Point & Teleport. This new technique enables users to configure portals using pointing actions, and supports seamless action at a distance and navigation without requiring line of sight. By supporting multiple portals, Point & Portalenables users to create dynamic portal configurations to manage multiple remote tasks. Additionally, this paper introduces Relative Portal Positioning for reliable portal interactions, and the concept of maintaining Level Portals. In a comparative user study, Point & Portal demonstrated significant advantages over the traditional Point & Teleport technique to bring interaction devices within-arm’s reach. In the presence of obstacles, Point & Portal exhibited faster speed, lower cognitive load and was preferred by participants. Overall, participants required less physical movement, pointing actions, and reported higher involvement and “good” usability.

This article presents a set of adaptive reset techniques for use with haptic retargeting systems focusing on interaction with hybrid virtual reality interfaces that align with a physical interface. Haptic retargeting between changing physical and virtual targets requires a reset where the physical and virtual hand positions are re-aligned. We present a modified Point technique to guide the user in the direction of their next interaction such that the remaining distance to the target is minimized upon completion of the reset. This, along with techniques drawn from existing work are further modified to consider the angular and translational gain of each redirection and identify the optimal position for the reset to take place. When the angular and translational gain is within an acceptable range, the reset can be entirely omitted. This enables continuous retargeting between targets removing interruptions from a sequence of retargeted interactions. These techniques were evaluated in a user study which showed that adaptive reset techniques can provide a significant decrease in task completion time, travel distance, and the number of user errors.

This article presents the results from a Spatial Augmented Reality (SAR) study which evaluated the cognitive cost of several predictive cues. Participants performed a validated procedural button pressing task, where the predictive cue annotations guided them to the upcoming task. While existing research has evaluated predictive cues based on their performance and self-rated mental effort, actual cognitive cost has yet to be investigated. To measure the user's brain activity, this study utilized electroencephalogram (EEG) recordings. Cognitive load was evaluated by measuring brain responses for a secondary auditory oddball task, with reduced brain responses to oddball tones expected when cognitive load in the primary task is highest. A simple monitor n-back task and procedural task comparing monitor versus SAR were conducted, followed by a version of the procedural task comparing the SAR predictive cues. Results from the brain responses were able to distinguish between performance enhancing cues with a high and low cognitive load. Electrical brain responses also revealed that having an arc or arrow guide towards the upcoming task required the least amount of mental effort.

We explore Spatial Augmented Reality (SAR) precues (predictive cues) for procedural tasks within and between workspaces and for visualizing multiple upcoming steps in advance. We designed precues based on several factors: cue type, color transparency, and multi-level (number of precues). Precues were evaluated in a procedural task requiring the user to press buttons in three surrounding workspaces. Participants performed fastest in conditions where tasks were linked with line cues with different levels of color transparency. Precue performance was also affected by whether the next task was in the same workspace or a different one.

Transformative portals are a novel technique for supporting visualizations and interactions between locations that are not co-located in virtual reality (VR). We are interested in improving upon current portal rendering for VR, and supporting VR users to create new portals quickly and accurately. In this paper, we introduce a new high-level algorithm for rendering transformative portals in VR and present the benefits of our approach. By leveraging single-pass stereo and stencil portal rendering, we developed our algorithm to support multiple portals with “infinite” recursion, while running performant on mobile VR devices. We then benchmarked our performance against other common portal rendering implementations. In addition, we focused our research on VR interactions for handheld portals, in which the initial placement of the portal’s destination is imperative to its effectiveness. In this paper, we present four new portal creation interactions for handheld portals: fishing reel, raycast with fishing reel, marker, and projectile curve. Based on existing research, we also established standard controls for fine-grained portal manipulation. In a user study, we compared and evaluated how well the creation interactions performed and argue that projectile curve is the most suitable general-purpose technique.

Recent research in immersive visualisations has explored the use of physical 3D models together with virtual data visualisations, with a simple data encoding. However, little is known about how this technique could be extended towards more complex multivariate data with multiple charts. We present augmented scale models, immersive visualisations that place charts of multivariate data via Augmented Reality (AR) registered to physical 3D models. We identified two main factors for presenting AR charts in the limited display space around the models: 1) how charts are laid out (Slides vs Dashboard), and 2) how the chart views are arranged in the 3D space (On Scale Model, On Table, On Virtual Board). In a within-subject user study, we evaluated these two design considerations. We found that chart layout and view arrangement do not affect task error but do vary in response time. Dashboard and Slides perform equally well on simple tasks that require comparison of a single chart across scale models. However, in more complex tasks such as comparing multiple charts within a scale model or across several scale models, Dashboard shows no decrease in time performance, while Slides’s time performance decreases significantly. We also found that On Scale Model has the fastest performance, has good chart-scale model integration, and supports charts comparison well. On Table and On Virtual Board on the other hand, show a trade-off between the ability to support charts comparison across models and the chart-scale model integration.

This work explores the interaction between Augmented Reality (AR) and eye accommodation for airborne surveillance by simulating AR environments in Virtual Reality (VR). We simulate the AR display as displays with the capabilities needed for airborne surveillance are limited and because it would be hazardous to experiment directly on surveillance aircraft. While there is precedent for simulating AR in a VR environment, our study account for two of the physical and physiological aspects of AR: we factor in the focal plane of the AR technology and simulate the eye accommodation reflex of the user to provide focus. We ran a study with 24 participants examining AR cues to support visual search. We also compare the effects of having secondary tasks (that surveillance operators are normally responsible for) directly on the observation window using AR. Our results show that the effectiveness of the AR cues is dependent on the modality of the secondary task. We also found that, under certain situations, operators’ performances for the search task are improved if the focal plane of the AR display is at the same distance as subsequent search targets.

This paper presents Shape Aware Haptic Retargeting, an extension of "state-of-the-art" haptic retargeting that is the first to support retargeted interaction between any part of the user’s hand and any part of the target object. In previous haptic retargeting algorithms, the maximum retargeting is applied only when the hand position aligns with the target position. Shape Aware Haptic Retargeting generalizes the distance computation process to instead consider the hand and target geometry. The shortest hand-target distance is then used to calculate the applied retargeting offset. This ensures the full amount of haptic retargeting is applied at the point of contact with the passive haptic regardless of contact position on the hand or target. We leverage existing geometry algorithms to implement three distance computation methods: Multi-Point, Primitive and Mesh Geometry, in addition to conventional single position approaches. These are evaluated through a set of simulated interactions instead of the single position representation used in previous haptic retargeting systems. The evaluation demonstrated all three approaches can provide improved interaction accuracy over a Point distance computation method, with Mesh Geometry being the most accurate and Primitive being the preferred method for combined performance and interaction accuracy.

This work investigates the use of Virtual Reality (VR) to present forensic evidence to the jury in a courtroom trial. The findings of a between-participant user study on comprehension of an expert statement are presented, examining the benefits and issues of using VR compared to traditional courtroom presentation (being still images). Participants listened to a forensic scientist explain bloodstain spatter patterns while viewing a mock crime scene in either VR or as still images in video format. Under these conditions, we compared understanding of the expert domain, mental effort and content recall. We found that VR significantly improves the understanding of spatial information and knowledge acquisition. We also identify different patterns of user behaviour depending on the display method. We conclude with suggestions on how to best adapt evidence presentation to VR.

Spatial Augmented Reality (SAR) is a useful tool for procedural tasks as virtual instructions can be co-located with the physical task. Predictive cue annotations have been utilized to further enhance user performance through SAR. These predictive cues have only been measured under ordinary circumstances. This paper aims to investigate predictive cues under a sub-optimal scenario by depriving users of sleep. Sleep deprivation is a common form of fatigue, which is known to have serious detriments on user performance. Based upon existing sleep literature, we expected predictive cue performance to degrade over time with sharp declines during the early hours of the morning. Despite these drops in performance, we hypothesized that having a predictive cue would benefit user performance in comparison to no cue. Results from a 62-hour sleep deprivation experiment indicated that providing SAR predictive cues was beneficial throughout sleep deprivation. Furthermore, having no predictive cue caused accuracy to decline earlier in the sleep deprivation period. From the predictive cues outlined in this paper, the line cue maintained the fastest response time and was least impacted by early morning performance declines.

When collaborating face-to-face, people commonly use the surfaces and spaces around them to perform sensemaking tasks, such as spatially organising documents, notes or images. However, when people collaborate remotely using desktop interfaces they no longer feel like they are sharing the same space. This limitation may be overcome through collaboration in immersive environments, which simulate the physical in-person experience. In this paper, we report on a between-groups study comparing collaborations on image organisation tasks, in an immersive Virtual Reality (VR) environment to more conventional desktop conferencing. Collecting data from 40 subjects in groups of four, we measured task performance, user behaviours, collaboration engagement and awareness. Overall, the VR and desktop interface resulted in similar speed, accuracy and social presence rating, but we observed more conversations and interaction with objects, and more equal contributions to the interaction from participants within groups in VR. We also identified differences in coordination and collaborative awareness behaviours between VR and desktop platforms. We report on a set of systematic measures for assessing VR collaborative experience and a new analysis tool that we have developed to capture user behaviours in collaborative setting. Finally, we provide design considerations and directions for future work.

Data physicalisations afford people the ability to directly interact with data using their hands, potentially achieving a more comprehensive understanding of a dataset. Due to their complex nature, the representation of graphs and networks could benefit from physicalisation, bringing the dataset from the digital world into the physical one. However, no empirical work exists investigating the effects physicalisations have upon comprehension as they relate to graph representations. In this work, we present initial design considerations for graph physicalisations, as well as an empirical study investigating differences in comprehension between virtual and physical representations. We found that participants perceived themselves as being more accurate via touch and sight (visual-haptic) than the graphical-only modality, and perceived a triangle count task as less difficult in visual-haptic than in the graphical-only modality. Additionally, we found that participants significantly preferred interacting with visual-haptic over other conditions, despite no significant effect on task time or error.

This paper investigates the use of immersive virtual reconstructions as an aid for jurors during a courtroom trial. The findings of a between-participant user study on memory and decision-making are presented in the context of viewing a simulated hit-run-death scenario. Participants listened to the opening statement of a prosecutor and a defence attorney before viewing the crime scene in Virtual Reality (VR) or as still images. We compare the effects on cognition and usability of using VR over images presented on a screen. We found several significant improvements, including that VR led to more consistent decision-making among participants. This shows that VR could provide a promising solution for the court to present crime scenes when site visitations are not possible.

In augmented reality (AR), gesture/hand-based interactions are becoming more common place over tangible user interfaces (TUIs) and physical controllers. Major concerns regard portability and battery-life with TUIs and physical controllers in an AR environment. For a TUI or physical controller to be usable ”on the go” and in the field, they need to be compact such that they can be easily deployable. Subsequently, it is desirable to be low-energy or powerless, such that the device remains functional after long periods. In this paper, we present our initial design towards a powerless, passive controller that leverages the optical camera of a AR head-mounted display (such as the Hololens or Magic Leap) for tracking. We discuss related work, design motivations, high and low fidelity designs, opportunistic/adaptable input modalities, and use cases. We conclude with propositions for future work.

In this work, we explore if the immersion afforded by Virtual Reality can improve the cognitive integration of information in Cinematic Virtual Reality (CVR). We conducted a user study examining participants' cognitive activities (recall performance and cortical response) when consuming visual information of emotional and emotionally neutral scenes in a non-CVR environment (i.e. a monitor) versus a CVR environment (i.e. a head-mounted display). Cortical response was recorded using electroencephalography. The results showed that participants had greater early visual attention with neutral emotions in a CVR environment, and showed higher overall alpha power in a CVR environment. The use of CVR did not significantly affect participants' recall performance.

Despite the technological advances in building design, visualizing 3D building layouts can be especially difficult for novice and expert users alike, who must take into account design constraints including line-of-sight and visibility. Using CADwalk, a commercial building design tool that utilizes floor-facing projectors to show 1:1 scale building plans, this work presents and evaluates two floor-based visual cues for assisting with evaluating line-of-sight and visibility. Additionally, we examine the impact of using virtual cameras looking from the inside-out (from user’s location to objects of interest) and outside-in (looking from an object of interest’s location back towards the user). Results show that floor-based cues led to participants more correctly rating visibility, despite taking longer to complete the task. This is an effective tradeoff, given the final outcome (the building design) where accuracy is paramount.

We present Embodied Axes, a controller which supports selection operations for 3D imagery and data visualisations in Augmented Reality. The device is an embodied representation of a 3D data space -- each of its three orthogonal arms corresponds to a data axis or domain specific frame of reference. Each axis is composed of a pair of tangible, actuated range sliders for precise data selection, and rotary encoding knobs for additional parameter tuning or menu navigation. The motor actuated sliders support alignment to positions of significant values within the data, or coordination with other input: e.g., mid-air gestures in the data space, touch gestures on the surface below the data, or another Embodied Axes device supporting multi-user scenarios. We conducted expert enquiries in medical imaging which provided formative feedback on domain tasks and refinements to the design. Additionally, a controlled user study was performed and found that the Embodied Axes was overall more accurate than conventional tracked controllers for selection tasks.

Remote assistance represents an important use case for mixed reality. With the rise of handheld and wearable devices, remote assistance has become practical in the wild. However, spontaneous provisioning of remote assistance requires an easy, fast and robust approach for capturing and sharing of unprepared environments. In this work, we make a case for utilizing interactive light fields for remote assistance. We demonstrate the advantages of object representation using light fields over conventional geometric reconstruction. Moreover, we introduce an interaction method for quickly annotating light fields in 3D space without requiring surface geometry to anchor annotations. We present results from a user study demonstrating the effectiveness of our interaction techniques, and we provide feedback on the usability of our overall system.

Immersive analytics turns the very space surrounding the user into a canvas for data analysis, supporting human cognitive abilities in myriad ways. We present the results of a design study, contextual inquiry, and longitudinal evaluation involving professional economists using a Virtual Reality (VR) system for multidimensional visualization to explore actual economic data. Results from our preregistered evaluation highlight the varied use of space depending on context (exploration vs. presentation), the organization of space to support work, and the impact of immersion on navigation and orientation in the 3D analysis space.

Data videos are a highly impactful method of communication and are becoming a prevalent medium for communicating information. While the majority of current research focuses on the cinematic aspects of data videos, very little is known about the narrative methodologies involved. This paper presents our insights derived from an initial exploration of this area. We present a taxonomy based on the analysis of 70 existing data videos examining their narrative and visual approaches. We propose that our taxonomy can be used to explain the characteristics or design of data videos. Applying this taxonomy, we present our observations, including the trend of popular technologies applied in current data videos, the under-utilization of promising methods, and highlight research opportunities in the field.

We propose a multi-scale Mixed Reality (MR) collaboration between the Giant, a local Augmented Reality user, and the Miniature, a remote Virtual Reality user, in Giant-Miniature Collaboration (GMC). The Miniature is immersed in a 360-video shared by the Giant who can physically manipulate the Miniature through a tangible interface, a combined 360-camera with a 6 DOF tracker. We implemented a prototype system as a proof of concept and conducted a user study (n=24) comprising of four parts comparing: A) two types of virtual representations, B) three levels of Miniature control, C) three levels of 360-video view dependencies, and D) four 360-camera placement positions on the Giant. The results show users prefer a shoulder mounted camera view, while a view frustum with a complimentary avatar is a good visualization for the Miniature virtual representation. From the results, we give design recommendations and demonstrate an example Giant-Miniature Interaction.

Three-dimensional scatterplots suffer from well-known perception and usability problems. In particular, overplotting and occlusion, mainly due to density and noise, prevent users from properly perceiving the data. Thanks to accurate head and hand tracking, immersive Virtual Reality (VR) setups provide new ways to interact and navigate with 3D scatterplots. VR also supports additional sensory modalities such as haptic feedback. Inspired by methods commonly used in Scientific Visualisation to visually explore volumes, we propose two techniques that leverage the immersive aspects of VR: first, a density-based haptic vibration technique (Scaptics) which provides feedback through the controller; and second, an adaptation of a cutting plane for 3D scatterplots (Highlight-Plane). We evaluated both techniques in a controlled study with two tasks involving density (finding high- and low-density areas). Overall, Scaptics was the most time-efficient and accurate technique, however, in some conditions, it was outperformed by Highlight-Plane.

Immersive analytics allows a large amount of data and complex structures to be concurrently investigated. We propose a collaborative analytics system that benefits from new advances in immersive technologies for collaborators working in the early stages of data exploration. We implemented a combination of Star Coordinates and Star Plot visualisation techniques to support the visualisation of multidimensional data and the encoding of datasets using simple and compact visual representations. To support data analytics tasks, we propose tools and interaction techniques for users to build decision trees for visualising and analysing data in a top-down method.

We introduce IATK, the Immersive Analytics Toolkit, a software package for Unity that allows interactive authoring and exploration of data visualisation in immersive environments. The design of IATK was informed by interdisciplinary expert-collaborations as well as visual analytics applications and iterative refinement over several years. IATK allows for easy assembly of visualisations through a grammar of graphics that a user can configure in a GUI-in addition to a dedicated visualisation API that supports the creation of novel immersive visualisation designs and interactions. IATK is designed with scalability in mind, allowing visualisation and fluid responsive interactions in the order of several million points at a usable frame rate. This paper outlines our design requirements, IATK's framework design and technical features, its user interface, as well as application examples.

This paper presents development on a conceptual method to remap supernumerary limbs using Virtual Reality (VR) as a platform for experimentation. Our VR system allows users to control a third arm through their own limbs such as their head, arms, and feet with the ability to switch between them. To realize and experiment with our remapping method, we used the Oculus Rift in conjunction with OptiTrack to track users in a room-scaled virtual environment. We present some initial findings from a small pilot study and conclude with suggestions for future work.

This paper proposes a novel interface for virtual reality in which physical interface components are mapped to multiple virtual counterparts using haptic retargeting illusions. This gives virtual reality interfaces the ability to have correct haptic sensations for many virtual buttons although in the physical space there is only one. This is a generic system that can be applied to areas including design, interaction tasks, product prototype development and interactive games in virtual reality. The system presented extends existing retargeting algorithms to support asymmetric bimanual interactions. A new warp technique, called interface warp, was developed to support remapped virtual reality user interfaces. Through an experimental user study, we explore the effects of bimanual retargeting and the interface warp technique on task response time, errors, presence, perceived manipulation compared to unimanual (single handed) retargeting and other existing warp techniques. The results demonstrated faster task response time and less errors for the interface warp technique and shows no significant effect of bimanual interactions.

This paper introduces GeoGate, an Augmented Reality tabletop system that extends the Space-Time Cube and utilizes a ring-shaped tangible user interface to explore correlations between entities in multiple location datasets. We demonstrate GeoGate in the context of the maritime domain, where operators seek to find geo-temporal associations between trajectories recorded from a global positioning system, and light data extracted from night time satellite images. GeoGate utilizes a tabletop system displaying a traditional 2D map in conjunction with a Microsoft Hololens to present a single view of the data with a novel Augmented Reality extension of the Space-Time Cube. To validate GeoGate, we present the results of a user study comparing GeoGate with the existing 2D approach used in a normal desktop environment. The outcomes of the user study show that GeoGate's approach reduces mistakes in the interpretation of the correlations between various datasets, while the qualitative results show that such a system is preferable for the majority of geo-temporal maritime tasks compared.

Robot assistance can reduce the user's workload of a task. However, the robot needs to be programmed or trained on how to assist the user. Virtual Reality (VR) can be used to train and validate the actions of the robot in a safer and cheaper environment. In this paper, we examine how a robotic arm can be trained using Coloured Petri Nets (CPN) and Partial Least Squares Regression (PLSR). Based upon these algorithms, we discuss the concept of using the user's acceleration and rotation as a sufficient means to train a robotic arm for a procedural task in VR. We present a work-in-progress system for training robotic limbs using VR as a cost effective and safe medium for experimentation. Additionally, we propose PLSR data that could be considered for training data analysis.

Spatial Augmented Reality (SAR) systems can be suitably combined with other existing Extended Reality (xR) technologies to support collaboration. In existing strategies, users unencumbered by a viewing technology, such as a tablet interface or a head-mounted display, must rely on the transmission of their collaborators' positioning through interpreting a first-person camera view. This design creates a seam between a user's experience of the augmented physical environment in SAR, and their collaborators' experience inside the virtual environment. To assist in development and evaluation of spatial cues to support spatial awareness in SAR environments, an egocentric spatial-communication taxonomy is presented given two determining dimensions, a cue's attachment (physical/virtual) and animation (local/world). We developed four egocentric cues which characterize the four independent dimensions of the matrix: arrow, path, glow, and radial, and a single exocentric world in miniature visualization. Our study shows that virtual attachment cues are preferred, providing the highest accuracy, highest performance when collaborators are occluded, and produce the least mental effort when used with a single virtual collaborator. For multiple collaborators however, the virtual attached, world animated radial cue produces significant increases in mental load and reductions in preference, demonstrating the impact of visual augmentation clutter. The single exocentric visualization produced higher levels of head movement, and poorer accuracy, however the novelty of the visualization produced positive qualitative results.

Presents the preface to the 2019 Virtual Reality Conference.

We present Mini-Me, an adaptive avatar for enhancing Mixed Reality (MR) remote collaboration between a local Augmented Reality (AR) user and a remote Virtual Reality (VR) user. The Mini-Me avatar represents the VR user's gaze direction and body gestures while it transforms in size and orientation to stay within the AR user's field of view. A user study was conducted to evaluate Mini-Me in two collaborative scenarios: an asymmetric remote expert in VR assisting a local worker in AR, and a symmetric collaboration in urban planning. We found that the presence of the Mini-Me significantly improved Social Presence and the overall experience of MR collaboration.

In this paper we present a floor-based user interface (UI) that allows multiple users to explore a spatial augmented reality (SAR) environment with both monoscopic and stereoscopic projections. Such environments are characterized by a low level of user instrumentation and the capability of providing a shared interaction space for multiple users. However, projector-based systems using stereoscopic display are usually single-user setups, since they can provide the correct perspective for only one tracked person. To address this problem, we developed a set of guidance cues, which are projected onto the floor in order to assist multiple users regarding (i) the interaction with the SAR system, (ii) the identification of regions of interest and ideal viewpoints, and (iii) the collaboration with each other. In a user study with 40 participants all cues were evaluated and a set of feedback elements, which are essential to guarantee an intuitive self-explaining interaction, was identified. The results of the study also indicate that the developed UI guides users to more favorable viewpoints and therefore is able to improve the experience in a multi-user SAR environment.

We are pleased to present the technical papers for IEEE VR 2018: the 25th IEEE Conference on Virtual Reality and 3D User Interfaces, held March 18-22, 2018 in Reutlingen, Germany. IEEE VR 2018 features two categories of submissions: (i) VR Journal Papers, and (ii) VR Conference Papers. Both categories have their own program committees, submission processes, and review processes. This year, 178 submissions were submitted to the Journal track from which 29 were accepted as articles to IEEE TVCG (16.3%). All of these will be presented at the IEEE VR 2018 conference, along with 6 additional papers in the VR area that were published in IEEE TVCG during the past year. Furthermore, 6 submission (3.4%) were recommended for a regular issue of TVCG with major revisions with reviewer continuity. Each of the papers in this special issue went through a rigorous two-round review process. All accepted journal papers are published in a special issue of IEEE Transactions on Visualization and Computer Graphics (TVCG). The Proceedings of the IEEE Conference on Virtual Reality and 3D User Interfaces contains all of the accepted conference papers and the poster abstracts.

The advancements in Mixed Reality (MR), Unmanned Aerial Vehicle, and multi-scale collaborative virtual environments have led to new interface opportunities for remote collaboration. This paper explores a novel concept of flying telepresence for multi-scale mixed reality remote collaboration. This work could enable remote collaboration at a larger scale such as building construction. We conducted a user study with three experiments. The first experiment compared two interfaces, static and dynamic IPD, on simulator sickness and body size perception. The second experiment tested the user perception of a virtual object size under three levels of IPD and movement gain manipulation with a fixed eye height in a virtual environment having reduced or rich visual cues. Our last experiment investigated the participant's body size perception for two levels of manipulation of the IPDs and heights using stereo video footage to simulate a flying telepresence experience. The studies found that manipulating IPDs and eye height influenced the user's size perception. We present our findings and share the recommendations for designing a multi-scale MR flying telepresence interface.

This paper showcases one way of how virtual reconstruction can be used in a courtroom. The results of a pilot study on narrative and spatial memory are presented in the context of viewing real and virtual copies of a simulated crime scene. Based on current court procedures, three different viewing options were compared: photographs, a real life visit, and a 3D virtual reconstruction of the scene viewed in a Virtual Reality headset. Participants were also given a written narrative that included the spatial locations of stolen goods and were measured on their ability to recall and understand these spatial relationships of those stolen items. The results suggest that Virtual Reality is more reliable for spatial memory compared to photographs and that Virtual Reality provides a compromise for when physical viewing of crime scenes are not possible. We conclude that Virtual Reality is a promising medium for the court.

Previous research has demonstrated that Augmented Reality can reduce a user's task response time and mental effort when completing a procedural task. This paper investigates techniques to improve user performance and reduce mental effort by providing projector-based Spatial Augmented Reality predictive cues for future responses. The objective of the two experiments conducted in this study was to isolate the performance and mental effort differences from several different annotation cueing techniques for simple (Experiment 1) and complex (Experiment 2) button-pressing tasks. Comporting with existing cognitive neuroscience literature on prediction, attentional orienting, and interference, we hypothesized that for both simple procedural tasks and complex search-based tasks, having a visual cue guiding to the next task's location would positively impact performance relative to a baseline, no-cue condition. Additionally, we predicted that direction-based cues would provide a more significant positive impact than target-based cues. The results indicated that providing a line to the next task was the most effective technique for improving the users' task time and mental effort in both the simple and complex tasks.

In this paper, we propose EarTouch, a new sensing technology for ear-based input for controlling applications by slightly pulling the ear and detecting the deformation by an enhanced earphone device. It is envisioned that EarTouch will enable control of applications such as music players, navigation systems, and calendars as an "eyes-free" interface. As for the operation of EarTouch, the shape deformation of the ear is measured by optical sensors. Deformation of the skin caused by touching the ear with the fingers is recognized by attaching optical sensors to the earphone and measuring the distance from the earphone to the skin inside the ear. EarTouch supports recognition of multiple gestures by applying a support vector machine (SVM). EarTouch was validated through a set of user studies.

We introduce ImAxes immersive system for exploring multivariate data using fluid, modeless interaction. The basic interface element is an embodied data axis. The user can manipulate these axes like physical objects in the immersive environment and combine them into sophisticated visualisations. The type of visualisation that appears depends on the proximity and relative orientation of the axes with respect to one another, which we describe with a formal grammar. This straight-forward composability leads to a number of emergent visualisations and interactions, which we review, and then demonstrate with a detailed multivariate data analysis use case.

This paper presents our new Virtual Reality (VR) interactive visualization techniques to assist users querying large image sets. The VR system allows users to query a set of images on four different filters such as locations and keywords. The goal is to investigate if a VR platform is preferred over a non-VR platform for viewing and querying large image sets. We employed an HTC Vive and a traditional desktop screen to represent VR and non-VR platforms. We found users preferred the VR platform over the traditional desktop screen.

In this paper, we propose a novel technology called "CheekInput" with a head-mounted display (HMD) that senses touch gestures by detecting skin deformation. We attached multiple photo-reflective sensors onto the bottom front frame of the HMD. Since these sensors measure the distance between the frame and cheeks, our system is able to detect the deformation of a cheek when the skin surface is touched by fingers. Our system uses a Support Vector Machine to determine the gestures: pushing face up and down, left and right. We combined these 4 directional gestures for each cheek to extend 16 possible gestures. To evaluate the accuracy of the gesture detection, we conducted a user study. The results revealed that CheekInput achieved 80.45 % recognition accuracy when gestures were made by touching both cheeks with both hands, and 74.58 % when by touching both cheeks with one hand.

This paper presents the results of two cognitive load studies comparing three augmented reality display technologies: spatial augmented reality, the optical see-through Microsoft HoloLens, and the video see-through Samsung Gear VR. In particular, the two experiments focused on isolating the cognitive load cost of receiving instructions for a button-pressing procedural task. The studies employed a self-assessment cognitive load methodology, as well as an additional dual-task cognitive load methodology. The results showed that spatial augmented reality led to increased performance and reduced cognitive load. Additionally, it was discovered that a limited field of view can introduce increased cognitive load requirements. The findings suggest that some of the inherent restrictions of head-mounted displays materialize as increased user cognitive load.

High-quality immersive display technologies are becoming mainstream with the release of head-mounted displays (HMDs) such as the Oculus Rift. These devices potentially represent an affordable alternative to the more traditional, centralised CAVE-style immersive environments. One driver for the development of CAVE-style immersive environments has been collaborative sense-making. Despite this, there has been little research on the effectiveness of collaborative visualisation in CAVE-style facilities, especially with respect to abstract data visualisation tasks. Indeed, very few studies have focused on the use of these displays to explore and analyse abstract data such as networks and there have been no formal user studies investigating collaborative visualisation of abstract data in immersive environments. In this paper we present the results of the first such study. It explores the relative merits of HMD and CAVE-style immersive environments for collaborative analysis of network connectivity, a common and important task involving abstract data. We find significant differences between the two conditions in task completion time and the physical movements of the participants within the space: participants using the HMD were faster while the CAVE2 condition introduced an asymmetry in movement between collaborators. Otherwise, affordances for collaborative data analysis offered by the low-cost HMD condition were not found to be different for accuracy and communication with the CAVE2. These results are notable, given that the latest HMDs will soon be accessible (in terms of cost and potentially ubiquity) to a massive audience.

Typical personal fabrication using a laser cutter allows objects to be created from raw material and the engraving of existing objects. Current methods to precisely align an object with the laser is a difficult process due to indirect manipulations. In this paper, we propose a marker-based system as a novel paradigm for direct interactive laser cutting on existing objects. Our system, MARCut, performs the laser cutting based on tangible markers that are applied directly onto the object to express the design. Two types of markers are available; hand constructed Shape Markers that represent the desired geometry, and Command Markers that indicate the operational parameters such as cut, engrave or material.

Sleep deprivation is known to have serious deleterious effects on executive functioning and job performance. Augmented reality has an ability to place pertinent information at the fore, guiding visual focus and reducing instructional complexity. This paper presents a study to explore how spatial augmented reality instructions impact procedural task performance on sleep deprived users. The user study was conducted to examine performance on a procedural task at six time points over the course of a night of total sleep deprivation. Tasks were provided either by spatial augmented reality-based projections or on an adjacent monitor. The results indicate that participant errors significantly increased with the monitor condition when sleep deprived. The augmented reality condition exhibited a positive influence with participant errors and completion time having no significant increase when sleep deprived. The results of our study show that spatial augmented reality is an effective sleep deprivation countermeasure under laboratory conditions.

Dynamic spatial augmented reality requires accurate real-time 3D pose information of the physical objects that are to be projected onto. Previous depth-based methods for tracking objects required strong features to enable recognition; making it difficult to estimate an accurate 6DOF pose for physical objects with a small set of recognizable features (such as a non-textured cube). We propose a more accurate method with fewer limitations for the pose estimation of a tangible object that has known planar faces and using depth data from an RGB-D camera only. In this paper, the physical object's shape is limited to cubes of different sizes. We apply this new tracking method to achieve dynamic projections onto these cubes. In our method, 3D points from an RGB-D camera are divided into a cluster of planar regions, and the point cloud inside each face of the object is fitted to an already-known geometric model of a cube. With the 6DOF pose of the physical object, SAR generated imagery is then projected correctly onto the physical object. The 6DOF tracking is designed to support tangible interactions with the physical object. We implemented example interactive applications with one or multiple cubes to show the capability of our method.

We draw from the domains of Visual Analytics and Augmented Reality to support a new form of in-situ interactive visual analysis. We present a Situated Analytics model, a novel interaction, and a visualization concept for reasoning support. Situated Analytics has four primary elements: situated information, abstract information, augmented reality interaction, and analytical interaction.

We propose a three dimensional position measurement method employing planar photo detectors to calibrate a Spatial Augmented Reality system of unknown geometry. In Spatial Augmented Reality, projectors overlay images onto an object in the physical environment. For this purpose, the alignment of the images and physical objects is required. Traditional camera based 3D position tracking systems, such as multi-camera motion capture systems, detect the positions of optical markers in two-dimensional image plane of each camera device, so those systems require multiple camera devices at known locations to obtain 3D position of the markers. We introduce a detection method of 3D position of a planar photo detector by projecting gradient patterns. The main contribution of our method is to realize an alignment of the projected images with the physical objects and measuring the geometry of the objects simultaneously for Spatial Augmented Reality applications.

Presents the introductory editorial for this issue of the publication.

Providing high resolution tangible user interface components without batteries such as dials and sliders that support dynamic user interface arrangement is challenging. Previous work uses RFID to support limited resolution custom-built components. We demonstrate improved techniques using commercial off the shelf input controls incorporated into passive RFID tags using an on-off key subcarrier to encode state information into the RFID signal. Our method supports high resolution components that do not require power cables or batteries. We provide exemplars demonstrating how the technique supports a range of user interface components including buttons, dials, sliders, flex and light sensors. Compared to previous work, we obtain a higher resolution, only limited by sample time, for all components and demonstrate 115 discrete dial positions. Our technique allows the TUI components to be freely placed and rearranged without hardwiring or batteries.

We demonstrate the geometrically-correct projection-based texture mapping onto a deformable object like a cloth. This system can be used to simulate design that involves change in shape, such as sheets of malleable material. The geometrically-correct projection-based texture mapping onto a cloth is conducted using the measurement of object's 3D shape and the detection of the retro-reflective marker on the object's surface. Rapid prototyping is used as an example application of this projection technique.

Mobile devices such as some recent phones are now fitted with projection capabilities that support Spatial Augmented Reality (SAR) and require investigation to uncover new interaction possibilities. This paper presents a study measuring user performance in a search and select task using a tracked handheld projector and data tags, a 3D physical cue. This physical cue is used to mark the location of hidden SAR information. The experiment required participants to search for virtual symbols presented on two 5ft, multi-sided control panels. Two methods of presenting AR information were employed, SAR alone and SAR with the inclusion of physical cues to indicate the location of the information. The results showed that attaching data tags, compared to virtual content alone lowered the overall task completion time and reduced handheld projector movement. Subjectively, participants also preferred the combination of virtual data with data tags across both task variations

Projection-based Augmented Reality commonly employs a rigid substrate as the projection surface and does not support scenarios where the substrate can be reshaped. This investigation presents a projection-based AR system that supports deformable substrates that can be bent, twisted or folded. We demonstrate a new invisible marker embedded into a deformable substrate and an algorithm that identifies deformations to project geometrically correct textures onto the deformable object. The geometrically correct projection-based texture mapping onto a deformable marker is conducted using the measurement of the 3D shape through the detection of the retro-reflective marker on the surface. In order to achieve accurate texture mapping, we propose a marker pattern that can be partially recognized and can be registered to an object’s surface. The outcome of this work addresses a fundamental vision recognition challenge that allows the underlying material to change shape and be recognized by the system. Our evaluation demonstrated the system achieved geometrically correct projection under extreme deformation conditions. We envisage the techniques presented are useful for domains including prototype development, design, entertainment and information based AR systems.

Welcome to the Twelfth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR 2013).

Ray tracing is an elegant and intuitive image generation method. The introduction of GPU-accelerated ray tracing and corresponding software frameworks makes this rendering technique a viable option for Augmented Reality applications. Spatial Augmented Reality employs projectors to illuminate physical models and is used in fields that require photorealism, such as design and prototyping. Ray tracing can be used to great effect in this Augmented Reality environment to create scenes of high visual fidelity. However, the peculiarities of SAR systems require that core ray tracing algorithms be adapted to this new rendering environment. This paper highlights the problems involved in using ray tracing in a SAR environment and provides solutions to overcome them. In particular, the following issues are addressed: ray generation, hybrid rendering and view-dependent rendering.

We present a passive deformable haptic (PDH) glove to enhance mobile immersive augmented reality manipulation with a sense of computer-captured touch, responding to objects in the physical environment. We extend our existing pinch glove design with a Digital Foam sensor, placed under the palm of the hand. The novel glove input device supports a range of touch-activated, precise, direct manipulation modeling techniques with tactile feedback including hole-punching, trench cutting, and chamfer creation. The PDH glove helps improve a user's task performance time, decrease error rate and erroneous hand movements, and reduce fatigue.

This paper explores enhancing mobile immersive augmented reality manipulations by providing a sense of computer-captured touch through the use of a passive deformable haptic glove that responds to objects in the physical environment. The glove extends our existing pinch glove design with a Digital Foam sensor that is placed under the palm of the hand. The novel glove input device supports a range of touch-activated, precise, direct manipulation modeling techniques with tactile feedback including hole cutting, trench cutting, and chamfer creation. A user evaluation study comparing an image plane approach to our passive deformable haptic glove showed that the glove improves a user's task performance time, decreases error rate and erroneous hand movements, and reduces fatigue.

Industrial uses of Augmented Reality (AR) are growing, however their uses are consistently fashioned with an emphasis on consumption, delivering additional information to the worker to assist them in the completion of their job. A promising alternative is to allow user data creation during the actual process by the worker performing their duties. This not only allows spatially located annotations to be produced, it also allows an AR scene to be developed in-situ and in real-time. Tangible markers offer a physical interface while also creating physical containers to allow for fluent interactions. This form factor allows both attached and detached annotations, whilst allowing the creation of an AR scene during the process. This annotated scene will allow asynchronous collaboration to be conducted between multiple stakeholders, both locally and remotely. In this paper we discuss our reasoning behind such an approach, and present the current work on our prototype created to test and validate our proposition.

This paper presents results of a study measuring user performance in a procedural task using Spatial Augmented Reality (SAR). The task required participants to press sequences of buttons on two control panel designs in the correct order. Instructions for the task were shown either on a computer monitor, or projected directly onto the control panels. This work was motivated by discrepancies between the expectations from AR proponents and experimental findings. AR is often promoted as a way of improving user performance and understanding. With notable exceptions however, experimental results do not confirm these expectations. Reasons cited for results include limitations of current display technologies and misregistration caused by tracking and calibration errors. Our experiment utilizes SAR to remove these effects. Our results show that augmented annotations lead to significantly faster task completion speed, fewer errors, and reduced head movement, when compared to monitor based instructions. Subjectively, our results show augmented annotations are preferred by users.

We present the preliminary results of our proposal: a region-based detection and tracking method of arbitrary shapes. The method is designed to be robust against orientation and scale changes and also occlusions. In this work, we study the effectiveness of sequence of shape descriptors for matching purpose. We detect and track surfaces by matching the sequences of descriptor so called relevance measures with their correspondences in the database. First, we extract stable shapes as the detection target using Maximally Stable Extreme Region (MSER) method. The keypoints on the stable shapes are then extracted by simplifying the outline of the stable regions. The relevance measures that are composed by three keypoints are then computed and the sequences of them are composed as descriptors. During runtime, the sequences of relevance measures are extracted from the captured image and are matched with those in the database. When a particular region is matched with one in the database, the orientation of the region is then estimated and virtual annotations can be superimposed. We apply this approach in an interactive task support system that helps users for creating paper craft objects.

This paper presents a method for using object based manipulation and spatial augmented reality for the purpose of remote guidance. Previous remote guidance methods have typically not made use of any semantic information about the physical properties of the environment and require the helper and worker to provide context. Our new prototype system introduces a level of abstraction to the remote expert, allowing them to directly specify the object movements required of a local worker. We use 3D tracking to create a hidden virtual reality scene, mirroring the real world, with which the remote expert interacts while viewing a camera feed of the physical workspace. The intended manipulations are then rendered to the local worker using Spatial Augmented Reality (SAR). We report on the implementation of a functional prototype that demonstrates an instance of this approach. We anticipate that techniques such as the one we present will allow more efficient collaborative remote guidance in a range of physical tasks.

We present a set of distance-based interaction techniques for modeling and manipulation, enabled by a new input device called the ultrasonic gloves. The ultrasonic gloves are built upon the original design of the pinch glove device for virtual reality systems with a tilt sensor and a pair of ultrasonic transducers in the palms of the gloves. The transducers are distance-ranging sensors that allow the user to specify a range of distances by natural gestures such as facing the palms towards each other or towards other surfaces. The user is able to create virtual models of physical objects by specifying their dimensions with hand gestures. We combine the reported distance with the tilt orientation data to construct virtual models. We also map the distance data to create a set of affine transformation techniques, including relative and fixed scaling, translation, and rotation. Our techniques can be generalized to different sensor technologies.

In current Computer Aided Design (CAD) systems, designers are commonly restricted to a traditional workstation environment with mouse and keyboard. This environment is indirect from the physical object they are designing, and as such they may lose the one to one correspondence between the virtual and physical magnification of the design. In order to reduce this, we propose a Spatial Augmented Reality (SAR) based CAD system which consists of a fixed camera-projector pair, a Light Emitting Diode (LED) pen with two buttons, a wireless communication module, and a physical drawing board.

This poster presents the concept of combining two display technologies to enhance graphics effects in spatial augmented reality (SAR) environments. This is achieved by using an ePaper surface as an adaptive substrate instead of a white painted surface allowing the development of novel image techniques to improve image quality and object appearance in projector-based SAR environments.

This demonstration presents an adaptive visual marker optimised to improve tracking performance in Spatial Augmented Reality environments. The adaptive marker uses a color light sensor to capture the projected light color from a SAR system. The color information is used to select the optimal tracking color that is displayed on a diffused Red, Green, Blue Light Emitting Diode marker attached to a user's finger. We have selected to use the visible light spectrum for the marker since it can be leveraged to present visual feedback to support user interface interactions in addition to the tracking system operation. Our initial results have shown a performance improvement compared to a fixed color passive marker.

Many real-world networks are multivariate, i.e., they have attributes associated with nodes and/or edges. Examples include social networks whose nodes represent people and edges represent relationships. There is usually information about each person (such as name, age, and gender) and the relationship (such type, duration, and strength). Besides common graph analysis tasks (such as identifying the most influential or structurally important nodes), there are more complex analyses for multivariate networks. One of these is the multivariate graph clustering, i.e., identifying clusters formed by nodes that have similar attributes and are close to each other in terms of graph distance. For instance, in social network analysis, it is interesting to sociologists whether or not people with similar characteristics (node attributes) are also connected to each other. Currently there are very few visualization methods available for such analysis.

This paper investigates the use of Spatial Augmented Reality in the prototyping of new human-machine interfaces, such as control panels or car dashboards. The prototyping system uses projectors to present the visual appearance of controls onto a mock-up of a product. Finger tracking is employed to allow real-time interactions with the controls. This technology can be used to quickly and inexpensively create and evaluate interface prototypes for devices. In the past, evaluating a prototype involved constructing a physical model of the device with working components such as buttons. We have conducted a user study to compare these two methods of prototyping and to validate the use of spatial augmented reality for rapid iterative interface prototyping. Participants of the study were required to press pairs of buttons in sequence and interaction times were measured. The results indicate that while slower, users can interact naturally with projected control panels.

We present a set of techniques to perform in-situ refinements on simple geo-referenced models using mobile augmented reality systems. The refinements include affine transformations to the model and surface feature additions, including high detail concave and convex features. The techniques employ pinch gloves and a single-point laser rangefinder augmented with an orientation sensor as input devices. Finished models can be exported for use with other geospatial applications. The proposed techniques are intended to be an effective and elegant approach to enhancing outdoor models using mobile augmented reality.

This paper presents a new user interface methodology for Spatial Augmented Reality systems. The methodology is based on a set of physical tools that are overloaded with logical functions. Visual feedback presents the logical mode of the tool to the user by projecting graphics onto the physical tools. This approach makes the tools malleable in their functionality, with this change conveyed to the user by changing the projected information. Our prototype application implements a two handed technique allowing an industrial designer to digitally airbrush onto an augmented physical model, masking the paint using a virtualized stencil.

Augmented reality x-ray vision allows users to see through walls and view real occluded objects and locations. We present an augmented reality x-ray vision system that employs multiple view modes to support new visualizations that provide depth cues and spatial awareness to users. The edge overlay visualization provides depth cues to make hidden objects appear to be behind walls, rather than floating in front of them. Utilizing this edge overlay, the tunnel cut-out visualization provides details about occluding layers between the user and remote location. Inherent limitations of these visualizations are addressed by our addition of view modes allowing the user to obtain additional detail by zooming in, or an overview of the environment via an overhead exocentric view.

We have developed a system built on our mobile AR platform that provides users with see-through vision, allowing visualization of occluded objects textured with real-time video information. We present a user study that evaluates the userpsilas ability to view this information and understand the appearance of an outdoor area occluded by a building while using a mobile AR computer. This understanding was compared against a second group of users who watched video footage of the same outdoor area on a regular computer monitor. The comparison found an increased accuracy in locating specific points from the scene for the outdoor AR participants. The outdoor participants also displayed more accurate results, and showed better speed improvement than the indoor group when viewing more than one video simultaneously.

This paper presents the design and development of an ARWeather simulation application, which can simulate various types of precipitation: rain, snow, and hail. We analysed various real occurrences weather types and how they could be simulated in a mobile Augmented Reality system. The Tinmith system is wearable computer system for the development and deployment of the final ARWeather system that allows for autonomous and free movement for the user. The users can move freely inside the simulated weather without limitation.

We present a design and implementation of an in-situation menu system for loading and visualising 3D models in a physical world context. The menu system uses 3D objects as menu items, and the whole menu is placed within the context of the augmented environment. The use of 3D objects supports the visualisation and placement of 3D models into the augmented world. The menu system employs techniques for the placement of 3D models in two relative coordinate systems: head relative and world relative.

This paper describes the results of a study conducted to determine the efficiency of visual cues for a collaborative navigation task in a mixed-space environment. The task required a user with an exocentric view of a virtual room to navigate a fully immersed user with an egocentric view to an exit. The study compares natural hand-based gestures, a mouse-based interface and an audio only technique to determine their relative efficiency on task completion times. The results show that visual cue-based collaborative navigation techniques are significantly more efficient than an audio-only technique.

This paper presents a new interaction metaphor for mixed space collaboration: HOG on a WIM. Hand of god (HOG) on a world in miniature (WIM) is the first collaborative WIM. It enables a table-top display user to collaborate with a virtual reality (VR) user. The tabletop display user has a god's eye view of the virtual world and communicates with the VR user through natural gestures and speech. The VR user controls a WIM to navigate and manipulate the virtual world with the aid of the tabletop display user's guidance.

Digital Foam is a new input sensor developed to support clay like sculpting and modeling operations. We present techniques facilitating navigation and manipulation operations performed using Spherical Digital Foam as a sole input device. Our free-form sculpting technique allows manipulation of new and existing 3D models using accumulated sculpting like motions. Digital Foam's multi-point pressure sensitive surface captures the separate locations of a user's fingertips allowing controlled manipulation of multiple model vertices simultaneously. Additionally, we developed a technique that allows the camera view and zoom to be controlled by applying varying pressure to the Digital Foam surface. Furthermore, we have designed a menu system tailored for operation using Spherical Digital Foam as a sole input device using both the internal orientation sensor and the pressure sensitive surface.

This paper describes techniques to allow both the visualization of hidden objects, and removal of real objects, for a mobile augmented reality user. A gesture based technique is also described which allows the user to select when to view occluded objects. By using the real-time modeling techniques of the Tinmith system, the user is able to create the required geometry to allow image based rendering techniques to be used to render corrected images on the user's display. The images of the hidden objects are captured by a mobile robot platform that is controlled by the mobile user.

This paper presents a new form of remote active tangible interactions built with the Display-based Measurement and Control System. A prototype system was constructed to demonstrate the concepts of coupled remote tangible objects on rear projected tabletop displays. A user evaluation measuring social presence for two users performing a furniture placement task was performed, to determine a difference between this new system and a traditional mouse.

This paper presents a new interaction metaphor we have termed "god-like interaction". This is a metaphor for improved communication of situational and navigational information between outdoor users, equipped with mobile augmented reality systems, and indoor users, equipped with tabletop projector display systems. Physical objects are captured by a series of cameras viewing a table surface indoors, the data is sent over a wireless network, and is then reconstructed at a real-world location for outdoor augmented reality users. Our novel god-like interaction metaphor allows users to communicate information using physical props as well as natural gestures. We have constructed a system that implements our god-like interaction metaphor as well as a series of novel applications to facilitate collaboration between indoor and outdoor users. We have extended a well-known video based rendering algorithm to make it suitable for use on outdoor wireless networks of limited bandwidth. This paper also describes the limitations and lessons learned during the design and construction of the hardware that supports this research.

This paper presents results from a study into the usability issues of two tasks (selection and annotation of a physical object) for users operating mobile augmented reality systems. The study compared the following three different modes of cursor manipulation: a handheld mouse, a head cursor, and an image-plane vision-tracked device. The selection task was evaluated based on number of mouse button clicks, completion time, and a subjective survey. The annotation task was evaluated based on accuracy of the annotation, completion time, and a subjective survey.

In this paper, we describe use of constraint-based methods for configuring ubiquitous workspaces. A declarative representation allows succinct, easily maintainable definitions of the dependencies inherent in setting up a meeting, and permits the use of general constraint reasoners for various standard tasks such as setting up meeting interfaces, switching between setting for different meetings, and saving and restoring settings. Personalisation techniques can be used for intelligently adapting the workspace to individual user needs.

This work describes the combination of two technologies, augmented reality and GIS, to provide a new way to visualise viticulture GIS data using outdoor mobile computers. Viticulturists use GIS to assist with accurately understanding the parameters that affect their yields and quality of the grapes from different vineyards. The ability to view this data in the field digitally would be advantageous to the viticulturist. This work describes the ARVino system; an AR platform that was built for visualising 3D data outdoors using a movable tripod-based computer. We describe the user interface, some problems that were encountered, and how the visualisation and interface were evaluated through an expert review.

This paper presents the design for our latest backpack to support mobile outdoor augmented reality, and how it evolved from lessons learned with our previous designs. We present a number of novel features which help to reduce size and weight, improve reliability and ease of configuration, and reduce CPU usage on laptop computers.

This paper introduces the concept of augmented reality working planes to action and construction at a distance for mobile outdoor augmented reality systems. We have based our new AR working planes technique on CAD working planes, but by using coordinate systems relative to the body they can be specified and used much more intuitively than on a desktop system. We demonstrate in this paper how AR working planes can be used for the display of information, the manipulation of existing 3D objects, and the creation of new geometry. AR working planes are particularly well suited to supporting 3D modelling in mobile outdoor AR systems. This modelling task is a particularly difficult problem, because the environment is typically at a scale much larger than the user, and direct manipulation techniques can not be used.

This paper describes a new hybrid tracking system that integrates standard outdoor augmented reality trackers with a low cost indoor tracker based on the use of fiducial markers. We use multiple cameras, separate orientation sensing, and scene graph integration to improve on similar previous systems. This hybrid tracker allows applications to operate within large indoor and outdoor environments with minimal scaling costs. Tracking is performed using GPS style world coordinates both indoors and outdoors, and has been integrated into our existing mobile 3D modelling applications.

This paper presents a new software architecture for 3D mixed reality applications, named Tinmith-evo5. Currently there are a limited number of existing toolkits for the development of 3D mixed reality applications, each optimized for particular feature but at the detriment of others. Complex interactive user interfaces and applications require extensive supporting infrastructure, and can be hampered by inadequate support. The Tinmith-evo5 architecture is optimised to develop mobile augmented reality and other interactive 3D applications on portable platforms with limited resources. This architecture is implemented in C++ with an object-oriented data flow design, an object store based on the Unix file system model, and uses other ideas from existing previous work.

This paper outlines some of the capabilities of the Tinmith-Metro modeling system, based on mobile outdoor augmented reality technology. This system implements a new user interface based on tracked pinch gloves and a series of techniques named construction at a distance for the capture and creation of 3D geometry. The user controls the modeling process using hand and head motions, with modelling accuracy guided by the requirements of the user. Using Tinmith-Metro, users are able to model outdoor geometry representing buildings and natural features in an intuitive fashion.

This paper presents user interface technology for mobile outdoor augmented reality systems, and a set of new techniques we have named construction at a distance for the capture and creation of 3D geometry. Using a pair of vision tracked pinch gloves and a specially designed menu system, a user is able to use the physical presence of their body to perform modelling tasks in ways not previously possible.

The authors present a model for bringing the coordination power of workflow management systems to outdoor wearable augmented reality (AR) systems. They portray how mobile equipment may be integrated with adaptive, context-aware work environments. A scenario of a medical emergency task is described to illustrate the functionality of this form of collaboration system. Appropriate information stickers are introduced to support data collection in medical emergency scenarios in a sophisticated form through a hands-free user interface for medical personnel. They propose the use of new user interface technology, including multimedia, AR information stickers, and the allocation of patient medical records to identified locations of the human body. A key feature is the access to relevant information for users in the mobile environment as well as for those in the advanced control room. An additional advantage is the automatic recording of on-site data, which helps to build the medical record of a patient without interfering with the work of the emergency team.

This paper presents a new technique we have created, known as Bread Crumbs, which allows the modelling of large outdoor ground features using a mobile augmented reality system and a user's physical presence. Using this technique, we demonstrate an example of modelling a large grassy area on campus. This technique is one component of a larger set of tools which allow users to capture and work with complex outdoor geometry interactively, and to view this geometry in new and interesting ways.

This paper presents a unified user interface technology, using 3D interaction techniques, constructive solid geometry and a glove-based menuing system, known as Tinmith-Hand, to support mobile outdoor augmented reality (AR) applications and indoor virtual reality (VR) applications. Tinmith-Hand uses similar concepts for both domains, so that AR and VR applications are consistent and simple to use. The future goal of this user interface is to allow collaboration between outdoor AR and indoor VR systems.

This paper describes an experiment to explore the effectiveness of animation in improving indirect manipulation operations. Indirect manipulation operations are those initiated by command menus and buttons to perform a transformation on a graphical object or set of graphical objects. The particular improvement is an operation's ability to show both what would happen if the operation were committed and what would happen if it were cancelled while an operation is being considered. The experiment required subjects to watch a simple alignment operation for a set of graphical objects. They were then asked to record the original placement of those graphical objects. Each task used one of four visual cues: modified telltale, wiggle, colour, or no visual cue. We found the modified telltale, wiggle, and colour visual effects significantly more effective than no visual feedback for cuing the user as to original position of the graphical objects. The modified telltale and colour effects were significantly more effective than the wiggle effect. The major conclusion drawn from this experiment is the use of visual feedback improves a user's ability to remember the previous position of graphical objects after an alignment operation.

User interface designers can easily become locked into the paradigm that is most conveniently supported by their tools. We interpret ‘tools’ broadly here to include analysis and design techniques, component engineering methods, and the widget libraries available. To avoid stagnation, we must constantly question (if not always overturn) the ways that we develop ‘conventional’ user interfaces. We must be on the lookout for new User Interface Tools (UITs). Toward that end, herein we present five papers, each of which can be considered as presenting a distinct direction for UIT innovation. These papers are extended and updated versions of select papers from the Proceedings of the First Australasian User Interface Conference (themed ‘Stepping Out of Windows’) held in Adelaide, South Australia in January 2000. They are interesting both for their diverse contributions to UIT thinking as well as to showcase the quality of work coming from the Australian and New Zealand computer science research communities.

If judiciously applied, animation techniques can enhance the look and feel of computer applications that present a graphical human interface. Such techniques can smooth the rough edges and abrupt transitions common in many current graphical interfaces, and strengthen the illusion of direct manipulation that many interfaces strive to present. To date, few applications include such animation techniques. One possible reason is that animated interfaces are difficult to implement: they are difficult to design, place great burdens on programmers, and demand high-performance from underlying graphics systems.This article describes how direct manipulation human computer interfaces can be augmented with techniques borrowed from cartoon animators. In particular, we wish to improve the visual feedback of a direct manipulation interface by smoothing the changes of an interface, giving manipulated objects a feeling of substance and providing cues that anticipate the result of a manipulation. Our approach is to add support for animation techniques such as object distortion and keyframe interpolation, and to provide prepackaged animation effects such as animated widgets for common user interface interactions.To determine if these tools and techniques are practical and effective, we built a prototype direct manipulation drawing editor with an animated interface and used the prototype editor to carry out a set of human factors experiments. The experiments show that the techniques are practical even on standard workstation hardware, and that the effects can indeed enhance direct manipulation interfaces.

This paper describes an architecture to support a hardware and software platform for research into the use of wearable computers and augmented reality in an outdoor environment. The architecture supports such devices as a GPS, compass, and head-mounted display. A prototype system was built to support novel applications by drawing graphics (such as maps, building plans, and compass bearings) on the head-mounted display, projecting information over that normally seen by the user and hence augmenting a user's perception of reality. This paper presents a set of novel augmented reality and multimedia applications operated in an outdoor environment.

Techniques of cartoon animation, when applied to the interface, provide the user with greater visual cues, and convey a feeling of substance to the objects being manipulated. The paper describes the effects used to animate many of the interactions in a structured graphical editor. The editor is used as a tool in a visual parallel programming environment and provides a new large application to test the effectiveness of the concept of animated direct manipulation. Also described are the implementation issues in order to provide animated graphical feedback for direct manipulation.

Article Free Access Share on Animating direct manipulation interfaces Authors: Bruce H. Thomas Advanced Computing Research Centre, School of Computer and Information Science, University of South Australia, The Levels, SA, Australia 5095 Advanced Computing Research Centre, School of Computer and Information Science, University of South Australia, The Levels, SA, Australia 5095View Profile , Paul Calder Department of Computer Science, The Flinders University of South Australia, GPO Box 2100, Adelaide, Australia Department of Computer Science, The Flinders University of South Australia, GPO Box 2100, Adelaide, AustraliaView Profile Authors Info & Claims UIST '95: Proceedings of the 8th annual ACM symposium on User interface and software technologyDecember 1995 Pages 3–12https://doi.org/10.1145/215585.215628Online:01 December 1995Publication History 21citation729DownloadsMetricsTotal Citations21Total Downloads729Last 12 Months29Last 6 weeks3 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF