Mastering the correct use of laboratory equipment is a fundamental skill for undergraduate science students involved in laboratory-based training. However, hands-on laboratory time is often limited, and remote students may struggle as their absence from the physical lab limits their skill development. An air displacement micropipette was selected for this investigation, as accuracy and correct technique are essential in generating reliable assay data. Handling small liquid volumes demands hand dexterity and practice to achieve proficiency. This research assesses the importance of tactile authenticity during training by faithfully replicating the micropipette's key physical and operational characteristics. We developed a custom haptic training approach called 'Smart Pipette' which promotes accurate operation and enhances laboratory dexterity training. A comparative user study with 34 participants evaluated the effectiveness of the Smart Pipette custom haptic device against training with off-the-shelf hardware, specifically the Quest VR hand controller, which was chosen because it is held mid-air similar to a laboratory micropipette. Both training conditions are integrated with the same self-paced virtual simulation displayed on a computer screen, offering audiovisual instructions and real-time guidance. Results demonstrated that participants trained with the Smart Pipette custom haptic exhibited increased accuracy and precision while making fewer errors than those trained with off-the-shelf hardware. The Smart Pipette and the Quest VR controller had no significant differences in cognitive load or system usability scores. Tactile authentic interaction devices address challenges faced by online learners, while their applicability extends to traditional classrooms, where real-time feedback significantly enhances overall training performance outcomes.

While visual channels (e.g., color, shape, size) have been explored for visualizing data in data physicalizations, there is a lack of understanding regarding how to encode data into physical material properties (e.g., roughness, hardness). This understanding is critical for ensuring data is correctly communicated and for potentially extending the channels and bandwidth available for encoding that data. We present a method to encode ordinal data into roughness, validated through user studies. In the first study, we identified just noticeable differences in perceived roughness from this method. In the second study, we 3D-printed proof of concepts for five different multivariate physicalizations using the model. These physicalizations were qualitatively explored (N=10) to understand people’s comprehension and impressions of the roughness channel. Our findings suggest roughness may be used for certain types of data encoding, and the context of the data can impact how people interpret roughness mapping direction.

User interfaces in virtual reality enable diverse interactions within the virtual world, though they typically lack the haptic cues provided by physical interface controls. Haptic retargeting enables flexible mapping between dynamic virtual interfaces and physical controls to provide real haptic feedback. This investigation aims to extend these remapped interfaces to support more diverse control types. Many interfaces incorporate sliders, switches, and knobs. These controls hold fixed states between interactions creating potential conflicts where a virtual control has a different state from the physical control. This paper presents two methods, “manual” and “automatic”, for synchronizing physical and virtual control states and explores the effects of these methods on the usability of remapped interfaces. Results showed that interfaces without retargeting were the ideal configuration, but they lack the flexibility that remapped interfaces provide. Automatic synchronization was faster and more usable; however, manual synchronization is suitable for a broader range of physical interfaces.

Existing situated visualization models assume the user is able to directly interact with the objects and spaces to which the data refers (known as physical referents). We review a growing body of work exploring scenarios where the user interacts with a proxy representation of the physical referent rather than immediately with the object itself. This introduces a complex mixture of immediate situatedness and proxies of situatedness that goes beyond the expressiveness of current models. We propose an extended model of situated visualization that encompasses Immediate Situated Visualization and ProxSituated (Proxy of Situated) Visualization. Our model describes a set of key entities involved in proxSituated scenarios and important relationships between them. From this model, we derive design dimensions and apply them to existing situated visualization work. The resulting design space allows us to describe and evaluate existing scenarios, as well as to creatively generate new conceptual scenarios.

This article presents a set of adaptive reset techniques for use with haptic retargeting systems focusing on interaction with hybrid virtual reality interfaces that align with a physical interface. Haptic retargeting between changing physical and virtual targets requires a reset where the physical and virtual hand positions are re-aligned. We present a modified Point technique to guide the user in the direction of their next interaction such that the remaining distance to the target is minimized upon completion of the reset. This, along with techniques drawn from existing work are further modified to consider the angular and translational gain of each redirection and identify the optimal position for the reset to take place. When the angular and translational gain is within an acceptable range, the reset can be entirely omitted. This enables continuous retargeting between targets removing interruptions from a sequence of retargeted interactions. These techniques were evaluated in a user study which showed that adaptive reset techniques can provide a significant decrease in task completion time, travel distance, and the number of user errors.

Recent research in immersive visualisations has explored the use of physical 3D models together with virtual data visualisations, with a simple data encoding. However, little is known about how this technique could be extended towards more complex multivariate data with multiple charts. We present augmented scale models, immersive visualisations that place charts of multivariate data via Augmented Reality (AR) registered to physical 3D models. We identified two main factors for presenting AR charts in the limited display space around the models: 1) how charts are laid out (Slides vs Dashboard), and 2) how the chart views are arranged in the 3D space (On Scale Model, On Table, On Virtual Board). In a within-subject user study, we evaluated these two design considerations. We found that chart layout and view arrangement do not affect task error but do vary in response time. Dashboard and Slides perform equally well on simple tasks that require comparison of a single chart across scale models. However, in more complex tasks such as comparing multiple charts within a scale model or across several scale models, Dashboard shows no decrease in time performance, while Slides’s time performance decreases significantly. We also found that On Scale Model has the fastest performance, has good chart-scale model integration, and supports charts comparison well. On Table and On Virtual Board on the other hand, show a trade-off between the ability to support charts comparison across models and the chart-scale model integration.

This paper presents Shape Aware Haptic Retargeting, an extension of "state-of-the-art" haptic retargeting that is the first to support retargeted interaction between any part of the user’s hand and any part of the target object. In previous haptic retargeting algorithms, the maximum retargeting is applied only when the hand position aligns with the target position. Shape Aware Haptic Retargeting generalizes the distance computation process to instead consider the hand and target geometry. The shortest hand-target distance is then used to calculate the applied retargeting offset. This ensures the full amount of haptic retargeting is applied at the point of contact with the passive haptic regardless of contact position on the hand or target. We leverage existing geometry algorithms to implement three distance computation methods: Multi-Point, Primitive and Mesh Geometry, in addition to conventional single position approaches. These are evaluated through a set of simulated interactions instead of the single position representation used in previous haptic retargeting systems. The evaluation demonstrated all three approaches can provide improved interaction accuracy over a Point distance computation method, with Mesh Geometry being the most accurate and Primitive being the preferred method for combined performance and interaction accuracy.

Data physicalisations afford people the ability to directly interact with data using their hands, potentially achieving a more comprehensive understanding of a dataset. Due to their complex nature, the representation of graphs and networks could benefit from physicalisation, bringing the dataset from the digital world into the physical one. However, no empirical work exists investigating the effects physicalisations have upon comprehension as they relate to graph representations. In this work, we present initial design considerations for graph physicalisations, as well as an empirical study investigating differences in comprehension between virtual and physical representations. We found that participants perceived themselves as being more accurate via touch and sight (visual-haptic) than the graphical-only modality, and perceived a triangle count task as less difficult in visual-haptic than in the graphical-only modality. Additionally, we found that participants significantly preferred interacting with visual-haptic over other conditions, despite no significant effect on task time or error.

In augmented reality (AR), gesture/hand-based interactions are becoming more common place over tangible user interfaces (TUIs) and physical controllers. Major concerns regard portability and battery-life with TUIs and physical controllers in an AR environment. For a TUI or physical controller to be usable ”on the go” and in the field, they need to be compact such that they can be easily deployable. Subsequently, it is desirable to be low-energy or powerless, such that the device remains functional after long periods. In this paper, we present our initial design towards a powerless, passive controller that leverages the optical camera of a AR head-mounted display (such as the Hololens or Magic Leap) for tracking. We discuss related work, design motivations, high and low fidelity designs, opportunistic/adaptable input modalities, and use cases. We conclude with propositions for future work.

We present Embodied Axes, a controller which supports selection operations for 3D imagery and data visualisations in Augmented Reality. The device is an embodied representation of a 3D data space -- each of its three orthogonal arms corresponds to a data axis or domain specific frame of reference. Each axis is composed of a pair of tangible, actuated range sliders for precise data selection, and rotary encoding knobs for additional parameter tuning or menu navigation. The motor actuated sliders support alignment to positions of significant values within the data, or coordination with other input: e.g., mid-air gestures in the data space, touch gestures on the surface below the data, or another Embodied Axes device supporting multi-user scenarios. We conducted expert enquiries in medical imaging which provided formative feedback on domain tasks and refinements to the design. Additionally, a controlled user study was performed and found that the Embodied Axes was overall more accurate than conventional tracked controllers for selection tasks.

Data videos are a highly impactful method of communication and are becoming a prevalent medium for communicating information. While the majority of current research focuses on the cinematic aspects of data videos, very little is known about the narrative methodologies involved. This paper presents our insights derived from an initial exploration of this area. We present a taxonomy based on the analysis of 70 existing data videos examining their narrative and visual approaches. We propose that our taxonomy can be used to explain the characteristics or design of data videos. Applying this taxonomy, we present our observations, including the trend of popular technologies applied in current data videos, the under-utilization of promising methods, and highlight research opportunities in the field.

This paper proposes a novel interface for virtual reality in which physical interface components are mapped to multiple virtual counterparts using haptic retargeting illusions. This gives virtual reality interfaces the ability to have correct haptic sensations for many virtual buttons although in the physical space there is only one. This is a generic system that can be applied to areas including design, interaction tasks, product prototype development and interactive games in virtual reality. The system presented extends existing retargeting algorithms to support asymmetric bimanual interactions. A new warp technique, called interface warp, was developed to support remapped virtual reality user interfaces. Through an experimental user study, we explore the effects of bimanual retargeting and the interface warp technique on task response time, errors, presence, perceived manipulation compared to unimanual (single handed) retargeting and other existing warp techniques. The results demonstrated faster task response time and less errors for the interface warp technique and shows no significant effect of bimanual interactions.

Research has shown that Mixed Presence Groupware (MPG) systems are a valuable collaboration tool. However research into MPG systems is limited to a handful of tabletop and Virtual Reality (VR) systems with no exploration of Head-Mounted Display (HMD) based Augmented Reality (AR) solutions. We present a new system with two local users and one remote user using HMD based AR interfaces. Our system provides tools allowing users to layout a room with the help of a remote user. The remote user has access to a marker and pointer tools to assist in directing the local users. Feedback collected from several groups of users showed that our system is easy to learn but could have increased accuracy and consistency.

This paper introduces GeoGate, an Augmented Reality tabletop system that extends the Space-Time Cube and utilizes a ring-shaped tangible user interface to explore correlations between entities in multiple location datasets. We demonstrate GeoGate in the context of the maritime domain, where operators seek to find geo-temporal associations between trajectories recorded from a global positioning system, and light data extracted from night time satellite images. GeoGate utilizes a tabletop system displaying a traditional 2D map in conjunction with a Microsoft Hololens to present a single view of the data with a novel Augmented Reality extension of the Space-Time Cube. To validate GeoGate, we present the results of a user study comparing GeoGate with the existing 2D approach used in a normal desktop environment. The outcomes of the user study show that GeoGate's approach reduces mistakes in the interpretation of the correlations between various datasets, while the qualitative results show that such a system is preferable for the majority of geo-temporal maritime tasks compared.

This paper presents the results of two cognitive load studies comparing three augmented reality display technologies: spatial augmented reality, the optical see-through Microsoft HoloLens, and the video see-through Samsung Gear VR. In particular, the two experiments focused on isolating the cognitive load cost of receiving instructions for a button-pressing procedural task. The studies employed a self-assessment cognitive load methodology, as well as an additional dual-task cognitive load methodology. The results showed that spatial augmented reality led to increased performance and reduced cognitive load. Additionally, it was discovered that a limited field of view can introduce increased cognitive load requirements. The findings suggest that some of the inherent restrictions of head-mounted displays materialize as increased user cognitive load.

Typical personal fabrication using a laser cutter allows objects to be created from raw material and the engraving of existing objects. Current methods to precisely align an object with the laser is a difficult process due to indirect manipulations. In this paper, we propose a marker-based system as a novel paradigm for direct interactive laser cutting on existing objects. Our system, MARCut, performs the laser cutting based on tangible markers that are applied directly onto the object to express the design. Two types of markers are available; hand constructed Shape Markers that represent the desired geometry, and Command Markers that indicate the operational parameters such as cut, engrave or material.

Sleep deprivation is known to have serious deleterious effects on executive functioning and job performance. Augmented reality has an ability to place pertinent information at the fore, guiding visual focus and reducing instructional complexity. This paper presents a study to explore how spatial augmented reality instructions impact procedural task performance on sleep deprived users. The user study was conducted to examine performance on a procedural task at six time points over the course of a night of total sleep deprivation. Tasks were provided either by spatial augmented reality-based projections or on an adjacent monitor. The results indicate that participant errors significantly increased with the monitor condition when sleep deprived. The augmented reality condition exhibited a positive influence with participant errors and completion time having no significant increase when sleep deprived. The results of our study show that spatial augmented reality is an effective sleep deprivation countermeasure under laboratory conditions.

We draw from the domains of Visual Analytics and Augmented Reality to support a new form of in-situ interactive visual analysis. We present a Situated Analytics model, a novel interaction, and a visualization concept for reasoning support. Situated Analytics has four primary elements: situated information, abstract information, augmented reality interaction, and analytical interaction.

We propose a three dimensional position measurement method employing planar photo detectors to calibrate a Spatial Augmented Reality system of unknown geometry. In Spatial Augmented Reality, projectors overlay images onto an object in the physical environment. For this purpose, the alignment of the images and physical objects is required. Traditional camera based 3D position tracking systems, such as multi-camera motion capture systems, detect the positions of optical markers in two-dimensional image plane of each camera device, so those systems require multiple camera devices at known locations to obtain 3D position of the markers. We introduce a detection method of 3D position of a planar photo detector by projecting gradient patterns. The main contribution of our method is to realize an alignment of the projected images with the physical objects and measuring the geometry of the objects simultaneously for Spatial Augmented Reality applications.

Providing high resolution tangible user interface components without batteries such as dials and sliders that support dynamic user interface arrangement is challenging. Previous work uses RFID to support limited resolution custom-built components. We demonstrate improved techniques using commercial off the shelf input controls incorporated into passive RFID tags using an on-off key subcarrier to encode state information into the RFID signal. Our method supports high resolution components that do not require power cables or batteries. We provide exemplars demonstrating how the technique supports a range of user interface components including buttons, dials, sliders, flex and light sensors. Compared to previous work, we obtain a higher resolution, only limited by sample time, for all components and demonstrate 115 discrete dial positions. Our technique allows the TUI components to be freely placed and rearranged without hardwiring or batteries.

We demonstrate the geometrically-correct projection-based texture mapping onto a deformable object like a cloth. This system can be used to simulate design that involves change in shape, such as sheets of malleable material. The geometrically-correct projection-based texture mapping onto a cloth is conducted using the measurement of object's 3D shape and the detection of the retro-reflective marker on the object's surface. Rapid prototyping is used as an example application of this projection technique.

Mobile devices such as some recent phones are now fitted with projection capabilities that support Spatial Augmented Reality (SAR) and require investigation to uncover new interaction possibilities. This paper presents a study measuring user performance in a search and select task using a tracked handheld projector and data tags, a 3D physical cue. This physical cue is used to mark the location of hidden SAR information. The experiment required participants to search for virtual symbols presented on two 5ft, multi-sided control panels. Two methods of presenting AR information were employed, SAR alone and SAR with the inclusion of physical cues to indicate the location of the information. The results showed that attaching data tags, compared to virtual content alone lowered the overall task completion time and reduced handheld projector movement. Subjectively, participants also preferred the combination of virtual data with data tags across both task variations

Projection-based Augmented Reality commonly employs a rigid substrate as the projection surface and does not support scenarios where the substrate can be reshaped. This investigation presents a projection-based AR system that supports deformable substrates that can be bent, twisted or folded. We demonstrate a new invisible marker embedded into a deformable substrate and an algorithm that identifies deformations to project geometrically correct textures onto the deformable object. The geometrically correct projection-based texture mapping onto a deformable marker is conducted using the measurement of the 3D shape through the detection of the retro-reflective marker on the surface. In order to achieve accurate texture mapping, we propose a marker pattern that can be partially recognized and can be registered to an object’s surface. The outcome of this work addresses a fundamental vision recognition challenge that allows the underlying material to change shape and be recognized by the system. Our evaluation demonstrated the system achieved geometrically correct projection under extreme deformation conditions. We envisage the techniques presented are useful for domains including prototype development, design, entertainment and information based AR systems.

Ray tracing is an elegant and intuitive image generation method. The introduction of GPU-accelerated ray tracing and corresponding software frameworks makes this rendering technique a viable option for Augmented Reality applications. Spatial Augmented Reality employs projectors to illuminate physical models and is used in fields that require photorealism, such as design and prototyping. Ray tracing can be used to great effect in this Augmented Reality environment to create scenes of high visual fidelity. However, the peculiarities of SAR systems require that core ray tracing algorithms be adapted to this new rendering environment. This paper highlights the problems involved in using ray tracing in a SAR environment and provides solutions to overcome them. In particular, the following issues are addressed: ray generation, hybrid rendering and view-dependent rendering.

We present a passive deformable haptic (PDH) glove to enhance mobile immersive augmented reality manipulation with a sense of computer-captured touch, responding to objects in the physical environment. We extend our existing pinch glove design with a Digital Foam sensor, placed under the palm of the hand. The novel glove input device supports a range of touch-activated, precise, direct manipulation modeling techniques with tactile feedback including hole-punching, trench cutting, and chamfer creation. The PDH glove helps improve a user's task performance time, decrease error rate and erroneous hand movements, and reduce fatigue.

This paper explores enhancing mobile immersive augmented reality manipulations by providing a sense of computer-captured touch through the use of a passive deformable haptic glove that responds to objects in the physical environment. The glove extends our existing pinch glove design with a Digital Foam sensor that is placed under the palm of the hand. The novel glove input device supports a range of touch-activated, precise, direct manipulation modeling techniques with tactile feedback including hole cutting, trench cutting, and chamfer creation. A user evaluation study comparing an image plane approach to our passive deformable haptic glove showed that the glove improves a user's task performance time, decreases error rate and erroneous hand movements, and reduces fatigue.

We are delighted to present the 12th edition of the demonstration program of the IEEE International Symposium on Mixed and Augmented Reality. The ISMAR demonstration program provide hands-on experience to the community of the most recent technical and applied advancement in Augmented Reality.

In current Computer Aided Design (CAD) systems, designers are commonly restricted to a traditional workstation environment with mouse and keyboard. This environment is indirect from the physical object they are designing, and as such they may lose the one to one correspondence between the virtual and physical magnification of the design. In order to reduce this, we propose a Spatial Augmented Reality (SAR) based CAD system which consists of a fixed camera-projector pair, a Light Emitting Diode (LED) pen with two buttons, a wireless communication module, and a physical drawing board.

This poster presents the concept of combining two display technologies to enhance graphics effects in spatial augmented reality (SAR) environments. This is achieved by using an ePaper surface as an adaptive substrate instead of a white painted surface allowing the development of novel image techniques to improve image quality and object appearance in projector-based SAR environments.

This demonstration presents an adaptive visual marker optimised to improve tracking performance in Spatial Augmented Reality environments. The adaptive marker uses a color light sensor to capture the projected light color from a SAR system. The color information is used to select the optimal tracking color that is displayed on a diffused Red, Green, Blue Light Emitting Diode marker attached to a user's finger. We have selected to use the visible light spectrum for the marker since it can be leveraged to present visual feedback to support user interface interactions in addition to the tracking system operation. Our initial results have shown a performance improvement compared to a fixed color passive marker.

This paper investigates the use of Spatial Augmented Reality in the prototyping of new human-machine interfaces, such as control panels or car dashboards. The prototyping system uses projectors to present the visual appearance of controls onto a mock-up of a product. Finger tracking is employed to allow real-time interactions with the controls. This technology can be used to quickly and inexpensively create and evaluate interface prototypes for devices. In the past, evaluating a prototype involved constructing a physical model of the device with working components such as buttons. We have conducted a user study to compare these two methods of prototyping and to validate the use of spatial augmented reality for rapid iterative interface prototyping. Participants of the study were required to press pairs of buttons in sequence and interaction times were measured. The results indicate that while slower, users can interact naturally with projected control panels.

Digital Foam is a new input sensor developed to support clay like sculpting and modeling operations. We present techniques facilitating navigation and manipulation operations performed using Spherical Digital Foam as a sole input device. Our free-form sculpting technique allows manipulation of new and existing 3D models using accumulated sculpting like motions. Digital Foam's multi-point pressure sensitive surface captures the separate locations of a user's fingertips allowing controlled manipulation of multiple model vertices simultaneously. Additionally, we developed a technique that allows the camera view and zoom to be controlled by applying varying pressure to the Digital Foam surface. Furthermore, we have designed a menu system tailored for operation using Spherical Digital Foam as a sole input device using both the internal orientation sensor and the pressure sensitive surface.

This paper describes the design of hand-worn gloves for interacting with mobile outdoor augmented reality systems. Most existing systems rely on more traditional 2D input devices such as mice and keyboards. Since augmented reality information is typically registered in 3D to the environment, user interfaces need to be designed that are capable of supporting the more complex operations possible. This paper describes how we used metallic thread and adhesive fabric to add conduction sensing to a standard set of gloves which can survive harsh treatment; how Bluetooth and MSP430 microcontrollers are used to build a small circuit that is wireless and highly portable; and how AR-ToolKit is used for 3D tracking of fiducial markers on the thumbs. While we have previously demonstrated this technology with a number of our previous systems, this paper explains the various techniques we use in the implementation.

Research has focused on formal and informal groups playing a significant role in organisational knowledge sharing. The concept of the Community of Practice (CoP) has emerged as one such form and has attracted the attention of researchers. This paper reports the initial stages of an ongoing study of an emerging CoP. Specifically, we report the initial stage of a longer-term action research study, investigating how individuals within an informal group in a knowledge intensive organisation perceive themselves, including whether they display the characteristics commonly attributed to a CoP (Wenger, 1998) and how their shared practice might be related to knowledge work. Furthermore, reflecting upon the results reported we capture, in a model, how an emerging CoP might contribute to Knowledge Management (KM) in a knowledge intensive organisation, linking top-down and bottom-up KM strategies.

This paper presents the design for our latest backpack to support mobile outdoor augmented reality, and how it evolved from lessons learned with our previous designs. We present a number of novel features which help to reduce size and weight, improve reliability and ease of configuration, and reduce CPU usage on laptop computers.

This paper describes research that seeks to facilitate the capture, representation, and reasoning about, temporal information by usability engineers. The product, a method we call KAT-LITTER, is an extension of Johnson and Johnson's (1991) Knowledge Analysis of Tasks (KAT). An evaluation of KAT-LITTER showed that it influenced the design process in two significant ways: firstly, designers using KAT-LITTER spent more time reasoning about temporal issues than designers using KAT alone, and secondly these same designers considered a broader spectrum of temporal issues. Further developments of KAT-LITTER are briefly discussed.