We introduce the notion of an Active Proxy interface, i.e. tangible models as proxies for physical data referents, supporting interactive exploration of data through active manipulation. We realise an active proxy data visualisation system, “MarioChart", using robot carts relocating themselves on a tabletop, e.g., to align with their data referents in a map or other visual layout. We consider a casual-data exploration scenario involving a multivariate campus sustainability dataset, using scale models as proxies for their physical building data referents. Our empirical study (n=12) compares active proxy use with conventional tablet interaction, finding that our active proxy system enhances short-term spatial memory of data and enables faster completion of certain data analytic tasks. It shows no significant differences compared to traditional touchscreens in long-term memory, physical fatigue, mental workload, or user engagement. Our study offers an initial baseline for active proxy techniques and advances understanding of tangible interfaces in situated data visualisation.

A central goal of Augmented Reality (AR)-based Situated Visualization (SV) is to seamlessly integrate digital information into its relevant physical context. While existing frameworks describe numerous design dimensions, the field lacks a rigorous model to evaluate this integration. To address this, we introduce SCORE, a framework for quantifying diegesis - a narratology concept describing the extent to which an element belongs to its narrative world. Grounded in a systematic analysis of 50 contemporary SV works, SCORE defines five dimensions of diegesis in SV: Spatial proximity, Concreteness, cOherence, Referential context, and Environmental context. In addition to qualitative comparison, our framework also provides a quantitative measure of diegesis, enabling SCORE to distinguish AR-based SVs that prior models have treated as theoretically equivalent. We validate the framework by demonstrating a consistent correlation between higher scores and positive usability outcomes. Based on these findings, we offer insights for SV designers.

Conventional Mixed Reality (MR) workspaces are frequently organized in cockpit-like layouts, where multiple floating windows surround the user. While this configuration facilitates access to digital content, it often induces occlusion, reducing understanding of the physical environment and limiting access to real-world objects. To overcome this challenge, we present the Contour-Adaptive Mixed Environment Overlays (CAMEO), a contour-adaptive MR interface that drapes virtual windows onto physical surfaces. This design integrates digital content with nearby items, thereby improving users’ visual access to background objects and supporting interaction with them. We evaluate CAMEO in two controlled studies. The first demonstrates that draping reduces hand-movement detours relative to flat mid-air surfaces, enabling more direct interaction with nearby items. The second shows that controlled window deformation does not significantly impair text legibility when compared to flat surfaces. Together, these findings contribute a novel design paradigm for MR workspaces that balances immersion, readability, and environmental understanding.

Physical augmentation technologies can extend human abilities beyond biological limitations, creating "superpower" experiences. However, as these technologies integrate with the human body, they can also introduce unfortunate negative effects due to the body’s constraints. Inspired by biology, especially the fly-catching mechanism of the flytrap plant, we designed "Flytrap Hand", a provocative artifact that grants users the superpower of accelerated grasping through a distance sensor and electrical muscle stimulation, while resulting in uncertainty. A mixed-method study (N = 12) revealed that participants appreciated the fortunate superpower effects, but also highlighted the unfortunate effects of discomfort, skepticism toward bodily automation, and ethical concerns. Building on these findings, we propose a design framework for designing more thoughtful superpower experiences that account for potential negative effects. Ultimately, with our work, we aim to help designers consider not only superpowers’ positive but also negative effects early on as they may be difficult to rectify in hindsight.

Designing superpowers in Human-Computer Interaction (HCI), often inspired by science fiction, has garnered increased attention. However, it is important to ask whether such superpower designs might have inherent negative side effects, especially considering that technological advances allow going beyond short demos to integrate these superpowers into everyday life. To understand the positive and negative side effects of superpower design, we created "EmoPals"and studied it in everyday life. EmoPals is a novel system inspired by telepathy, where one user's emotions are detected through a brain-computer interface and replicated on the other user's face through electrical muscle stimulation, therefore one user's happiness makes the other smile and vice versa. A 5-day field study with 12 participants suggests that EmoPals can strengthen emotional connections and facilitate empathy, however, it also highlights the negative side effects of amplifying negative emotions and social discomfort. We propose five design recommendations for designing superpowers that account for negative side effects. Ultimately, we aim to deepen our understanding of superpower design for everyday life.

Mid-air gestures serve as a common interaction modality across Extended Reality (XR) applications, enhancing engagement and ownership through intuitive body movements. However, prolonged arm movements induce shoulder fatigue—known as "Gorilla Arm Syndrome"—degrading user experience and reducing interaction duration. Although existing ergonomic techniques derived from Fitts’ law (such as reducing target distance, increasing target width, and modifying control-display gain) provide some fatigue mitigation, their implementation in XR applications remains challenging due to the complex balance between user engagement and physical exertion. We present AlphaPIG, a meta-technique designed to Prolong Interactive Gestures by leveraging real-time fatigue predictions. AlphaPIG assists designers in extending and improving XR interactions by enabling automated fatigue-based interventions. Through adjustment of intervention timing and intensity decay rate, designers can explore and control the trade-off between fatigue reduction and potential effects such as decreased body ownership. We validated AlphaPIG’s effectiveness through a study (N=22) implementing the widely-used Go-Go technique. Results demonstrated that AlphaPIG significantly reduces shoulder fatigue compared to non-adaptive Go-Go, while maintaining comparable perceived body ownership and agency. Based on these findings, we discuss positive and negative perceptions of the intervention. By integrating real-time fatigue prediction with adaptive intervention mechanisms, AlphaPIG constitutes a critical first step towards creating fatigue-aware applications in XR.

Interactive technology has a complicated relationship with recreation in nature. Many people have praised, and many have lamented the impact of interactive technology on recreation in nature. Because nature recreation has important wellness benefits and interactive technology is likely to remain a part of nature recreation, there is a need to design interactive technology for nature recreation. Unfortunately, little generalized knowledge exists on how to design such technology. We create new intermediate design knowledge for interactive technology in nature recreation by drawing from others’ work, our prior work, and specifically Borgmann and Verbeek’s philosophies of technology. Our contribution is a framework based on a decomposition of engagement into nine facets related to engagement with place, time, and community. Four examples demonstrate the descriptive and generative power of the framework. This framework may enable the creation of interactive systems that complement rather than compete with nature recreation and may better preserve the wellness benefits of nature recreation.

Cycling has gained popularity due to growing interest in healthy and sustainable lifestyles. Simultaneously, Augmented Reality (AR) Head-Mounted Displays (HMDs) can assist cyclists by presenting notifications within their field of view without diverting their attention to external devices. While previous studies have investigated these advantages, safety concerns have primarily limited them to lab settings, creating a notable gap in understanding their real-world feasibility. We conducted a user study with 20 participants on a shared-use outdoor path and explored the impact of text-based HMD notification placement (top, right, bottom), on attentional tunneling and cyclists' experiences. Our results suggested that while the bottom placement received higher scores for perceived safety and noticeability, HMD notifications induced attentional tunneling, regardless of placement. We discuss our findings and present design insights for future HMD systems aimed at enhancing cyclists' safety and experience.

Room-scale immersive data visualisations provide viewers a wide-scale overview of a large dataset, but to interact precisely with individual data points they typically have to navigate to change their point of view. In traditional screen-based visualisations, focus-and-context techniques allow visualisation users to keep a full dataset in view while making detailed selections. Such techniques have been studied extensively on desktop to allow precise selection within large data sets, but they have not been explored in immersive 3D modalities. In this paper we develop a novel immersive focus-and-context technique based on a "magic portal" metaphor adapted specifically for data visualisation scenarios. An extendable-hand interaction technique is used to place a portal close to the region of interest. The other end of the portal then opens comfortably within the user's physical reach such that they can reach through to precisely select individual data points. Through a controlled study with 12 participants, we find strong evidence that portals reduce overshoots in selection and overall hand trajectory length, reducing arm and shoulder fatigue compared to ranged interaction without the portal. The portals also enable us to use a robot arm to provide haptic feedback for data within the limited volume of the portal region. In a second study with another 12 participants we found that haptics provided a positive experience (qualitative feedback) but did not significantly reduce fatigue. We demonstrate applications for portal-based selection through two use-case scenarios.

Augmented Reality (AR) is touted to be beneficial in supporting situated information display, allowing virtual information panels to be overlaid on real-world scenes. People must then use their spatial memory to navigate among these virtual panels effectively. While spatial memory has been studied in physical environments (wall displays) and virtual reality environments, there has been little research on how physical surroundings might affect memorisation of virtual content in a mixed environment like AR. Therefore, we provide the first AR study of spatial memory, comparing two different room settings with two different situated layouts of virtual targets on an abstract spatial memory task. We find that participants recall spatial patterns with greater accuracy and higher subjective ratings in a room with furniture compared to an empty room. Our findings lead to important design implications for mixed-reality user interfaces, particularly in information-rich applications like situated analytics and small-multiples information visualisation.

Technology-facilitated “superpowers” are a popular topic in human-computer interaction, providing people with the experience of having superhero-like fantastic abilities. However, science fiction literature tells us that superpowers can also be unfortunate. We designed Wi-Fi Twinge, a system that uses electrical muscle stimulation to extend human perception to sense unseen Wi-Fi signals, giving people an allergy-like twinge reaction to Wi-Fi: a superpower that can be seen as unfortunate while also having potential benefits. Through an in-the-wild study, interviews with 12 participants revealed that although the superpower from Wi-Fi Twinge induced negative feelings and impacted certain day-to-day activities, it improved awareness of self and the surrounding environment. We used these results to offer design tactics for creating future superpower systems that consider potential unfortunate side effects. Ultimately, we aim to advance our understanding of technology-facilitated superpowers.

COVID-19 regulations presented clinicians with a new set of challenges that affected their ability to efficiently provide patient care and, as a result, telemedicine was rapidly adopted to deliver care remotely. However, these telemedicine platforms undermine patient care due to clinicians' inability to acquire all the relevant patient information required to diagnose and treat the patient. To explore this gap, we conducted a requirements analysis for the development of a 3D remote patient monitoring and diagnosis platform, by using a user-centric design methodology. In this requirements analysis, we elicited information about the clinical domain, identified clinicians’ requirements, elicited clinicians’ insights, and documented the clinicians' requirements. The outcome was the emergence of refined clinicians' requirements to guide the implementation of the Digital Twin concept paired with holoportation for remote 3D monitoring and diagnosis of patients. We anticipate that the application of a 3D telemedicine platform with these requirements for patient care during a pandemic could potentially enhance clinicians' efficiency and the effectiveness of remote patient care.

Natural gestures are crucial for mid-air interaction, but predicting and managing muscle fatigue is challenging. Existing torque-based models are limited in their ability to model above-shoulder interactions and to account for fatigue recovery. We introduce a new hybrid model, NICER , which combines a torque-based approach with a new term derived from the empirical measurement of muscle contraction and a recovery factor to account for decreasing fatigue during rest. We evaluated NICER in a mid-air selection task using two interaction methods with different degrees of perceived fatigue. Results show that NICER can accurately model above-shoulder interactions as well as reflect fatigue recovery during rest periods. Moreover, both interaction methods show a stronger correlation with subjective fatigue measurement ( ρ = 0.978/0.976) than a previous model, Cumulative Fatigue ( ρ = 0.966/0.923), confirming that NICER is a powerful analytical tool to predict fatigue across a variety of gesture-based interactive applications.

This paper presents the design and evaluation of GestureExplorer, an Immersive Analytics tool that supports the interactive exploration, classification and sensemaking with large sets of 3D temporal gesture data. GestureExplorer features 3D skeletal and trajectory visualisations of gestures combined with abstract visualisations of clustered sets of gestures. By leveraging the large immersive space afforded by a Virtual Reality interface our tool allows free navigation and control of viewing perspective for users to gain a better understanding of gestures.

Recent studies have explored how users of immersive visualisation systems arrange data representations in the space around them. Generally, these have focused on placement centred at eye-level in absolute room coordinates. However, work in HCI exploring full-body interaction has identified zones relative to the user’s body with different roles. We encapsulate the possibilities for visualisation view management into a design space (called “DataDancing”). From this design space we extrapolate a variety of view management prototypes, each demonstrating a different combination of interaction techniques and space use. The prototypes are enabled by a full-body tracking system including novel devices for torso and foot interaction. We explore four of these prototypes, encompassing standard wall and table-style interaction as well as novel foot interaction, in depth through a qualitative user study. Learning from the results, we improve the interaction techniques and propose two hybrid interfaces that demonstrate interaction possibilities of the design space.

Automatic layout optimisation allows users to arrange augmented reality content in the real-world environment without the need for tedious manual interactions. This optimisation is often based on modelling the intended content placement as constraints, defined as cost functions. Then, applying a cost minimization algorithm leads to a desirable placement. However, such an approach is limited by the lack of user control over the optimisation results. In this paper we explore the concept of user-driven constraints for augmented reality layout optimisation. With our approach users can define and set up their own constraints directly within the real-world environment. We first present a design space composed of three dimensions: the constraints, the regions of interest and the constraint parameters. Then we explore which input gestures can be employed to define the user-driven constraints of our design space through a user elicitation study. Using the results of the study, we propose a holistic system design and implementation demonstrating our user-driven constraints, which we evaluate in a final user study where participants had to create several constraints at the same time to arrange a set of virtual contents.

It is our great pleasure to welcome you to the 22nd IEEE International Symposium on Mixed and Augmented Reality (ISMAR), held from 16 to 20 October 2023 in Sydney, Australia. ISMAR stands as the foremost international academic conference in the fields of Augmented Reality (AR), Mixed Reality (MR) and Virtual Reality (VR). Over the years, it has continually expanded its horizon, delving into the latest developments in AR, MR, and VR within both commercial and research domains. The conference is organized and supported by IEEE, IEEE Computer Society, and IEEE VGTC.

Virtual Reality (VR) is increasingly being adopted in fitness, gaming, and workplace productivity applications for its natural interaction with body movement. A widely accepted method for quantifying the physical fatigue caused by VR interactions is through metrics such as Consumed Endurance (CE). Proposed in 2014, CE calculates the shoulder torque to infer endurance time (ET)—i.e. the maximum amount of time a pose can be maintained—during mid-air interactions. This model remains widely cited but has not been closely examined beyond its initial evaluation, leaving untested assumptions about exertion from low-intensity interactions and its basis on torque. In this paper, we present two VR studies where we (1) collect a baseline dataset that replicates the foundation of CE and (2) extend the initial evaluation in a pointing task from a two-dimensional (2D) screen to a three-dimensional (3D) immersive environment. Our baseline dataset collected from a high-precision tracking system found that the CE model overestimates ET for low-exertion interactions. Further, our studies reveal that a biomechanical model based on only torque cannot account for additional exertion measured when the shoulder angle exceeds 90° elevation. Based on these findings, we propose a revised formulation of CE to highlight the need for a hybrid approach in future fatigue modelling.

We explore the design of situated visual links in outdoor augmented reality (AR) for connecting miniature buildings on a virtual map to their real-world counterparts. We first distill design criteria from prior work, then conduct two user studies to evaluate a set of proposed link designs to better understand users' preferences for different design choices of the links. In two user studies we evaluated, respectively, a set of link geometries in a virtual environment and a refined AR prototype in two different outdoor environments. The studies reveal that links help in identifying buildings in the environments. Participants prefer straight rather than curved links, simple and thin links to avoid information occlusion, and links and maps aligned with their direction of view. We recommend using a consistent color with a strong contrast to the background color for all links in a scene. To improve visibility, the diameter of links should grow with distance to the viewer and optional animated stripes can be placed on links. The findings of this study have the potential to bolster the development of various situated visualization applications, such as those used in urban planning, tourism, smart agriculture, and other fields.

Immersive analytics research has explored how embodied data representations and interactions can be used to engage users in sensemaking. Prior research has broadly overlooked the potential of immersive space for supporting analytic provenance, the understanding of sensemaking processes through users’ interaction histories. We propose the concept of embodied provenance, the use of three-dimensional space and embodied interactions in supporting recalling, reproducing, annotating and sharing analysis history in immersive environments. We design a conceptual framework for embodied provenance by highlighting a set of design criteria for analytic provenance drawn from prior work and identifying essential properties for embodied provenance. We develop a prototype system in virtual reality to demonstrate the concept and support the conceptual framework by providing multiple data views and embodied interaction metaphors in a large virtual space. We present a use case scenario of energy consumption analysis and evaluated the system through a qualitative evaluation with 17 participants, which show the system’s potential for assisting analytic provenance using embodiment. Our exploration of embodied provenance through this prototype provides lessons learnt to guide the design of immersive analytic tools for embodied provenance.

Immersive environments offer new possibilities for exploring three-dimensional volumetric or abstract data. However, typical mid-air interaction offers little guidance to the user in interacting with the resulting visuals. Previous work has explored the use of haptic controls to give users tangible affordances for interacting with the data, but these controls have either: been limited in their range and resolution; were spatially fixed; or required users to manually align them with the data space. We explore the use of a robot arm with hand tracking to align tangible controls under the user's fingers as they reach out to interact with data affordances. We begin with a study evaluating the effectiveness of a robot-extended slider control compared to a large fixed physical slider and a purely virtual mid-air slider. We find that the robot slider has similar accuracy to the physical slider but is significantly more accurate than mid-air interaction. Further, the robot slider can be arbitrarily reoriented, opening up many new possibilities for tangible haptic interaction with immersive visualisations. We demonstrate these possibilities through three use-cases: selection in a time-series chart; interactive slicing of CT scans; and finally exploration of a scatter plot depicting time-varying socio-economic data.

Head-mounted augmented reality (AR) displays allow for the seamless integration of virtual visualisation with contextual tangible references, such as physical (tangible) globes. We explore the design of immersive geospatial data visualisation with AR and tangible globes. We investigate the “tangible-virtual interplay” of tangible globes with virtual data visualisation, and propose a conceptual approach for designing immersive geospatial globes. We demonstrate a set of use cases, such as augmenting a tangible globe with virtual overlays, using a physical globe as a tangible input device for interacting with virtual globes and maps, and linking an augmented globe to an abstract data visualisation. We gathered qualitative feedback from experts about our use case visualisations, and compiled a summary of key takeaways as well as ideas for envisioned future improvements. The proposed design space, example visualisations and lessons learned aim to guide the design of tangible globes for data visualisation in AR.

We propose TimeTables, a novel prototype system that aims to support data exploration, using embodiment with space-time cubes in virtual reality. TimeTables uses multiple space-time cubes on virtual tabletops, which users can manipulate by extracting time layers or individual buildings to create new tabletop views. The surrounding environment includes a large space for multiple linked tabletops and a storage wall. TimeTables presents information at different time scales by stretching layers to drill down in time. Users can also jump into tabletops to inspect data from an egocentric perspective. We present a use case scenario of energy consumption displayed on a university campus to demonstrate how our system could support data exploration and analysis over space and time. From our experience and analysis we believe the system has a high potential in assisting spatio-temporal data exploration and analysis.

In immersive environments, positioning data visualisations around the user in a wraparound layout has been advocated as advantageous over flat arrangements more typical of traditional screens. However, other than limiting the distance users must walk, there is no clear design rationale behind this common practice, and little research on the impact of wraparound layouts on visualisation tasks. The ability to remember the spatial location of elements of visualisations within the display space is crucial to support visual analytical tasks, especially those that require users to shift their focus or perform comparisons. This ability is influenced by the user's spatial memory but how spatial memory is affected by different display layouts remains unclear. In this paper, we perform two user studies to evaluate the effects of three layouts with varying degrees of curvature around the user (flat-wall, semicircular-wraparound, and circular-wraparound) on a visuo-spatial memory task in a virtual environment. The results show that participants are able to recall spatial patterns with greater accuracy and report more positive subjective ratings using flat than circular-wraparound layouts. While we didn't find any significant performance differences between the flat and semicircular-wraparound layouts, participants overwhelmingly preferred the semicircular-wraparound layout suggesting it is a good compromise between the two extremes of display curvature.

Worlds-in-Miniature (WiMs) are interactive worlds within a world and combine the advantages of an input space, a cartographic map, and an overview+detail interface. They have been used across the extended virtuality spectrum for a variety of applications. Building on an analysis of examples of WiMs from the research literature we contribute a design space for WiMs based on seven design dimensions. Further, we expand upon existing definitions of WiMs to provide a definition that applies across the extended reality spectrum. We identify the design dimensions of size-scope-scale, abstraction, geometry, reference frame, links, multiples, and virtuality. Using our framework we describe existing Worlds-in-Miniature from the research literature and reveal unexplored research areas. Finally, we generate new examples of WiMs using our framework to fill some of these gaps. With our findings, we identify opportunities that can guide future research into WiMs.

Immersive Analytics is a quickly evolving field that unites several areas such as visualisation, immersive environments, and human-computer interaction to support human data analysis with emerging technologies. This research has thrived over the past years with multiple workshops, seminars, and a growing body of publications, spanning several conferences. Given the rapid advancement of interaction technologies and novel application domains, this paper aims toward a broader research agenda to enable widespread adoption. We present 17 key research challenges developed over multiple sessions by a diverse group of 24 international experts, initiated from a virtual scientific workshop at ACM CHI 2020. These challenges aim to coordinate future work by providing a systematic roadmap of current directions and impending hurdles to facilitate productive and effective applications for Immersive Analytics.

Geographic data visualisation on virtual globes is intuitive and widespread, but has not been thoroughly investigated. We explore two main design factors for quantitative data visualisation on virtual globes: i) commonly used primitives (2D bar, 3D bar, circle) and ii) the orientation of these primitives (tangential, normal, billboarded). We evaluate five distinctive visualisation idioms in a user study with 50 participants. The results show that aligning primitives tangentially on the globe’s surface decreases the accuracy of area-proportional circle visualisations, while the orientation does not have a significant effect on the accuracy of length-proportional bar visualisations. We also find that tangential primitives induce higher perceived mental load than other orientations. Guided by these results we design a novel globe visualisation idiom, Geoburst, that combines a virtual globe and a radial bar chart. A preliminary evaluation reports potential benefits and drawbacks of the Geoburst visualisation.

Navigation is a primary interaction in virtual reality. Previous research has explored different forms of artificial locomotion techniques for navigation, including hand gestures and body motions. However, few studies have investigated force-based foot gestures as a locomotion technique. We present three force-based foot gestures (Foot Fly, Foot Step and Foot Teleportation) for navigation in a virtual environment, relying on surface electromyography sensors readings from leg muscles. A pilot study comparing our techniques with controller-based techniques indicates that force-based foot gestures can provide a fun and engaging alternative. Of all six input techniques evaluated, Foot Fly was often most preferred despite requiring more exertion than the Controller Fly technique.

Collaborative visual analytics leverages social interaction to support data exploration and sensemaking. These processes are typically imagined as formalised, extended activities, between groups of dedicated experts, requiring expertise with sophisticated data analysis tools. However, there are many professional domains that benefit from support for short 'bursts' of data exploration between a subset of stakeholders with a diverse breadth of knowledge. Such 'casual collaborative' scenarios will require engaging features to draw users' attention, with intuitive, 'walk-up and use' interfaces. This paper presents Uplift, a novel prototype system to support 'casual collaborative visual analytics' for a campus microgrid, co-designed with local stakeholders. An elicitation workshop with key members of the building management team revealed relevant knowledge is distributed among multiple experts in their team, each using bespoke analysis tools. Uplift combines an engaging 3D model on a central tabletop display with intuitive tangible interaction, as well as augmented-reality, mid-air data visualisation, in order to support casual collaborative visual analytics for this complex domain. Evaluations with expert stakeholders from the building management and energy domains were conducted during and following our prototype development and indicate that Uplift is successful as an engaging backdrop for casual collaboration. Experts see high potential in such a system to bring together diverse knowledge holders and reveal complex interactions between structural, operational, and financial aspects of their domain. Such systems have further potential in other domains that require collaborative discussion or demonstration of models, forecasts, or cost-benefit analyses to high-level stakeholders.

We introduce Corsican Twin, a tool for authoring augmented reality data visualisations in virtual reality using digital twins. The system provides users with the contextual information necessary to design embedded and situated data visualisations in a safe and convenient remote setting. We created system via a co-design process which involved people with little or no programming experience. Using the system, we illustrate three potential use cases for situated visualizations in the context of building maintenance, including: (1) on-site equipment debugging and diagnosis; (2) remote incident playback; and (3) operations simulations for future buildings. From feedback gathered during formative evaluations of our prototype tool with domain experts, we discuss implications, opportunities, and challenges for future in situ visualisation design tools.

Zippers are common in a wide variety of objects that we use daily. This work investigates how we can take advantage of such common daily activities to support seamless interaction with technology. We look beyond simple zipper-sliding interactions explored previously to determine how to weave foreground and background interactions into a vocabulary of natural usage patterns. We begin by conducting two user studies to understand how people typically interact with zippers. The findings identify several opportunities for zipper input and sensing, which inform the design of Zippro, a self-contained prototype zipper slider, which we evaluate with a standard jacket zipper. We conclude by demonstrating several applications that make use of the identified foreground and background input methods.

We explore the adaptation of 2D small-multiples visualisation on flat screens to 3D immersive spaces. We use a "shelves" metaphor for layout of small multiples and consider a design space across a number of layout and interaction dimensions. We demonstrate the applicability of a prototype system informed by this design space to data sets from different domains. We perform two user studies comparing the effect of the shelf curvature dimension from our design space on users’ ability to perform comparison and trend analysis tasks. Our results suggest that, with fewer multiples, a flat layout is more performant despite the need for participants to walk further. With an increase in the number of multiples, this performance difference disappears due to the time participants had to spend walking. In the latter case, users prefer a semi-circular layout over either a fully surrounding or a flat arrangement.

Visual exploration of maps often requires a contextual understanding at multiple scales and locations. Multiview map layouts, which present a hierarchy of multiple views to reveal detail at various scales and locations, have been shown to support better performance than traditional single-view exploration on desktop displays. This paper investigates the extension of such layouts of 2D maps into 3D immersive spaces, which are not limited by the real-estate barrier of physical screens and support sensemaking through spatial interaction. Based on our initial implementation of immersive multiview maps, we conduct an exploratory study with 16 participants aimed at understanding how people place and view such maps in immersive space. We observe the layouts produced by users performing map exploration search, comparison and route-planning tasks. Our qualitative analysis identifies patterns in layoutgeometry (spherical, spherical cap, planar),overview-detail relationship (central window, occluding, coordinated) andinteraction strategy. Based on these observations, along with qualitative feedback from a user walkthrough session, we identify implications and recommend features for immersive multiview map systems. Our main findings are that participants tend to prefer and arrange multiview maps in a spherical cap layout around them and that they often rearrange the views during tasks.

This article reviews two decades of research in topics in Information Visualisation emerging from the Data Visualisation and Immersive Analytics Lab at Monash University Australia (Monash IA Lab). The lab has been influential with contributions in algorithms, interaction techniques and experimental results in Network Visualisation, Interactive Optimisation and Geographic and Cartographic visualisation. It has also been a leader in the emerging topic of Immersive Analytics, which explores natural interactions and immersive display technologies in support of data analytics. We reflect on advances in these areas but also sketch our vision for future research and developments in data visualisation more broadly.

We propose a multi-scale Mixed Reality (MR) collaboration between the Giant, a local Augmented Reality user, and the Miniature, a remote Virtual Reality user, in Giant-Miniature Collaboration (GMC). The Miniature is immersed in a 360-video shared by the Giant who can physically manipulate the Miniature through a tangible interface, a combined 360-camera with a 6 DOF tracker. We implemented a prototype system as a proof of concept and conducted a user study (n=24) comprising of four parts comparing: A) two types of virtual representations, B) three levels of Miniature control, C) three levels of 360-video view dependencies, and D) four 360-camera placement positions on the Giant. The results show users prefer a shoulder mounted camera view, while a view frustum with a complimentary avatar is a good visualization for the Miniature virtual representation. From the results, we give design recommendations and demonstrate an example Giant-Miniature Interaction.

Three-dimensional scatterplots suffer from well-known perception and usability problems. In particular, overplotting and occlusion, mainly due to density and noise, prevent users from properly perceiving the data. Thanks to accurate head and hand tracking, immersive Virtual Reality (VR) setups provide new ways to interact and navigate with 3D scatterplots. VR also supports additional sensory modalities such as haptic feedback. Inspired by methods commonly used in Scientific Visualisation to visually explore volumes, we propose two techniques that leverage the immersive aspects of VR: first, a density-based haptic vibration technique (Scaptics) which provides feedback through the controller; and second, an adaptation of a cutting plane for 3D scatterplots (Highlight-Plane). We evaluated both techniques in a controlled study with two tasks involving density (finding high- and low-density areas). Overall, Scaptics was the most time-efficient and accurate technique, however, in some conditions, it was outperformed by Highlight-Plane.

Welcome and Overview

In immersive display environments, such as virtual or augmented reality, we can make explicit the connections between data points in visualisations and their context in the world, or in other visualisations. This paper considers the requirements and design space for drawing such links in order to minimise occlusion and clutter. A novel possibility in immersive environments is to optimise the link layout with respect to a particular point of view. In collaborative scenarios there is the need to do this for multiple points of view. We present an algorithm to achieve such link layouts and demonstrate its applicability in a variety of practical use cases.

Freehand gesture interaction has long been proposed as a `natural' input method for Augmented Reality (AR) applications, yet has been little explored for intensive applications like multiscale navigation. In multiscale navigation, such as digital map navigation, pan and zoom are the predominant interactions. A position-based input mapping (e.g. grabbing metaphor) is intuitive for such interactions, but is prone to arm fatigue. This work focuses on improving digital map navigation in AR with mid-air hand gestures, using a horizontal intangible map display. First, we conducted a user study to explore the effects of handedness (unimanual and bimanual) and input mapping (position-based and rate-based). From these findings we designed DiveZoom and TerraceZoom, two novel hybrid techniques that smoothly transition between position- and rate-based mappings. A second user study evaluated these designs. Our results indicate that the introduced input-mapping transitions can reduce perceived arm fatigue with limited impact on performance.

Understanding the endpoint distribution of pointing selection tasks can reveal the underlying patterns on how users tend to acquire a target, which is one of the most essential and pervasive tasks in interactive systems. It could further aid designers to create new graphical user interfaces and interaction techniques that are optimized for accuracy, efficiency, and ease of use. Previous research has explored the modeling of endpoint distribution outside of virtual reality (VR) systems that have shown to be useful in predicting selection accuracy and guide the design of new interactive techniques. This work aims at developing an endpoint distribution of selection tasks for VR systems which has resulted in EDModel , a novel model that can be used to predict endpoint distribution of pointing selection tasks in VR environments. The development of EDModel is based on two users studies that have explored how factors such as target size, movement amplitude, and target depth affect the endpoint distribution. The model is built from the collected data and its generalizability is subsequently tested in complex scenarios with more relaxed conditions. Three applications of EDModel inspired by previous research are evaluated to show the broad applicability and usefulness of the model: correcting the bias in Fitts's law, predicting selection accuracy, and enhancing pointing selection techniques. Overall, EDModel can achieve high prediction accuracy and can be adapted to different types of applications in VR.

Head and eye movement can be leveraged to improve the user's interaction repertoire for wearable displays. Head movements are deliberate and accurate, and provide the current state-of-the-art pointing technique. Eye gaze can potentially be faster and more ergonomic, but suffers from low accuracy due to calibration errors and drift of wearable eye-tracking sensors. This work investigates precise, multimodal selection techniques using head motion and eye gaze. A comparison of speed and pointing accuracy reveals the relative merits of each method, including the achievable target size for robust selection. We demonstrate and discuss example applications for augmented reality, including compact menus with deep structure, and a proof-of-concept method for on-line correction of calibration drift.

We present Mini-Me, an adaptive avatar for enhancing Mixed Reality (MR) remote collaboration between a local Augmented Reality (AR) user and a remote Virtual Reality (VR) user. The Mini-Me avatar represents the VR user's gaze direction and body gestures while it transforms in size and orientation to stay within the AR user's field of view. A user study was conducted to evaluate Mini-Me in two collaborative scenarios: an asymmetric remote expert in VR assisting a local worker in AR, and a symmetric collaboration in urban planning. We found that the presence of the Mini-Me significantly improved Social Presence and the overall experience of MR collaboration.

The advancements in Mixed Reality (MR), Unmanned Aerial Vehicle, and multi-scale collaborative virtual environments have led to new interface opportunities for remote collaboration. This paper explores a novel concept of flying telepresence for multi-scale mixed reality remote collaboration. This work could enable remote collaboration at a larger scale such as building construction. We conducted a user study with three experiments. The first experiment compared two interfaces, static and dynamic IPD, on simulator sickness and body size perception. The second experiment tested the user perception of a virtual object size under three levels of IPD and movement gain manipulation with a fixed eye height in a virtual environment having reduced or rich visual cues. Our last experiment investigated the participant's body size perception for two levels of manipulation of the IPDs and heights using stereo video footage to simulate a flying telepresence experience. The studies found that manipulating IPDs and eye height influenced the user's size perception. We present our findings and share the recommendations for designing a multi-scale MR flying telepresence interface.

Distributed display environments (DDEs) allow use of various specialized devices but challenge designers to provide a clean flow of data across multiple displays. Upcoming consumer-ready head-worn displays (HWDs) can play a central role in unifying the interaction experience in such ecosystems. In this paper, we report on the design and development of Gluey, a user interface that acts as a 'glue' to facilitate seamless input transitions and data movement across displays. Based on requirements we refine for such an interface, Gluey leverages inherent headworn display attributes such as field-of-view tracking and an always-available canvas to redirect input and migrate content across multiple displays, while minimizing device switching costs. We implemented a functional prototype integrating Gluey's numerous interaction possibilities. From our experience in this integration and from user evaluation results, we identify the open challenges in using HWDs to unify the interaction experience in DDEs.

The growth of mobile and wearable technologies has made it often difficult to understand what people in our surroundings are doing with their technology. In this paper, we introduce the concept of candid interaction: techniques for providing awareness about our mobile and wearable device usage to others in the vicinity. We motivate and ground this exploration through a survey on current attitudes toward device usage during interpersonal encounters. We then explore a design space for candid interaction through seven prototypes that leverage a wide range of technological enhancements, such as Augmented Reality, shape memory muscle wire, and wearable projection. Preliminary user feedback of our prototypes highlights the trade-offs between the benefits of sharing device activity and the need to protect user privacy.

As wearable computing goes mainstream, we must improve the state of interface design to keep users productive with natural-feeling interactions. We present the Personal Cockpit, a solution for mobile multitasking on head-worn displays. We appropriate empty space around the user to situate virtual windows for use with direct input. Through a design-space exploration, we run a series of user studies to fine-tune our layout of the Personal Cockpit. In our final evaluation, we compare our design against two baseline interfaces for switching between everyday mobile applications. This comparison highlights the deficiencies of current view-fixed displays, as the Personal Cockpit provides a 40% improvement in application switching time. We demonstrate of several useful implementations and a discussion of important problems for future implementation of our design on current and near-future wearable devices.

We propose the use of Hand-to-Face input, a method to interact with head-worn displays (HWDs) that involves contact with the face. We explore Hand-to-Face interaction to find suitable techniques for common mobile tasks. We evaluate this form of interaction with document navigation tasks and examine its social acceptability. In a first study, users identify the cheek and forehead as predominant areas for interaction and agree on gestures for tasks involving continuous input, such as document navigation. These results guide the design of several Hand-to-Face navigation techniques and reveal that gestures performed on the cheek are more efficient and less tiring than interactions directly on the HWD. Initial results on the social acceptability of Hand-to-Face input allow us to further refine our design choices, and reveal unforeseen results: some gestures are considered culturally inappropriate and gender plays a role in selection of specific Hand-to-Face interactions. From our overall results, we provide a set of guidelines for developing effective Hand-to-Face interaction techniques.

Applying visual analytics to large software systems can help users comprehend the wealth of information produced by source repository mining. One concept of interest is the co-evolution of test code with source code, or how source and test files develop together over time. For example, understanding how the testing pace compares to the development pace can help test managers gauge the effectiveness of their testing strategy. A useful concept that has yet to be effectively incorporated into a co-evolution visualization is co-change. Co-change is a quantity that identifies correlations between software artifacts, and we propose using this to organize our visualization in order to enrich the analysis of co-evolution. In this paper, we create, implement, and study an interactive visual analytics tool that displays source and test file changes over time (co-evolution) while grouping files that change together (co-change). Our new technique improves the analyst's ability to infer information about the software development process and its relationship to testing. We discuss the development of our system and the results of a small pilot study with three participants. Our findings show that our visualization can lead to inferences that are not easily made using other techniques alone.

Tabletop systems provide a versatile space for collaboration, yet, in many cases, are limited by the inability to differentiate the interactions of simultaneous users. We present See Me, See You, a lightweight approach for discriminating user touches on a vision-based tabletop. We contribute a valuable characterization of finger orientation distributions of tabletop users. We exploit this biometric trait with a machine learning approach to allow the system to predict the correct position of users as they touch the surface. We achieve accuracies as high as 98% in simple situations and above 92% in more challenging conditions, such as two-handed tasks. We show high acceptance from users, who can self-correct prediction errors without significant costs. See Me, See You is a viable solution for providing simple yet effective support for multi-user application features on tabletops.

Colocated groups using mobile devices do not share all of the benefits of face-to-face collaborators. Close interaction requires application support for awareness features, allowing participants to establish common ground. Following an overview of research on awareness and grounding, the results of an informal user study are presented, which demonstrate how current systems can deter users from engaging in close collaboration. Literature on awareness provides hope for improving this situation, but a naive transfer to mobile distributed user interfaces will not necessarily succeed. From prior art, a concise list of guidelines has been compiled to assist designers in providing awareness information to users of shared mobile workspaces. These guidelines can also serve as heuristics for the evaluation of future systems. An example is provided to demonstrate how these guidelines can be applied to the development of features for providing awareness of current location and browsing history to colocated users of mobile distributed user interfaces.

The limited viewport size of mobile devices requires that users continuously acquire information that lies beyond the edge of the screen. Recent hardware solutions are capable of continually tracking a user's finger around the device. This has created new opportunities for interactive solutions, such as direct off-screen pointing: the ability to directly point at objects that are outside the viewport. We empirically characterize user performance with direct off-screen pointing when assisted by target cues. We predict time and accuracy outcomes for direct off-screen pointing with existing and derived models. We validate the models with good results (R2 ≥ 0.9) and reveal that direct off-screen pointing takes up to four times longer than pointing at visible targets, depending on the desired accuracy tradeoff. Pointing accuracy degrades logarithmically with target distance. We discuss design implications in the context of several real-world applications.