The goal of this workshop is to present an overview of ongoing research on Human–Computer Interaction (HCI) for teaching and learning. The workshop focuses on how emerging interaction paradigms, such as adaptive interfaces, multimodal interaction, and game-based learning, can support richer, more engaging, and more inclusive Technology-Enhanced Learning (TEL) experiences. The discussion will be structured around three central research directions that remain insufficiently explored: facilitating the design and evaluation of TEL systems that promote effective learning and quality of experience through innovative HCI approaches; adapting TEL systems to the complexity of educational environments; and modeling learners and users in TEL systems to account for their diversity, their preferences, and constraints. The ultimate goal of the workshop is to bring together researchers interested in discussing and converging on the future challenges and opportunities of the field.

Data visualization benefits from non-visual cues to enable people to understand information by engaging with it through multimodality, yet most approaches rely on cumbersome technologies or large scale artifacts, making them difficult to adapt to dynamic or complex datasets. In this paper, we explore the use of cutaneous haptics as a lightweight quantitative channel for visualization tasks, allowing users to feel data and interact with it dynamically. We present Haptixel, an open-source DIY encountered-type wearable providing force-feedback on the users’ fingertips’ pulp. We propose an interaction framework illustrating how Haptixel can be used to complement visualization tasks through combinations of force levels and contact types. We evaluate our approach in a pixel-art-like VR user study (n=16) where pixels color/height are associated to forces as a univariate value mapping. Results show that participants can retrieve information with Haptixel, and significantly discriminate 3D-data with at least four levels of forces; suggesting that cutaneous force-feedback can function for quantitative distinctions in visualization tasks.

We characterize 16 challenges faced by those investigating and developing remote and synchronous collaborative experiences around visualization. Our work reflects the perspectives and prior research efforts of an international group of 29 experts from across human-computer interaction and visualization sub-communities. The challenges are anchored around five collaborative activities that exhibit a centrality of visualization and multimodal communication. These activities include exploratory data analysis, creative ideation, visualization-rich presentations, joint decision making grounded in data, and real-time data monitoring. The challenges also reflect the changing dynamics of these activities in the face of recent advances in extended reality (XR) and artificial intelligence (AI). As an organizing scheme for future research at the intersection of visualization and computer-supported cooperative work, we align the challenges with a sequence of four sets of research and development activities: technological choices, social factors, AI assistance, and evaluation.

Computer-mediated collaboration often relies on symmetrical interactions between users, where all the collaborators use identical devices. However, in some cases, either due to constraints (e.g. users in different environments) or by choice (e.g. using devices with different properties), users engage in asymmetrical interactions. Addressing such asymmetries in heterogeneous systems can be difficult as there has been no systematic analysis of how to define them, or their impact on collaboration. In this paper, we characterize the asymmetries that can arise between users’ interactions within collaborative heterogeneous systems. To this end, we conduct a systematic literature review of asymmetric collaborative systems, coding their properties, including the interaction spaces, their input and output modalities, and shared feedback. We then define the dimensions of asymmetry that emerge from this review. We discuss their impact on collaboration and outline a set of challenges and opportunities for future research.

Most work on collaborative immersive systems mimics real-world settings, using fully shared virtual workspaces that foster close collaboration. However, recent work in educational contexts using remote desktop environments suggests these shared approaches may not be optimal for learning, as it showed that individual workspaces lead to better learning outcomes. In this paper, we investigate whether individual workspaces also lead to better outcomes in a collaborative VR learning environment. We compare three distinct workspace configurations in a problem-solving task: (1) a fully shared environment where two users work on the same materials, (2) a replicated environment where each user has their own copy of the materials but can still see their partner and their workspace, and (3) a separated environment where users cannot see each other nor each other's workspace and each has individual materials. We evaluate how these configurations influenced collaborative interaction, problem-solving strategies, and learning. Our results suggest the replicated workspace reduced social experience and did not improve learning outcomes compared to the shared one, however, it allowed broader exploration of the problem space.

International audience

While data visualizations are typically abstract, there is a growing body of work around concrete visualizations, which use familiar objects to convey data. Concrete visualizations can complement abstract ones, especially in immersive analytics, but it is unclear how to design smoothly animated transitions between these two kinds of representations. We investigate a design space of abstract and concrete visualizations, where animated transitions are pathways through the design space. The design space is defined with four axes, each corresponding to a different transformation. We consider different ways to design animated transitions by staging and ordering the transformations along these axes. In a controlled experiment conducted in virtual reality with 16 participants, we compared four types of animated transitions and found quantitative and qualitative evidence of the superiority of a specific staging approach over the simultaneous application of all transformations. Our study pre-registration is available at https://osf.io/8mu73.

Schizophrenia is a serious mental health disorder which may include symptoms such as hallucinations, delusions, and disorganized behavior. In addition to experiencing a diverse symptomatology, individuals with schizophrenia suffer from significant stigmatization which can interfere with effective treatment of the disorder among other issues. As a primary source of stigmatization comes from healthcare professionals, we were motivated to explore new ways in which to educate healthcare students about the symptoms of schizophrenia. Despite its potential to immerse users in new experiences within a real environment, little research concerning the use of augmented reality (AR) to simulate schizophrenia exists. In this paper, we present Live-It, a tool designed using recommendations from prior work as well as inspiration from lived experiences to educate mental health students about schizophrenia. The simulation uses the video passthrough capabilities of the Meta Quest 3 headset to simulate delusions, auditory hallucinations and additional symptomatology. Using thematic analysis, we evaluated our simulation with nineteen students and eighteen experts in healthcare to understand its ability to engage users and reliably represent symptoms of the disorder, as well as to determine how best to improve upon the design before the tool is widely deployed in mental health curricula.Our findings suggest that participants better understood symptoms of schizophrenia after experiencing the simulation, highlighting the potential of Live-It to be used as an educational tool. We present our design, provide a detailed analysis of our findings, and underline next steps in the development of our tool.

Situated visualizations are a type of visualization where data is presented next to its physical referent (i.e., the physical object, space, or person it refers to), often using augmented-reality displays. While situated visualizations can be beneficial in various contexts and have received research attention, they are typically designed with the assumption that the physical referent is visible. However, in practice, a physical referent may be obscured by another object, such as a wall, or may be outside the user's visual field. In this paper, we propose a conceptual framework and a design space to help researchers and user interface designers handle non-visible referents in situated visualizations. We first provide an overview of techniques proposed in the past for dealing with non-visible objects in the areas of 3D user interfaces, 3D visualization, and mixed reality. From this overview, we derive a design space that applies to situated visualizations and employ it to examine various trade-offs, challenges, and opportunities for future research in this area.

Explaining to novice users how to interact in immersive VR applications may be challenging. This is in particular due to the fact that the learners are isolated from the real world, and they are asked to manipulate hardware and software objects they are not used to. Consequently, the onboarding phase, which consists in teaching the user how to interact with the application is particularly crucial. In this paper, we aim at giving a better understanding of current VR onboarding methods, their benefits and challenges. We performed 21 VR tutorial ergonomic reviews and 15 interviews with VR experts with experience in VR onboarding. Building on the results, we propose a conceptual framework for VR onboarding and discuss important research directions to explore the design of future efficient onboarding solutions adapted to VR.

We present Deimos, a grammar for specifying dynamic embodied immersive visualisation morphs and transitions. A morph is a collection of animated transitions that are dynamically applied to immersive visualisations at runtime and is conceptually modelled as a state machine. It is comprised of state, transition, and signal specifications. States in a morph are used to generate animation keyframes, with transitions connecting two states together. A transition is controlled by signals, which are composable data streams that can be used to enable embodied interaction techniques. Morphs allow immersive representations of data to transform and change shape through user interaction, facilitating the embodied cognition process. We demonstrate the expressivity of Deimos in an example gallery and evaluate its usability in an expert user study of six immersive analytics researchers. Participants found the grammar to be powerful and expressive, and showed interest in drawing upon Deimos’ concepts and ideas in their own research.

Recent studies have explored how users of immersive visualisation systems arrange data representations in the space around them. Generally, these have focused on placement centred at eye-level in absolute room coordinates. However, work in HCI exploring full-body interaction has identified zones relative to the user’s body with different roles. We encapsulate the possibilities for visualisation view management into a design space (called “DataDancing”). From this design space we extrapolate a variety of view management prototypes, each demonstrating a different combination of interaction techniques and space use. The prototypes are enabled by a full-body tracking system including novel devices for torso and foot interaction. We explore four of these prototypes, encompassing standard wall and table-style interaction as well as novel foot interaction, in depth through a qualitative user study. Learning from the results, we improve the interaction techniques and propose two hybrid interfaces that demonstrate interaction possibilities of the design space.

We present a novel, web-based visual eye-tracking analytics tool called Gazealytics. Our open-source toolkit features a unified combination of gaze analytics features that support flexible exploratory analysis, along with annotation of areas of interest (AOI) and filter options based on multiple criteria to visually analyse eye tracking data across time and space. Gazealytics features coordinated views unifying spatiotemporal exploration of fixations and scanpaths for various analytical tasks. A novel matrix representation allows analysis of relationships between such spatial or temporal features. Data can be grouped across samples, user-defined AOIs or time windows of interest (TWIs) to support aggregate or filtered analysis of gaze activity. This approach exceeds the capabilities of existing systems by supporting flexible comparison between and within subjects, hypothesis generation, data analysis and communication of insights. We demonstrate in a walkthrough that Gazealytics supports multiple types of eye tracking datasets and analytical tasks.

As mixed-reality (MR) technologies become more mainstream, the delineation between data visualisations displayed on screens or other surfaces and those floating in space becomes increasingly blurred. Rather than the choice of using either a 2D surface or the 3D space for visualising data being a dichotomy, we argue that users should have the freedom to transform visualisations seamlessly between the two as needed. However, the design space for such transformations is large, and practically uncharted. To explore this, we first establish an overview of the different states that a data visualisation can take in MR, followed by how transformations between these states can facilitate common visualisation tasks. We then describe a design space of how these transformations function, in terms of the different stages throughout the transformation, and the user interactions and input parameters that affect it. This design space is then demonstrated with multiple exemplary techniques based in MR.

We propose TimeTables, a novel prototype system that aims to support data exploration, using embodiment with space-time cubes in virtual reality. TimeTables uses multiple space-time cubes on virtual tabletops, which users can manipulate by extracting time layers or individual buildings to create new tabletop views. The surrounding environment includes a large space for multiple linked tabletops and a storage wall. TimeTables presents information at different time scales by stretching layers to drill down in time. Users can also jump into tabletops to inspect data from an egocentric perspective. We present a use case scenario of energy consumption displayed on a university campus to demonstrate how our system could support data exploration and analysis over space and time. From our experience and analysis we believe the system has a high potential in assisting spatio-temporal data exploration and analysis.

In immersive environments, positioning data visualisations around the user in a wraparound layout has been advocated as advantageous over flat arrangements more typical of traditional screens. However, other than limiting the distance users must walk, there is no clear design rationale behind this common practice, and little research on the impact of wraparound layouts on visualisation tasks. The ability to remember the spatial location of elements of visualisations within the display space is crucial to support visual analytical tasks, especially those that require users to shift their focus or perform comparisons. This ability is influenced by the user's spatial memory but how spatial memory is affected by different display layouts remains unclear. In this paper, we perform two user studies to evaluate the effects of three layouts with varying degrees of curvature around the user (flat-wall, semicircular-wraparound, and circular-wraparound) on a visuo-spatial memory task in a virtual environment. The results show that participants are able to recall spatial patterns with greater accuracy and report more positive subjective ratings using flat than circular-wraparound layouts. While we didn't find any significant performance differences between the flat and semicircular-wraparound layouts, participants overwhelmingly preferred the semicircular-wraparound layout suggesting it is a good compromise between the two extremes of display curvature.

Eye tracking is growing in popularity for multiple application areas, yet analysing and exploring the large volume of complex data remains difficult for most users. We present a comprehensive eye tracking visual analytics system to enable the exploration and presentation of eye-tracking data across time and space in an efficient manner. The application allows the user to gain an overview of general patterns and perform deep visual analysis of local gaze exploration. The ability to link directly to the video of the underlying scene allows the visualisation insights to be verified on the fly. The system was motivated by the need to analyse eye-tracking data collected from an ‘in the wild’ study with energy network operators and has been further evaluated via interviews with 14 eye-tracking experts in multiple domains. Results suggest that, thanks to state-of-the-art visualisation techniques and by providing context with videos, our system could enable an improved analysis of eye-tracking data through interactive exploration, facilitating comparison between different participants or conditions, thus enhancing the presentation of complex data analysis to non-experts. This research paper provides four contributions: (1) analysis of a motivational use case demonstrating the need for rich visual-analytics workflow tools for eye-tracking data; (2) a highly dynamic system to visually explore and present complex eye-tracking data; (3) insights from our applied use case evaluation and interviews with experienced users demonstrating the potential for the system and visual analytics for the wider eye-tracking community.

Immersive Analytics is a quickly evolving field that unites several areas such as visualisation, immersive environments, and human-computer interaction to support human data analysis with emerging technologies. This research has thrived over the past years with multiple workshops, seminars, and a growing body of publications, spanning several conferences. Given the rapid advancement of interaction technologies and novel application domains, this paper aims toward a broader research agenda to enable widespread adoption. We present 17 key research challenges developed over multiple sessions by a diverse group of 24 international experts, initiated from a virtual scientific workshop at ACM CHI 2020. These challenges aim to coordinate future work by providing a systematic roadmap of current directions and impending hurdles to facilitate productive and effective applications for Immersive Analytics.

Text editing is important and at the core of most complex tasks, like writing an email or browsing the web. Efficient and sophisticated techniques exist on desktops and touch devices, but are still underexplored for Augmented Reality Head Mounted Display (AR-HMD). Performing text selection, a necessary step before text editing, in AR display commonly uses techniques such as hand-tracking, voice commands, eye/head-gaze, which are cumbersome and lack precision. In this paper, we explore the use of a smartphone as an input device to support text selection in AR-HMD because of its availability, familiarity, and social acceptability. We propose four eyes-free text selection techniques, all using a smartphone - continuous touch, discrete touch, spatial movement, and raycasting. We compare them in a user study where users have to select text at various granularity levels. Our results suggest that continuous touch, in which we used the smartphone as a trackpad, outperforms the other three techniques in terms of task completion time, accuracy, and user preference.

Previous work has shown that increased effort and use of one’s body can improve memory. When positioning windows inside a virtual reality, does the use of a larger volume, and using one’s legs to move around, improve ability to later find the windows? The results of our experiment indicate there can be a modest benefit for spatial memory and retrieval time, but at the cost of increased time spent initially positioning the windows.

Collaborative visual analytics leverages social interaction to support data exploration and sensemaking. These processes are typically imagined as formalised, extended activities, between groups of dedicated experts, requiring expertise with sophisticated data analysis tools. However, there are many professional domains that benefit from support for short 'bursts' of data exploration between a subset of stakeholders with a diverse breadth of knowledge. Such 'casual collaborative' scenarios will require engaging features to draw users' attention, with intuitive, 'walk-up and use' interfaces. This paper presents Uplift, a novel prototype system to support 'casual collaborative visual analytics' for a campus microgrid, co-designed with local stakeholders. An elicitation workshop with key members of the building management team revealed relevant knowledge is distributed among multiple experts in their team, each using bespoke analysis tools. Uplift combines an engaging 3D model on a central tabletop display with intuitive tangible interaction, as well as augmented-reality, mid-air data visualisation, in order to support casual collaborative visual analytics for this complex domain. Evaluations with expert stakeholders from the building management and energy domains were conducted during and following our prototype development and indicate that Uplift is successful as an engaging backdrop for casual collaboration. Experts see high potential in such a system to bring together diverse knowledge holders and reveal complex interactions between structural, operational, and financial aspects of their domain. Such systems have further potential in other domains that require collaborative discussion or demonstration of models, forecasts, or cost-benefit analyses to high-level stakeholders.

Immersive technologies offer new opportunities to support collaborative visual data analysis by providing each collaborator a personal, high-resolution view of a flexible shared visualisation space through a head mounted display. However, most prior studies of collaborative immersive analytics have focused on how groups interact with surface interfaces such as tabletops and wall displays. This paper reports on a study in which teams of three co-located participants are given flexible visualisation authoring tools to allow a great deal of control in how they structure their shared workspace. They do so using a prototype system we call FIESTA: the Free-roaming Immersive Environment to Support Team-based Analysis. Unlike traditional visualisation tools, FIESTA allows users to freely position authoring interfaces and visualisation artefacts anywhere in the virtual environment, either on virtual surfaces or suspended within the interaction space. Our participants solved visual analytics tasks on a multivariate data set, doing so individually and collaboratively by creating a large number of 2D and 3D visualisations. Their behaviours suggest that the usage of surfaces is coupled with the type of visualisation used, often using walls to organise 2D visualisations, but positioning 3D visualisations in the space around them. Outside of tightly-coupled collaboration, participants followed social protocols and did not interact with visualisations that did not belong to them even if outside of its owner's personal workspace.

We introduce Corsican Twin, a tool for authoring augmented reality data visualisations in virtual reality using digital twins. The system provides users with the contextual information necessary to design embedded and situated data visualisations in a safe and convenient remote setting. We created system via a co-design process which involved people with little or no programming experience. Using the system, we illustrate three potential use cases for situated visualizations in the context of building maintenance, including: (1) on-site equipment debugging and diagnosis; (2) remote incident playback; and (3) operations simulations for future buildings. From feedback gathered during formative evaluations of our prototype tool with domain experts, we discuss implications, opportunities, and challenges for future in situ visualisation design tools.

Shared displays are well suited to public viewing and collaboration, however they lack personal space to view private information and act without disturbing others. Combining them with Augmented Reality (AR) headsets allows interaction without altering the context on the shared display. We study a set of such interaction techniques in the context of network navigation, in particular path following, an important network analysis task. Applications abound, for example planning private trips on a network map shown on a public display. The proposed techniques allow for hands-free interaction, rendering visual aids inside the headset, in order to help the viewer maintain a connection between the AR cursor and the network that is only shown on the shared display. In two experiments on path following, we found that adding persistent connections between the AR cursor and the network on the shared display works well for high precision tasks, but more transient connections work best for lower precision tasks. More broadly, we show that combining personal AR interaction with shared displays is feasible for network navigation.

We explore the adaptation of 2D small-multiples visualisation on flat screens to 3D immersive spaces. We use a "shelves" metaphor for layout of small multiples and consider a design space across a number of layout and interaction dimensions. We demonstrate the applicability of a prototype system informed by this design space to data sets from different domains. We perform two user studies comparing the effect of the shelf curvature dimension from our design space on users’ ability to perform comparison and trend analysis tasks. Our results suggest that, with fewer multiples, a flat layout is more performant despite the need for participants to walk further. With an increase in the number of multiples, this performance difference disappears due to the time participants had to spend walking. In the latter case, users prefer a semi-circular layout over either a fully surrounding or a flat arrangement.

Three-dimensional scatterplots suffer from well-known perception and usability problems. In particular, overplotting and occlusion, mainly due to density and noise, prevent users from properly perceiving the data. Thanks to accurate head and hand tracking, immersive Virtual Reality (VR) setups provide new ways to interact and navigate with 3D scatterplots. VR also supports additional sensory modalities such as haptic feedback. Inspired by methods commonly used in Scientific Visualisation to visually explore volumes, we propose two techniques that leverage the immersive aspects of VR: first, a density-based haptic vibration technique (Scaptics) which provides feedback through the controller; and second, an adaptation of a cutting plane for 3D scatterplots (Highlight-Plane). We evaluated both techniques in a controlled study with two tasks involving density (finding high- and low-density areas). Overall, Scaptics was the most time-efficient and accurate technique, however, in some conditions, it was outperformed by Highlight-Plane.

We present FIESTA, a prototype system for collaborative immersive analytics (CIA). In contrast to many existing CIA prototypes, FIESTA allows users to collaboratively work together wherever and however they wish---untethered from mandatory physical display devices. Users can freely move around in a shared room-sized environment, author and generate immersive data visualisations, position them in the space around them, and share and communicate their insights to one another. Certain visualisation tasks are also supported to facilitate this process, such as details on demand and brushing and linking.

In immersive display environments, such as virtual or augmented reality, we can make explicit the connections between data points in visualisations and their context in the world, or in other visualisations. This paper considers the requirements and design space for drawing such links in order to minimise occlusion and clutter. A novel possibility in immersive environments is to optimise the link layout with respect to a particular point of view. In collaborative scenarios there is the need to do this for multiple points of view. We present an algorithm to achieve such link layouts and demonstrate its applicability in a variety of practical use cases.

In multi-display environments (MDEs) that include large shared displays and desktops, users can engage in both close collaboration and parallel or personal work. To transition between the displays can be challenging in complex settings, such as crisis management rooms. To provide workspace awareness and to factilitate these transitions, we design and implement three interactions techniques that display users' activities. We explore how and where to display this activity: briefly on the shared display, or more persistently on a peripheral floor display. In a user study, motivated by the context of a crisis room where multiple operators with different roles need to cooperate, we tested the usability of the techniques and provided insights on such transitions in systems running on MDEs.

Large vertical displays are considered well adapted for collaboration, due to their display surface and the space in front of them that can accommodate multiple people. However, there are few studies that empirically support this assertion, and they do not quantitatively assess the differences of collaboration in front of a shared display compared to a non-shared setup, such as multiple desktops with a common view. In this paper, we compare a large shared vertical display with two desktops, when pairs of users learn to perform a path-planning task. Our results did not indicate a significant difference in learning between the two setups, but found that participants adopted different task strategies. Moreover, while pairs were overall faster with the two desktops, quality was more consistent in the vertical shared display where pairs spent more time communicating, even though there is a-priori more implicit collaboration in this setup.

Wall-displays allow multiple users to simultaneously view and analyze large amounts of information, such as the increasingly complex graphs present in domains like biology or social network analysis. We focus on how pairs explore graphs on a touch enabled wall-display using two techniques, both adapted for collaboration: a basic localized selection, and a propagation selection technique that uses the idea of diffusion/transmission from an origin node. We assess in a controlled experiment the impact of selection technique on a shortest path identification task. Pairs consistently divided space even if the task is not spatially divisible, and for the basic selection technique that has a localized visual effect, it led to parallel work that negatively impacted accuracy. The large visual footprint of the propagation technique led to close coordination, improving speed and accuracy for complex graphs only. We then observed the use of propagation on additional graph topology tasks, confirming pair strategies on spatial division and coordination.

Road traffic control centers are of vital importance to modern cities. Interviews with controllers in two such centers identified the need to incorporate the visualization of results from predictive traffic models with real traffic, to help operators choose among different interventions on the network. We explore this idea in a prototype that runs on a wall display, and supports direct touch and input from workstations and mobile devices. Apart from basic functionality to manage the current traffic such as changing traffic light duration or speed limits, the prototype incorporates traffic simulations for forecasting results of possible actions, highlighting their differences to current traffic. Based on needs identified in our interviews, we offer two techniques that visually combine simulated and real situations, taking advantage of the large display space: multiple independent views and DragMagic, a variation of magic lenses. A preliminary laboratory experiment suggests that both techniques are viable design options, even for monitoring several simulations and areas of interest, contrary to expectations from previous work. However DragMagics are easier to master.