We characterize 16 challenges faced by those investigating and developing remote and synchronous collaborative experiences around visualization. Our work reflects the perspectives and prior research efforts of an international group of 29 experts from across human-computer interaction and visualization sub-communities. The challenges are anchored around five collaborative activities that exhibit a centrality of visualization and multimodal communication. These activities include exploratory data analysis, creative ideation, visualization-rich presentations, joint decision making grounded in data, and real-time data monitoring. The challenges also reflect the changing dynamics of these activities in the face of recent advances in extended reality (XR) and artificial intelligence (AI). As an organizing scheme for future research at the intersection of visualization and computer-supported cooperative work, we align the challenges with a sequence of four sets of research and development activities: technological choices, social factors, AI assistance, and evaluation.

Advances in emerging technologies, such as on-body mechanical actuators and electrical muscle stimulation, have allowed computers to take control over our bodies. This presents opportunities as well as challenges, raising fundamental questions about agency and the role of our bodies when interacting with technology. To advance this research field as a whole, we brought together expert perspectives in a week-long seminar to articulate the grand challenges that should be tackled when it comes to the design of computers’ control over our bodies. These grand challenges span technical, design, user, and ethical aspects. By articulating these grand challenges, we aim to begin initiating a research agenda that positions bodily control not only as a technical feature but as a central, experiential, and ethical concern for future human–computer interaction endeavors.

Contemporary museum experiences often incorporate digital media through modern technologies, such as handheld augmented reality (AR). However, these often fall short of providing a holistic visitor experience, as exhibits are still thought, designed, and experienced in isolation and fail to consider the user’s contexts (e. g., physical, social, and personal). To address this issue, we investigate leveraging a digital twin for designing and evaluating interactive exhibitions in large connected spaces through a case study: our publicly available handheld AR-based exhibition “Stayin’ Alive”. During this exhibition, we gained insights from the interaction data of 1303 visitors, post-visit interviews, as well as rich experiences and observations, based on which we identify four opportunity-challenge pairs that contribute design process insights for practitioners and a road map for future research.

We investigate hybrid user interfaces (HUIs), aiming to establish a cohesive understanding and to adopt consistent terminology for this nascent research area. HUIs combine heterogeneous devices in complementary roles, leveraging the distinct benefits of each. Our work focuses on cross-device interaction between 2D devices and mixed reality environments, which are particularly compelling, leveraging the familiarity of traditional 2D platforms while providing spatial awareness and immersion. Although prior work has prominently explored such HUIs in the context of mixed reality, we still lack a cohesive understanding of the unique design possibilities and challenges of such combinations, resulting in a fragmented research landscape. We conducted a systematic survey and present a taxonomy of HUIs that combine conventional display technology and mixed reality environments. Based on this, we discuss past and current challenges, the evolution of definitions, and prospective opportunities to tie together the past 30 years of research with our vision of future HUIs.

We introduce the SpatialMouse, a hybrid pointing device that combines the capabilities of a desktop mouse with the spatial input of a virtual reality (VR) controller, enabling seamless transitions between 2D and 3D interaction spaces in immersive mixed reality environments. Holistic usage scenarios in mixed reality involve tasks suited alternately to 2D or 3D information spaces. Yet, existing input devices excel in either 2D or 3D, but not both, making it necessary to switch between multiple input devices (e.g., mouse and VR controller). Our SpatialMouse addresses this issue, offering the affordances of a desktop mouse for indirect 2D pointing and the spatial capabilities of VR controllers with six degrees of freedom. In a user study with 12 participants, our prototype significantly reduced perceived task load and improved user experience compared to switching between separate devices. We extract design recommendations to further support such hybrid input approaches.

The manipulation of virtual 3D objects is essential for a variety of handheld AR scenarios. However, the mapping of commonly supported 2D touch gestures to manipulations in 3D space is not trivial. As an alternative, our work explores the use of haptic props that facilitate direct manipulation of virtual 3D objects with 6 degrees of freedom. In an experiment, we instructed 20 participants to solve 2D and 3D docking tasks in AR, to compare traditional 2D touch gestures with prop-based interactions using three prop shapes (cube, rhombicuboctahedron, sphere). Our findings highlight benefits of haptic props for 3D manipulation tasks with respect to task performance, user experience, preference, and workload. For 2D tasks, the benefits of haptic props are less pronounced. Finally, while we found no significant impact of prop shape on task performance, this appears to be subject to personal preference.

Smartphones provide convenient access to vast data collections (e.g., online shops, social media) within a compact, portable form factor. While the prevalent infinite scroll lists address the inherently restricted screen space, they also introduce navigation and orientation challenges. Users often lose track of their position within these lists and find it difficult to efficiently access, compare, and filter items of interest. To address this challenge, we introduce Push2AR, a novel interaction concept that extends the phone’s high-resolution display and familiar touch interaction with the virtual display space offered by Augmented Reality (AR) headsets. Push2AR enables users to transfer individual list items from their phone to its surrounding AR space, facilitating bookmarking, filtering, and side-by-side comparisons while maintaining orientation through visual links to the original scroll position. Our evaluation shows that our approach enhances user experience and reduces subjective workload involved in locating and comparing list items in contrast to conventional phone-only lists.

There is an increasing interest in combining interactive technology with food, leading to a new research area called human-food interaction. While food experiences are increasingly benefiting from interactive technology, for example in the form of food tracking apps, 3D-printed food and projections on dining tables, a more systematic advancement of the field is hindered because, so far, there is no comprehensive articulation of the grand challenges the field is facing. To further and consolidate conversations around this topic, we invited 21 HFI experts to a 5-day seminar. The goal was to review our own and prior work to identify the grand challenges in human-food interaction. The result is an articulation of 10 grand challenges in human-food interaction across 4 categories (technology, users, design and ethics). By presenting these grand challenges, we aim to help researchers move the human-food interaction research field forward.

Supporting remote collaboration through augmented reality facilitates the experience of co-presence by presenting the collaborator’s avatar in the user’s physical environment. While visual user representations are continuously researched and advanced, the audio configuration – especially in combination with different visualizations – is rarely considered. In a user study (n = 48, 24 dyads), we evaluate the combination of two visual (Simple vs. Rich Avatar) with two auditory (Mono vs. Spatial Audio) user representations, to investigate their impact on user’s overall experience, performance, and perceived social presence during collaboration. Our results show a preference for rich auditory and visual user representations, as Spatial Audio supports completion of parallel tasks and the Rich Avatar positively influence user experience. However, the Simple Avatar draws less attention, which potentially benefits task efficiency, advocating for simpler visualizations in performance-oriented settings. Our findings contribute to a deeper understanding of how visual and auditory user representation combinations impact remote collaboration in augmented reality.

Smartphones conveniently place large information spaces in the palms of our hands. While research has shown that larger screens positively affect spatial memory, workload, and user experience, smartphones remain fairly compact for the sake of device ergonomics and portability. Thus, we investigate the use of hybrid user interfaces to virtually increase the available display size by complementing the smartphone with an augmented reality head-worn display. We thereby combine the benefits of familiar touch interaction with the near-infinite visual display space afforded by augmented reality. To better understand the potential of virtually-extended displays and the possible issues of splitting the user’s visual attention between two screens (real and virtual), we conducted a within-subjects experiment with 24 participants completing navigation tasks using different virtually-augmented display sizes. Our findings reveal that a desktop monitor size represents a “sweet spot” for extending smartphones with augmented reality, informing the design of hybrid user interfaces.

In image quality assessment, a collective visual quality score for an image or video is obtained from the individual ratings of many subjects. One commonly used format for these experiments is the two-alternative forced choice method. Two stimuli with the same content but differing visual quality are presented sequentially or side-by-side. Subjects are asked to select the one of better quality, and when uncertain, they are required to guess. The relaxed alternative forced choice format aims to reduce the cognitive load and the noise in the responses due to the guessing by providing a third response option, namely, “not sure”. This work presents a large and comprehensive crowdsourcing experiment to compare these two response formats: the one with the “not sure” option and the one without it. To provide unambiguous ground truth for quality evaluation, subjects were shown pairs of images with differing numbers of dots and asked each time to choose the one with more dots. Our crowdsourcing study involved 254 participants and was conducted using a within-subject design. Each participant was asked to respond to 40 pair comparisons with and without the “not sure” response option and completed a questionnaire to evaluate their cognitive load for each testing condition. The experimental results show that the inclusion of the “not sure” response option in the forced choice method reduced mental load and led to models with better data fit and correspondence to ground truth. We also tested for the equivalence of the models and found that they were different. The dataset is available at http://database.mmsp-kn.de/cogvqa-database.html.

Peripheral vision plays a significant role in human perception and orientation. However, its relevance for human-computer interaction, especially head-mounted displays, has not been fully explored yet. In the past, a few specialized appliances were developed to display visual cues in the periphery, each designed for a single specific use case only. A multi-purpose headset to exclusively augment peripheral vision did not exist yet. We introduce MoPeDT: Modular Peripheral Display Toolkit, a freely available, flexible, reconfigurable, and extendable headset to conduct peripheral vision research. MoPeDT can be built with a 3D printer and off-the-shelf components. It features multiple spatially configurable near-eye display modules and full 3D tracking inside and outside the lab. With our system, researchers and designers may easily develop and prototype novel peripheral vision interaction and visualization techniques. We demonstrate the versatility of our headset with several possible applications for spatial awareness, balance, interaction, feedback, and notifications. We conducted a small study to evaluate the usability of the system. We found that participants were largely not irritated by the peripheral cues, but the headset's comfort could be further improved. We also evaluated our system based on established heuristics for human-computer interaction toolkits to show how MoPeDT adapts to changing requirements, lowers the entry barrier for peripheral vision research, and facilitates expressive power in the combination of modular building blocks.

Safety-critical interactive spaces for supervision and time-critical control tasks are usually characterized by many small displays and physical controls, typically found in control rooms or automotive, railway, and aviation cockpits. Using Virtual Reality (VR) simulations instead of a physical system can significantly reduce the training costs of these interactive spaces without risking real-world accidents or occupying expensive physical simulators. However, the user's physical interactions and feedback methods must be technologically mediated. Therefore, we conducted a within-subjects study with 24 participants and compared performance, task load, and simulator sickness during training of authentic aircraft cockpit manipulation tasks. The participants were asked to perform these tasks inside a VR flight simulator (VRFS) for three feedback methods (acoustic, haptic, and acoustic+haptic) and inside a physical flight simulator (PFS) of a commercial airplane cockpit. The study revealed a partial equivalence of VRFS and PFS, control-specific differences input elements, irrelevance of rudimentary vibrotactile feedback, slower movements in VR, as well as a preference for PFS.

The nascent field of mixed reality is seeing an ever-increasing need for user studies and field evaluation, which are particularly challenging given device heterogeneity, diversity of use, and mobile deployment. Immersive analytics tools have recently emerged to support such analysis in situ, yet the complexity of the data also warrants an ex-situ analysis using more traditional non-immersive visual analytics setups. To bridge the gap between both approaches, we introduce ReLive: a mixed-immersion visual analytics framework for exploring and analyzing mixed reality user studies. ReLive combines an in-situ virtual reality view with a complementary ex-situ desktop view. While the virtual reality view allows users to relive interactive spatial recordings replicating the original study, the synchronized desktop view provides a familiar interface for analyzing aggregated data. We validated our concepts in a two-step evaluation consisting of a design walkthrough and an empirical expert user study.

Handheld augmented reality (AR) applications allow users to interact with their virtually augmented environment on the screen of their tablet or smartphone by simply pointing its camera at nearby objects or “points of interest” (POIs). However, this often requires users to carefully scan their surroundings in search of POIs that are out of view. Proposed 2D guides for out-of-view POIs can, unfortunately, be ambiguous due to the projection of a 3D position to 2D screen space. We address this by using 3D visualizations that directly encode the POI’s 3D direction and distance. Based on related work, we implemented three such visualization techniques: (1) 3D Arrow, (2) 3D Bézier Curve, and (3) 3D Halos. We confirmed the applicability of these three techniques in a case study and then compared them in a user study, evaluating performance, workload, and user experience. Participants performed best using 3D Arrow, while surprisingly, 3D Halos led to poor results. We discuss the design implications of these results that can inform future 3D out-of-view object visualization techniques.

Augmented reality (AR) can create the illusion of being virtually co-located during remote collaboration, e.g., by visualizing remote co-workers as avatars. However, spatial awareness of each other's activities is limited as physical spaces, including the position of physical devices, are often incongruent. Therefore, alignment methods are needed to support activities on physical devices. In this paper, we present the concept of Re-locations, a method for enabling remote collaboration with augmented reality in incongruent spaces. The idea of the concept is to enrich remote collaboration activities on multiple physical devices with attributes of co-located collaboration such as spatial awareness and spatial referencing by locally relocating remote user representations to user-defined workspaces. We evaluated the Re-locations concept in an explorative user study with dyads using an authentic, collaborative task. Our findings indicate that Re-locations introduce attributes of co-located collaboration like spatial awareness and social presence. Based on our findings, we provide implications for future research and design of remote collaboration systems using AR.

Nurses frequently transfer patients as part of their daily work. However, manual patient transfers pose a major risk to nurses’ health. Although the Kinaesthetics care conception can help address this issue, existing support to learn the concept is low. We present KiTT, a tablet-based system, to promote the learning of ergonomic patient transfers based on the Kinaesthetics care conception. KiTT supports the training of Kinaesthetics-based patient transfers by two nurses. The nurses are guided by the phases (i) interactive instructions, (ii) training of transfer conduct, and (iii) feedback and reflection. We evaluated KiTT with 26 nursing-care students in a nursing-care school. Our results indicate that KiTT provides a good subjective support for the learning of Kinaesthetics. Our results also suggest that KiTT can promote the ergonomically correct conduct of patient transfers while providing a good user experience adequate to the nursing-school context, and reveal how KiTT can extend existing practices.

Recent research in the area of immersive analytics demonstrated the utility of head-mounted augmented reality devices for visual data analysis. However, it can be challenging to use the by default supported mid-air gestures to interact with visualizations in augmented reality (e.g. due to limited precision). Touch-based interaction (e.g. via mobile devices) can compensate for these drawbacks, but is limited to two-dimensional input. In this work we present STREAM: Spatially-aware Tablets combined with Augmented Reality Head-Mounted Displays for the multimodal interaction with 3D visualizations. We developed a novel eyes-free interaction concept for the seamless transition between the tablet and the augmented reality environment. A user study reveals that participants appreciated the novel interaction concept, indicating the potential for spatially-aware tablets in augmented reality. Based on our findings, we provide design insights to foster the application of spatially-aware touch devices in augmented reality and research implications indicating areas that need further investigation.

Augmented Reality (AR) supported collaboration is a popular topic in HCI research. Previous work has shown the benefits of collaborative 3D object manipulation and identified two possibilities: Either separate or compose users’ inputs. However, their experimental comparison using handheld AR displays is still missing. We, therefore, conducted an experiment in which we tasked 24 dyads with collaboratively positioning virtual objects in handheld AR using three manipulation techniques: 1) Separation – performing only different manipulation tasks (i. e., translation or rotation) simultaneously, 2) Composition – performing only the same manipulation tasks simultaneously and combining individual inputs using a merge policy, and 3) Hybrid – performing any manipulation tasks simultaneously, enabling dynamic transitions between Separation and Composition. While all techniques were similarly effective, Composition was least efficient, with higher subjective workload and worse user experience. Preferences were polarized between clear work division (Separation) and freedom of action (Hybrid). Based on our findings, we offer research and design implications.

Large Interactive Displays (LIDs), such as tabletops or interactive walls, are promising innovations, which are increasingly used to support co-located collaboration. Yet the current evidence base on the impact of LID use on collaborative processes and outcomes, and associated influencing factors, is fragmented, particularly in comparison with other media. To address this gap, a systematic review was carried out in the databases Web of Science, Psych.Info, ACM, Elsevier, JSTOR and Springer and in the ACM CHI conference database. A corpus of 38 articles with experimental study designs met the eligibility criteria and was analyzed in-depth. With regard to collaboration processes, the findings suggest a relatively clear advantage of the use of LIDs over classic forms of collaboration, in particular over single-user environments (e.g. laptops). With attention to collaborative outcomes, positive effects of LIDs were identified for knowledge gains and social encounters, and mixed effects for task-related outcomes. The analysis further shows relevant influencing factors of LID, such as the separation of personal and joint work spaces and the deployment of horizontal instead of vertical displays. Conceptual and practice implications are discussed.

Nurses are frequently required to transfer patients as part of their daily duties. However, the manual transfer of patients is a major risk factor for injuries to the back. Although the Kinaesthetics Care Conception can help to address this issue, existing support for the integration of the concept into nursing-care practice is low. We present NurseCare, a mobile system that aims to promote the practical application of ergonomic patient transfers based on the Kinaesthetics Care Conception. NurseCare consists of a wearable and a smartphone app. Key features of NurseCare include mobile accessible instructions for ergonomic patient transfers, in-situ feedback for the risky bending of the back, and long-term feedback. We evaluated NurseCare in a nine participant 'in-the-wild' evaluation. Results indicate that NurseCare can facilitate ergonomic work while providing a high user experience adequate to the nurses' work domain, and reveal how NurseCare can be incorporated in given practices.

Human-Computer Integration (HInt) is an emerging paradigm in which computational and human systems are closely interwoven. Integrating computers with the human body is not new. however, we believe that with rapid technological advancements, increasing real-world deployments, and growing ethical and societal implications, it is critical to identify an agenda for future research. We present a set of challenges for HInt research, formulated over the course of a five-day workshop consisting of 29 experts who have designed, deployed and studied HInt systems. This agenda aims to guide researchers in a structured way towards a more coordinated and conscientious future of human-computer integration.

Cross-device interaction with tablets is a popular topic in HCI research. Recent work has shown the benefits of including multiple devices into users' workflows while various interaction techniques allow transferring content across devices. However, users are only reluctantly using multiple devices in combination. At the same time, research on cross-device interaction struggles to find a frame of reference to compare techniques or systems. In this paper, we try to address these challenges by studying the interplay of interaction techniques, device utilization, and task-specific activities in a user study with 24 participants from different but complementary angles of evaluation using an abstract task, a sensemaking task, and three interaction techniques. We found that different interaction techniques have a lower influence than expected, that work behaviors and device utilization depend on the task at hand, and that participants value specific aspects of cross-device interaction.

Programming assignments in computer science courses are often processed in pairs or groups of students. While working together, students face several shortcomings in today's software: The lack of real-time collaboration capabilities, the setup time of the development environment, and the use of different devices or operating systems can hamper students when working together on assignments. Text processing platforms like Google Docs solve these problems for the writing process of prose text, and computational notebooks like Google Colaboratory for data analysis tasks. However, none of these platforms allows users to implement interactive applications. We deployed a web-based literate programming system for three months during an introductory course on application development to explore how collaborative programming practices unfold and how the structure of computational notebooks affect the development. During the course, pairs of students solved weekly programming assignments. We analyzed data from weekly questionnaires, three focus groups with students and teaching assistants, and keystroke-level log data to facilitate the understanding of the subtleties of collaborative programming with computational notebooks. Findings reveal that there are distinct collaboration patterns; the preferred collaboration pattern varied between pairs and even varied within pairs over the course of three months. Recognizing these distinct collaboration patterns can help to design future computational notebooks for collaborative programming assignments.

Mid-air arm movements are important for various activities. However, common resources for their self-directed practice require practitioners to divide their focus between an external source (e.g., a video screen) and moving. Past research found benefits for egocentric guidance visualizations compared to common resources. However, there is limited evidence about how such visualizations should look and behave. EGuide supports the investigation of different egocentric visualizations for the guidance of mid-air arm movements. We compared two visual appearances for egocentric guidance visualizations that differ in their shape (look), and three guidance techniques that differ by how they guide a user (behavior). For visualizations with a continuously moving guidance technique, our results suggest a higher movement accuracy for a realistic than an abstract shape. For user experience and preference, our results suggest that visualizations with an abstract shape and a guidance technique that visualizes important postures should not pause at important postures.

Building data analysis skills is part of modern elementary school curricula. Recent research has explored how to facilitate children's understanding of visual data representations through completion exercises which highlight links between concrete and abstract mappings. This approach scaffolds visualization activities by presenting a target visualization to children. But how can we engage children in more free-form visual data mapping exercises that are driven by their own mapping ideas? How can we scaffold a creative exploration of visualization techniques and mapping possibilities? We present Construct-A-Vis, a tablet-based tool designed to explore the feasibility of free-form and constructive visualization activities with elementary school children. Construct-A-Vis provides adjustable levels of scaffolding visual mapping processes. It can be used by children individually or as part of collaborative activities. Findings from a study with elementary school children using Construct-A-Vis individually and in pairs highlight the potential of this free-form constructive approach, as visible in children's diverse visualization outcomes and their critical engagement with the data and mapping processes. Based on our study findings we contribute insights into the design of free-form visualization tools for children, including the role of tool-based scaffolding mechanisms and shared interactions to guide visualization activities with children.

Analysing and understanding brain function and disorder is the main focus of neuroscience. Due to the high complexity of the brain, directionality of the signal and changing activity over time, visual exploration and data analysis are difficult. For this reason, a vast amount of research challenges are still unsolved. We explored different challenges of the visual analysis of brain data and the design of corresponding immersive environments in collaboration with experts from the biomedical domain. We built a prototype of an immersive virtual reality environment to explore the design space and to investigate how brain data analysis can be supported by a variety of design choices. Our environment can be used to study the effect of different visualisations and combinations of brain data representation, as for example network layouts, anatomical mapping or time series. As a long-term goal, we aim to aid neuro-scientists in a better understanding of brain function and disorder.

Immersive technologies such as augmented reality devices are opening up a new design space for the visual analysis of data. This paper studies the potential of an augmented reality environment for the purpose of collaborative analysis of multidimensional, abstract data. We present ART, a collaborative analysis tool to visualize multidimensional data in augmented reality using an interactive, 3D parallel coordinates visualization. The visualization is anchored to a touch-sensitive tabletop, benefiting from well-established interaction techniques. The results of group-based, expert walkthroughs show that ART can facilitate immersion in the data, a fluid analysis process, and collaboration. Based on the results, we provide a set of guidelines and discuss future research areas to foster the development of immersive technologies as tools for the collaborative analysis of multidimensional data.

PolarTrack is a novel camera-based approach to detecting and tracking mobile devices inside the capture volume. In PolarTrack, a polarization filter continuously rotates in front of an off-the-shelf color camera, which causes the displays of observed devices to periodically blink in the camera feed. The periodic blinking results from the physical characteristics of current displays, which shine polarized light either through an LC overlay to produce images or through a polarizer to reduce light reflections on OLED displays. PolarTrack runs a simple detection algorithm on the camera feed to segment displays and track their locations and orientations, which makes PolarTrack particularly suitable as a tracking system for cross-device interaction with mobile devices. Our evaluation of PolarTrack's tracking quality and comparison with state-of-the-art camera-based multi-device tracking showed a better tracking accuracy and precision with similar tracking reliability. PolarTrack works as standalone multi-device tracking but is also compatible with existing camera-based tracking systems and can complement them to compensate for their limitations.

Previous research shows that territories help people coordinate their task and social interaction at large interactive tabletops. However, little is known about the interplay between territorially and the reorientation of digital objects and their influence on task performance. In this paper, we advance the hypothesis that territories are states of spatial arrangements continually changing during the collaborative activity and seek to better understand their role as a main mechanism in coordinating group activities. We report results from an explorative tabletop study that compares two types of technical settings workspaces supporting a brainstorming task. Our results show evidence of different territorial strategies dependent on the two conditions. We discuss the role of territoriality and orientation of digital notes as a mechanism for coordinating group activity and their influence on task performance and outcome. Finally, we present design recommendations derived from our findings.

"Rebuild Palmyra?" is a multimedia exhibition about the ancient city of Palmyra, its destruction by Daesh, and the question of whether it should be rebuilt. As such, it tackles today's pressing question of how humanity should deal with the destruction of cultural heritage. In the design of the exhibition, we pursued the Blended Museum approach, in which we strive to seamlessly integrate interactive media into exhibition design to increase the overall visitor experience. In this work, we present the exhibition, which consists of four rooms. We focus on three interactive installations in which the topic of reconstructing Palmyra is mediated using new technologies such as 3D Printing, Augmented Reality, and Virtual Reality. The installations helped visitors in developing their own point of view on the question of rebuilding Palmyra. Lastly, we provide insights into the technical implementation of the installations and discuss the results of quantitative and qualitative evaluations.

HCI research has demonstrated Mixed Reality (MR) as being beneficial for co-located collaborative work. For remote collaboration, however, the collaborators' visual contexts do not coincide due to their individual physical environments. The problem becomes apparent when collaborators refer to physical landmarks in their individual environments to guide each other's attention. In an experimental study with 16 dyads, we investigated how the provisioning of shared virtual landmarks (SVLs) influences communication behavior and user experience. A quantitative analysis revealed that participants used significantly less ambiguous spatial expressions and reported an improved user experience when SVLs were provided. Based on these findings and a qualitative video analysis we provide implications for the design of MRs to facilitate remote collaboration.

A sizable part of HCI research on cross-device interaction is driven by the vision of users conducting complex knowledge work seamlessly across multiple mobile devices. This is based on the Weiserian assumption that people will be inclined to distribute their work across multiple ``pads' if such are available. We observed that this is not the reality today, even when devices were in abundance. We present a study with 24 participants in 12 dyads completing a collaborative visualization task with up to six tablets. They could choose between three different visualization types to answer questions about economic data. Tasks were designed to afford simultaneous use of tablets, either with linked or independent views. We found that users typically utilized only one tablet per user. A quantitative and qualitative analysis revealed a ``legacy bias' that introduced barriers for using more tablets and reduced the overall benefit of multi-device visualization.

People's ability to remember and recall spatial information can be harnessed to improve navigation and search performances in interactive systems. In this paper, we investigate how display size and input modality influence spatial memory, especially in relation to efficiency and user satisfaction. Based on an experiment with 28 participants, we analyze the effect of three input modalities (trackpad, direct touch, and gesture-based motion controller) and two display sizes (10.6" and 55") on people's ability to navigate to spatially spread items and recall their positions. Our findings show that the impact of input modality and display size on spatial memory is not straightforward, but characterized by trade-offs between spatial memory, efficiency, and user satisfaction.

Augmented Reality (AR) displays have been suggested as shared-space technology to support remote collaboration, e.g., in design and building tasks. But with AR displays, the shared space typically consists of only the virtual work objects (e.g., design artifacts) while collaborators' interaction is grounded in their individual, physical environment. This can become problematic during activities that involve the positioning of virtual objects because the collaborators may require shared spatial references to coordinate their actions. In a lab experiment with 16 dyads, we studied how collaborators deal with that issue, and whether the provisioning of additive, virtual landmarks influences collaboration. As a result the landmarks improved user experience and decreased the reported temporal demand. In addition, we identified task-specific problem situations and provide implications for the design of distributed ARs to facilitate the collaborative positioning of virtual objects.

In recent years, research on cross-device interaction has become a popular topic in HCI leading to novel interaction techniques mutually interfering with new evolving theoretical paradigms. Building on previous research, we implemented an individual multi-device work environment for creative activities. In a study with 20 participants, we compared a traditional toolbar-based condition with two conditions facilitating spatially distributed tools on digital panels and on physical devices. We analyze participants' interactions with the tools, encountered problems and corresponding solutions, as well as subjective task load and user experience. Our findings show that the spatial distribution of tools indeed offers advantages, but also elicits new problems, that can partly be leveraged by the physical affordances of mobile devices.

Distortion-based visualization techniques allow users to examine focused regions of a multiscale space at high scales but preserve their contextual information. However, the distortion can come at the coast of confusion, disorientation and impairment of the users' spatial memory. Yet, how distortions influence users' ability to build up spatial memory, while taking into account human skills of perception, interpretation and comprehension, remains underexplored. This note reports findings of an experimental comparison between a distortion-based focus+context interface and an undistorted overview+detail interface. The focus+context technique follows guidelines for the design of comprehensible distortions: make use of real-world metaphors, visual clues like shading, smooth transitions and scaled-only focus regions. The results show that the focus+context technique designed following these guidelines help to keep track of the position within the multiscale space and does not impair users' spatial memory.

OPAC interfaces, still the dominant access point to library catalogs, support systematic search but are problematic for open-ended exploration and generally unpopular with visitors. As a result, libraries start subscribing to simplified search paradigms as exemplified by web-search systems. This is a problem considering that systematic search is a crucial skill in the light of today's abundance of digital information. Inspired by novel approaches to facilitating search, we designed CollectionDiver, an installation for supporting systematic search in public libraries. The CollectionDiver combines tangible and large display direct-touch interaction with a visual representation of search criteria and filters. We conducted an in-situ qualitative study to compare participants' search approaches on the CollectionDiver with those on the OPAC interface. Our findings show that while both systems support a similar search process, the CollectionDiver (1) makes systematic search more accessible, (2) motivates proactive search approaches by (3) adding transparency to the search process, and (4) facilitates shared search experiences. We discuss the CollectionDiver's design concepts to stimulate new ideas toward supporting engaging approaches to systematic search in the library context and beyond.

In collaborative activities, collaborators can use physical objects in their shared environment as spatial cues to guide each other's attention. Collaborative mixed reality environments (MREs) include both, physical and digital objects. To study how virtual objects influence collaboration and whether they are used as spatial cues, we conducted a controlled lab experiment with 16 dyads. Results of our study show that collaborators favored the digital objects as spatial cues over the physical environment and the physical objects: Collaborators used significantly less deictic gestures in favor of more disambiguous verbal references and a decreased subjective workload when virtual objects were present. This suggests adding additional virtual objects as spatial cues to MREs to improve user experience during collaborative mixed reality tasks.

Cross-device collaboration with tablets is an increasingly popular topic in HCI. Previous work has shown that tablet-only collaboration can be improved by an additional shared workspace on an interactive tabletop. However, large tabletops are costly and need space, raising the question to what extent the physical size of shared horizontal surfaces really pays off. In order to analyse the suitability of smaller-than-tabletop devices (e.g. tablets) as a low-cost alternative, we studied the effect of the size of a shared horizontal interactive workspace on users' attention, awareness, and efficiency during cross-device collaboration. In our study, 15 groups of two users executed a sensemaking task with two personal tablets (9.7") and a horizontal shared display of varying sizes (10.6", 27", and 55"). Our findings show that different sizes lead to differences in participants' interaction with the tabletop and in the groups' communication styles. To our own surprise we found that larger tabletops do not necessarily improve collaboration or sensemaking results, because they can divert users' attention away from their collaborators and towards the shared display.

In this workshop, we will review and discuss open issues, technical challenges and conceptual models for multi-device spatial or proxemic interaction. We aim to bring together researchers, students and practitioners working on technical infrastructures, studies and designs of spatial in-terfaces, or domain specific multi-device applications that use space as a unit of analysis. We focus specifically on analysing how such interfaces, tools and tracking technolo-gy can be deployed "in the wild". The workshop will facili-tate knowledge exchange about the current state of spatial and proxemic interactive systems, identify application do-mains and enabling technologies for cross-surface interac-tions in the wild, and establish a research community to develop effective strategies for successful design of cross-device interactions.

Dynamic peephole navigation represents a technique for navigating large information spaces in an egocentric way. Studies have shown cognitive benefits for a vertical peephole orientation, when compared to non-egocentric interaction styles. To see how the aspect of canvas orientation effects user performance, we conducted a study (N=16) which revealed that canvas orientation has no significant effect on either navigation performance or spatial memory. We also found a significantly lower physical demand and a higher mental demand in the horizontal orientation. For short-term activities we therefore propose a vertical orientation, while for long-term activities horizontal dynamic peephole navigation is more suitable.

Cross-device interaction between multiple mobile devices is a popular field of research in HCI. However, the appropriate design of this interaction is still an open question, with competing approaches such as spatially-aware vs. spatially-agnostic techniques. In this paper, we present the results of a two-phase user study that explores this design space: In phase 1, we elicited gestures for typical mobile cross-device tasks from 4 focus groups (N=17). The results show that 71% of the elicited gestures were spatially-aware and that participants strongly associated cross-device tasks with interacting and thinking in space. In phase 2, we implemented one spatially-agnostic and two spatially-aware techniques from phase 1 and compared them in a controlled experiment (N=12). The results indicate that spatially-aware techniques are preferred by users and can decrease mental demand, effort, and frustration, but only when they are designed with great care. We conclude with a summary of findings to inform the design of future cross-device interactions.

Many tasks performed in multiscale visual spaces require the user to have several foci. Using bimanual interaction, multitouch devices can facilitate the simultaneous definition and exploration of several foci. However, multitouch is rarely used for multifocus navigation, and may limit the interaction to a sequential definition of areas of interest. We introduce two novel navigation techniques that combine multiple foci and bimanual touch, and therefore enable the isochronic definition of areas of interest, leading to simultaneous multifocus navigation. SpaceFold folds the visual space in the third dimension, allowing users to bring objects closer to each other. Our technique enables a direct, bimanual manipulation of a folded space and therefore provides high flexibility. PhysicLenses uses multiple magnification lenses to compare objects. Using a physics model, PhysicLenses introduces a general solution for the arrangement of multiple lenses within the viewport. We conducted a controlled experiment with 24 participants to compare the techniques with split screen. The results show that SpaceFold significantly outperformed all other techniques, whereas PhysicLenses was just as fast as split screen.

In today's digital control rooms, desktop computers represent the most common interface for process control. Compared to their predecessors -- manual control actuators -- desktop computers enable quick and effective process intervention but they lack in process-related interaction qualities such as haptic feedback and the involvement of motor skills. Thus, design trade-offs have to be made to combine the strengths of both paradigms: today's processing power with the interaction qualities of former control room interfaces. In this paper related interaction concepts are presented and evaluated. In a control room scenario, participants were tasked with adjusting numerical values -- so-called process variables -- under two traditional conditions (mouse, keyboard) and two post-WIMP conditions (touch, tangible). Task completion time and recall accuracy of the adjusted values were measured. As a result, traditional desktop interaction proved to be faster, whereas control actions could be recalled significantly better using the tangible control elements. We therefore suggest providing both tangible control for process maintenance and traditional desktop interaction in critical situations that require quick intervention.

Dynamic peephole navigation is an increasingly popular technique for navigating large information spaces such as maps. Users can view the map through handheld, spatially aware displays that serve as peepholes and navigate the map by moving these displays in physical space. We conducted a controlled experiment of peephole map navigation with 16 participants to better understand the effect of a peephole's size on users' map navigation behavior, navigation performance, and task load. Simulating different peephole sizes from 4' (smartphone) up to 120' (control condition), we confirmed that larger peepholes significantly improve learning speed, navigation speed, and reduce task load; however, this added benefit diminishes with growing sizes. Our data shows that a relatively small, tablet-sized peephole can serve as a 'sweet spot' between peephole size and both user navigation performance and user task load.

Interactive spaces are ubiquitous computing environments for computer-supported collaboration that exploit and enhance the existing cognitive, physical and social skills of users or groups of users. The workshop aims at documenting and advancing the current state-of-the-art of co-located collaboration in interactive spaces and identifying research challenges and formulating a research agenda by inviting high-quality position and research papers from HCI, Information Visualization, CSCW and CSCL.

Recent findings from Embodied Cognition reveal strong effects of arm and hand movement on spatial memory. This suggests that input devices may have a far greater influence on users' cognition and users' ability to master a system than we typically believe -- especially for spatial panning or zooming & panning user interfaces. We conducted two experiments to observe whether multi-touch instead of mouse input improves users' spatial memory and navigation performance for such UIs. We observed increased performances for panning UIs but not for zooming & panning UIs. We present our results, provide initial explanations and discuss opportunities and pitfalls for interaction designers.

Context-sensitive information is of vital importance to the monitoring of processes in control rooms, but its incorporation in current state-of-the-art user interfaces is inadequate. By looking at a traffic control center as an example, this paper proposes an interaction concept for monitoring complex networks on large remote displays. Our approach is based on content-aware navigation with the goal to improve the navigation of the road network and the availability of context-sensitive information. We conducted a study comparing different navigation techniques as well as two techniques for visualizing context-sensitive information.

This paper describes the perspective+detail visualization concept, which extends the conventional overview+detail pattern by adding a perspective viewing area and a further, text-based area containing partial details. Known problems of such extensions are mitigated using a vertically curved display. Our concept aims to bridge both the geographical and the content gap between the two main display areas and to provide an improved overview. An experimental case study based on a typical traffic control room operator task provided first insights on our visualization. These initial findings revealed that perspective+detail assists users in fulfilling their tasks and keeping their orientation in the information space.

“Interactive spaces” are physical environments or rooms for collaborative work that are augmented with ubiquitous computing technology. Their purpose is to enable a computer-supported collaboration between multiple users that is based on a seamless use of different devices for natural “post-WIMP” interaction (e.g., multitouch walls, interactive tabletops, tablet PCs, or digital pen and paper). However, to this day, there are no well-established guidelines or toolkits for designing and implementing such distributed user interfaces (DUIs). Therefore, this article introduces the Zoomable Object-Oriented Information Landscape (ZOIL), a novel design approach and software framework for post-WIMP DUIs in interactive spaces. In the following, the ZOIL design principles are first introduced and illustrated. They provide recommendations and examples of DUI interaction design for interactive spaces. Then the different software patterns and architectures that have been employed for implementing the open-source ZOIL software framework are described. This framework facilitates the implementation of ZOIL's design principles in practice. Lessons learned from ZOIL's implementation are shared, and the implementation is discussed and compared with related work and approaches. The results of an evaluation of ZOIL with designers and developers conclude the article.

Using affinity diagramming as an example, we investigate reality-based interfaces for supporting creative group work. Based on an observational study grounded in the reality-based interaction framework, we identified power vs. reality tradeoffs that can be addressed to find a close fit to embodied practice. Using this knowledge, we designed and implemented a digital workspace for supporting affinity diagramming. Its hybrid interaction techniques combine digital pen & paper with an interactive table and tangible tokens. An additional vertical display is used to support reflection-in-action and for enhancing discussion and coordination. A preliminary user study confirmed the applicability of our tradeoffs and the general acceptance of the tool design.

Application programming interfaces (APIs) are the interfaces to existing code structures, such as widgets, frameworks, or toolkits. Therefore, they very much do have an impact on the quality of the resulting system. So, ensuring that developers can make the most out of them is an important challenge. However standard usability evaluation methods as known from HCI have limitations in grasping the interaction between developer and API as most IDEs (essentially the GUI) capture only part of it. In this paper we present the Concept Map method to study the usability of an API over time. This allows us to elicit the mental model of a programmer when using an API and thereby identify usability issues and learning barriers and their development over time.

We introduce "Facet-Streams", a hybrid interactive surface for co-located collaborative product search on a tabletop. Facet-Streams combines techniques of information visualization with tangible and multi-touch interaction to materialize collaborative search on a tabletop. It harnesses the expressive power of facets and Boolean logic without exposing users to complex formal notations. Two user studies reveal how Facet-Streams unifies visual and tangible expressivity with simplicity in interaction, supports different strategies and collaboration styles, and turns product search into a fun and social experience.

Pen and paper support the rapid production of sketches. However, the paper interface is not always optimal for collaborative sketching as seen in brainstorming sessions where multiple parties would often like to communicate and participate in the sketching synchronously. Novel interactive paper solutions may provide the answer by bridging the paper-digital divide and allowing users to sketch on paper simultaneously while capturing the actions digitally. We present an analysis of collaborative sketching activities in working environments with remote participation. After highlighting the importance of paper for natural interaction in these settings, we introduce PaperSketch, an interactive paper-digital tool for collaborative remote sketching. We discuss the collaborative development of ideas based on the prototype and outline how important feedback issues have been addressed by utilising spatial constraints and multimodal features.

Cross-display interaction is a challenge for HCI researchers, because a naive adaptation of mechanisms in single-display workspaces might be inappropriate for teamwork in multi-user multi-display environments. In this paper, we identify a set of primitive cross-display interaction tasks, which, so far, have not been addressed in the literature. Additionally, we present our ideas to design a prototype that incorporates the identified primitive tasks, using an iPhone as a mobile input device with an integrated display.

Selecting a graphical item by pointing with a computer mouse is a ubiquitous task in many graphical user interfaces. Several techniques have been suggested to facilitate this task, for instance, by reducing the required movement distance. Here we measure the natural coordination of eye and mouse pointer control across several search and selection tasks. We find that users automatically minimize the distance to likely targets in an intelligent, task dependent way. When target location is highly predictable, top-down knowledge can enable users to initiate pointer movements prior to target fixation. These findings question the utility of existing assistive pointing techniques and suggest that alternative approaches might be more effective.

MedioVis is a visual information seeking system that aims to support users' natural seeking behavior, particularly in complex information spaces. To achieve this goal we introduce multiple complementary visualization techniques together with an easy-to-use and consistent interaction concept. Over the last four years, MedioVis was developed in the context of digital libraries following a user-centered design process. The focus of this paper is the presentation of our interaction model and further to give an overview of the applied visualization techniques.

We employ Laserpointer-Interaction as an intuitive, direct and flexible interaction concept particularly for large, highresolution displays which enforce physical navigation. We therefore demonstrate general applicability and suitability by applying Laserpointer-Interaction to a wide range of domains -- from scientific applications and formal experiments to artistic installations for the broad public.

Previous research has shown that alphasliders are an effective tool for searching an alphabetically sorted list when only limited screen space is available for the graphical user interface. To improve user satisfaction, we propose equipping the widget with a novel text filter to dynamically limit the slider range. In this way, users are supported in locating target items and in identifying records that are missing. The results of a comparative user evaluation run on a personal digital assistant showed that 8 out of 12 participants preferred the filter widget to the classic interface. We further suggest an enhanced alphaslider design to speed up user interaction.

While zoomable user interfaces can improve the usability of applications by easing data access, a drawback is that some users tend to become lost after they have zoomed in. Previous studies indicate that this effect could be related to individual differences in spatial ability. To overcome such orientation problems, many desktop applications feature an additional overview window showing a miniature of the entire information space. Small devices, however, have a very limited screen real estate and incorporating an overview window often means pruning the size of the detail view considerably. Given this context, we report the results of a user study in which 24 participants solved search tasks by using two zoomable scatterplot applications on a PDA - one of the applications featured an overview, the other relied solely on the detail view. In contrast to similar studies for desktop applications, there was no significant difference in user preference between the interfaces. On the other hand, participants solved search tasks faster without the overview. This indicates that, on small screens, a larger detail view can outweigh the benefits gained from an overview window. Individual differences in spatial ability did not have a significant effect on task-completion times although results suggest that participants with higher spatial ability were slowed down by the overview more than low spatial-ability users.

Existing information-visualization techniques that target small screens are usually limited to exploring a few hundred items. In this article we present a scatterplot tool for Personal Digital Assistants that allows the handling of many thousands of items. The application's scalability is achieved by incorporating two alternative interaction techniques: a geometric-semantic zoom that provides smooth transition between overview and detail, and a fisheye distortion that displays the focus and context regions of the scatterplot in a single view. A user study with 24 participants was conducted to compare the usability and efficiency of both techniques when searching a book database containing 7500 items. The study was run on a pen-driven Wacom board simulating a PDA interface. While the results showed no significant difference in task-completion times, a clear majority of 20 users preferred the fisheye view over the zoom interaction. In addition, other dependent variables such as user satisfaction and subjective rating of orientation and navigation support revealed a preference for the fisheye distortion. These findings partly contradict related research and indicate that, when using a small screen, users place higher value on the ability to preserve navigational context than they do on the ease of use of a simplistic, metaphor-based interaction style.

ZuiScat is a visualization concept for querying large information spaces on Personal Digital Assistants (PDA). Retrieval results are presented in a dynamic scatterplot, which is enhanced by geometric and semantic zoom techniques to provide smooth transitions from abstract visual encodings to data content. The same visualization is also used to manage bookmarks and to serve as a powerful query history tool. User feedback suggests that ZuiScat provides intuitive and efficient data access but still needs further improvement in terms of zoom usability and visual mapping.

We present a new framework for metadata visualisation systems called VisMeB (visual metadata browser). It is based upon redesign ideas from the IN-SYDER system that were come under extensive evaluations. The aim of our approach is to improve the process of finding relevant information in an intuitive yet multifunctional way. We use a ScatterPlot in combination with a so called SuperTable for visualisation. The two techniques are tightly coupled and present unique possibilities of interaction through the use of visual filters.

We present a new visualization approach for metadata combining different visualizations into a so-called SuperTable accompanied by a Scatterplot. The goal is to improve user experience during the information seeking process. Our new visualizations are based on our experiences developing a visual information retrieval system called INSYDER to supply small and medium size enterprises with business information front the Internet. Based on extensive user tests the original visualizations have been redesigned in two different design variants. Instead of offering multiple visualizations to choose front the SuperTable + Scatterplot combines them in a new way. Therefore, the user has the feeling that he is working with one single visualization in different states. Further the SuperTable solves a problem which seemed to be immanent to visualization's in document retrieval: the change of modalities.

The WWW is the most important resource for external business information. This paper presents a tool called INSYDER, an information assistant for finding and analysis business information from the WWW. INSYDER is a system using different agents for crawling the Web, evaluating and visualising the results. These agents, the used visualisations, and a first summary of user studies are presented.

Abstract. There is an increasing need for practical and comprehensive evaluation methods and tools for conformance testing with ISO standards. In this study, we focus on ISO 9241 which is an important ergonomic standard. A brief description shows its content and structure. Practical evaluations include the amount of time and resources which must be managed in software projects, while comprehensive evaluations require that the context of use be considered during the evaluation of user interfaces. In order to complete a comprehensive evaluation of usability, it is necessary to use more than one evaluation method. Therefore, an overview of diŒerent evaluation approaches is given, describing their advantages and dis-advantages. Finally, the ISO 9241 evaluator is presented in detail as an example of a practical expert-based evaluation method for conformance testing with the ISO 9241 standard, that can be integrated in a comprehensive evaluation approach. 1.

As a result of the importance of the usability approach in system development and the EC's ‘Directive concerning the minimum safety and health requirements for VDT workers’ (EWG 1990), there is an accepted need for practical evaluation methods for user interfaces. The usability approach and the EC Directive are not restricted to user interface design, as they include the design of appropriate hardware and software, as well as organization, job, and task design. Therefore system designers are faced with many, often conflicting, requirements and need to address the question, ‘How can usability requirements comprehensively be considered and evaluated in system development?’ Customers buying hardware and software and introducing them into their organization ask, (How can I select easy-to-use hardware and software?’ Both designers and customers need an evaluation procedure that covers all the organizational, user, hard- and software requirements. The evaluation method, EVADIS.II, we present in this paper overcomes characteristic deficiencies of previous evaluation methods. In particular, it takes the tasks, the user, and the organizational context into consideration during the evaluation process, and provides computer support for the use of the evaluation procedure.