Augmented displays (ADs), where AR extends the workspace of a physical display, are gaining momentum across productivity, visual analytics, and collaborative scenarios; yet, the field lacks a unified conceptual foundation. We present the first scoping survey on ADs, synthesizing 62 papers (2010-2024), including one earlier paper from 2005. From the corpus, we derive the AR-Display Spatial Integration Framework, capturing recurring patterns across AD systems along four core dimensions: extension type, AR content type, placement, and layout. We map existing systems across the framework to identify design patterns and translate them into recommendations. We further consolidate insights on development and evaluation practices, followed by a discussion on using the framework, AD applications, cross-cutting factors, and managing the complexity of AD systems. Our survey also outlines research gaps for advancing the field, particularly in the design space of AR-Display integration and the broader support and use of AD.

The most common learning materials for handcraft today are videos and figures, which are limited in their ability to express embodied knowledge as an in-person tutor could. We developed WeavAR, an application for headworn augmented reality (AR) displays designed to teach basic bead weaving patterns. WeavAR combines virtual 3D hands showing weaving sequences recorded from an experienced bead weaver and a dynamic 3D bead model showing how the work progresses. Using a mixed within/between-subjects user study (n=30), we compared learning materials (AR to videos and figures) and learning material placement (in the area of work or to the side). Results show that the AR learning materials had comparable effectiveness to video and figures. Hand visualizations were found to lack crucial context, however, making them less useful than the 3D bead model. Extra measures to prevent obstruction are required when placing learning materials at the area of work.

Group sports such as cycling promote a healthy lifestyle and contribute to an active social life among individuals with a common interest. HCI research has mainly focused on supporting individual riders like commuters, young children learning to ride safely, and tourists using bikes. In this work, we present the results of a study exploring the work done by road cyclists while training together, following a Contextual Design methodology involving participants from three different cycling groups across different geographic regions. We share their strategies and challenges with group coordination and communication, and uncover a lack of support for sharing individual metrics such as effort in common tools.

In this work, we present the design and comparative evaluation of techniques for increasing awareness of out-of-view targets in virtual reality. We first compare two variants of SOUS-a technique that guides the user to out-of-view targets using circle cues in their peripheral vision-with the existing FlyingARrow technique, in which arrows fly from the user's central (foveal) vision toward the target. fSOUS, a variant with low visual salience, performed well in a simple environment but not in visually complex environments, while bSOUS, a visually salient variant, yielded faster target selection than both fSous and FlyingARrow across all environments. We then compare hybrid techniques in which aspects of SOUS relating to unobtrusiveness and visual persistence were reflected in design modifications made to FlyingARrow. Increasing persistence by adding trails to arrows improved performance but there were concerns about obtrusiveness, while other modifications yielded slower and less accurate target acquisition. Nevertheless, since fSOUS and bSOUS are exclusively for head-mounted display with wide field-of-view, FlyingARrow with trail can still be beneficial for devices with limited field-of-view.

Headworn Augmented Reality (AR) and Virtual Reality (VR) displays are an exciting new medium for locative storytelling. Authors face challenges planning and testing the placement of story elements when the story is experienced in multiple locations or the environment is large or complex. We present Story CreatAR, the first locative AR/VR authoring tool that integrates spatial analysis techniques. Story CreatAR is designed to help authors think about, experiment with, and reflect upon spatial relationships between story elements, and between their story and the environment. We motivate and validate our design through developing different locative AR/VR stories with several authors.

Very large floor displays can promote engaging public experiences, but incur perspective-related warping, making it challenging to comprehend and interact with distal objects when standing on the display. We introduce a perspective compensated view technique that maintains the relative size and shape of objects as they move away from the viewer, and explore the technique in SpaceHopper, a large-scale, floor-projected version of the game Asteroids. Players bounce on a hopper ball to control their ship in one of two control modes: bounce to repel or bounce to shoot. We evaluated SpaceHopper in a field experiment with 59 participants, finding that perspective compensated view yielded longer playing times (and higher scores) in the bounce to fire modality. Bystanders were highly engaged with players and also seemed to be unaware of the perspective warping, suggesting that visually compensating for the interactor’s perspective does not adversely impact the enjoyment of passive participants and audience members, at least under some circumstances.

Persuasive technologies (PTs) motivate behaviour change using various persuasive strategies. However, there is still a dearth of knowledge on how PTs motivate behaviour change and how to design systems to increase their persuasiveness. To provide empirical insight into the mechanism through which PTs persuade, we conducted a large-scale study with 543 participants to investigate the relation between Attention, Relevance, Confidence, and Satisfaction constructs from the ARCS model of motivation and 10 strategies that are commonly used in persuasive systems design. Our results show that the ARCS constructs collectively explain between 82% and 91% of the variance in persuasiveness across the ten strategies. Relevance, followed by Attention, has the strongest association with persuasiveness. The result generalises across gender groups. Therefore, to increase a system’s persuasiveness, designers should focus on designing to increase relevance and to capture user’s attention, while also promoting confidence and a feeling of satisfaction. We contribute to Human–Computer Interaction (HCI) and Persuasive Technology by offering design guidelines for PTs to increase their persuasiveness and hence efficacy.

Museums are constantly searching for new ways to increase engagement with their exhibits, from electronic guides to modern digital technologies such as special-purpose tablets, smartphones, and virtual and augmented reality (AR). For AR exhibits in particular, promoting shared experience and group cohesion is not straightforward. In this work, we investigate scenarios in which not everyone is using a head-worn display (HWD), either because there aren't enough available or simply because someone might feel uncomfortable using it. We propose two sharing techniques for AR experiences and evaluate them in a long term in-the-wild study: Over-the-Shoulder AR, which renders a real-time virtual representation of the augmented reality content on a large secondary display; Semantic Linking, which displays contextual information about the virtual content on the same large display. We also introduce a complementary technique: Indicator Rings, which display the locations of the HWD user's objects-of-focus. We observed that participants in the Over-the-Shoulder AR and Semantic Linking conditions stayed together and exhibited more verbal exchanges than participants in a Baseline condition, which could indicate that they were more engaged. Self-reported measures indicated an increase in pair communication and increased comprehension of the virtual content for participants without the HWD. Participants without the HWD also displayed a greater understanding of the location of virtual elements with support from the Indicator Rings, and used them as a tool to guide the HWD user through the virtual content. We discuss design implications for interactive augmented reality exhibits and possible applications outside the cultural heritage scenario.

to create an integrated space. We consider an MR configuration in which collocated collaborators work around a tabletop display, while remote collaborators wear an HMD to interact with a connected virtual environment that gives a 3D perspective, and consider the impact of varying degrees of view congruence with their collaborators. In a within-subjects study with 18 groups of 3, groups completed task scenarios involving 3D object manipulation around a physical-virtual mapped tabletop. We compare a synchronized Tabletop display baseline and two MR conditions with different levels of view congruence: Fishtank and Hover. Fishtank has a high degree of congruence as it shares a top-down perspective of the 3D objects with the tabletop collaborators. The Hover condition has less view congruence since 3D content hovers front of the remote collaborator above the table. The MR conditions yielded higher self-reported awareness and co-presence than the Tabletop condition for both collocated and remote participants. Remote collaborators significantly preferred the MR conditions for manipulating shared 3D models and communicating with their collaborators. Our findings illustrate strengths and weaknesses of both MR techniques but show that more participants preferred the less-congruent Hover condition overall. Reasons include that it facilitated interaction and viewing 3D objects.

Mobile wayfinding and guide apps have become indispensable tools for navigating unfamiliar urban spaces. Such applications address targeted, "just-in-time" queries, but are not optimally designed for multi-point expeditions that can quickly build route and survey-level familiarity with a neighbourhood. We first conducted an experimental simulation involving a homebuying scenario to assess the usefulness of a popular mobile wayfinding and search application (Google Maps) for exploring a neighbourhood. We then designed a prototype application called Block Party that addresses a number of limitations of Google Maps for this purpose, and evaluated it in a second replica study. The results suggested that application designs that facilitate switching among distinct but synchronized navigation views such as Block Party might support more efficient usage and the selection of task-appropriate views, leading to better overall spatial awareness.

In this paper, we explore how people use touchscreens to express emotional intensity, and whether these intensities can be understood by oneself at a later date or by others. In a controlled study, 26 participants were asked to express a set of emotions mapped to predefined gestures, at range of different intensities. One week later, participants were asked to identify the emotional intensity visualized in animations of the gestures made by themselves and by other participants. Our participants expressed emotional intensity using gesture length, pressure, and speed primarily; the choice of attributes was impacted by the specific emotion, and the range and rate of increase of these attributes varied by individual and by emotion. Recognition accuracy of emotional intensity was higher at extreme ends, and was higher for one's own gestures than those made by others. The attributes of size and pressure (mapped to color in the animation) were most readily interpreted, while speed was more difficult to differentiate. We discuss human gesture drawing patterns to express emotional intensities and implications for developers of annotation systems and other touchscreen interfaces that wish to capture affect.

We present a study examining the impact of physical and cognitive challenge on reported immersion for a mixed reality game called Beach Pong. Contrary to prior findings for desktop games, we find significantly higher reported immersion among players who engage physically, regardless of their actual game performance. Building a mental map of the real, virtual, and sensed world is a cognitive challenge for novices, and this appears to influence immersion: in our study, participants who actively attended to both physical and virtual game elements reported higher immersion levels than those who attended mainly or exclusively to virtual elements. Without an integrated mental map, in-game cognitive challenges were ignored or offloaded to motor response when possible in order to achieve the minimum required goals of the game. From our results we propose a model of immersion in mixed reality gaming that is useful for designers and researchers in this space.

Given the widespread adoption of mobile devices and the private personal and work information they carry, casual or deliberate shoulder surfing is an increasing concern with these devices. We iteratively designed a tablet interface that detects when people nearby are looking at the screen, providing awareness through glyph notifications, and response through visual protections, and evaluated its use in two experiments. The results indicate that mobile content privacy management systems such as ours could help alleviate the cognitive and social burden of managing mobile device privacy in dynamic settings. We identify physical privacy behaviours and preferences that can inform the design of privacy notification and management protocols on mobile devices. We argue that such systems require subtlety so as not to advertise the users' intention for privacy, flexibility in addressing dynamic privacy needs and trustworthiness to promote adoption.

We present a study exploring privacy behaviours in mixed reality collaborative environments. We consider two scenarios involving hiding and sharing blended physical-virtual documents around a tabletop, under two vertical display conditions: a solo display showing the remote collaborator as an avatar, and circumambient displays showing the rest of the connected virtual environment. We explore four types of cues (interactive, communication-based, ambient, and infrastructure) and their impact on how collaborators hide and share blended physical-virtual documents. Many participants did not obviously develop a sense that the physical and virtual surroundings were fused; as a result, certain physical privacy behaviours (e.g., orienting one's body to shield documents from the remote collaborator) were less apparent in the study. However, the circumambient displays generated curiosity about how the spaces were connected, and episodes where breaches were enacted or the spatial correlation was otherwise suggested led some participants to trust the environment less. On the table, the presence of fiducial markers and digital document "shadows" served to cue participants about the impact of hiding and sharing physical documents, however accidental breaches usually went unnoticed.

In this paper, we explore the use of multimodal interfaces (tangible and gestural) in collaborative gameplay. In our setup, gestural interaction is performed to perform tasks with reference content on a wall display, in support of a main activity involving tangible interaction on a tabletop display. We designed two games using this configuration, in order to explore how children share these multimodal tasks during collaborative gameplay. We conducted a pilot within-subjects user study, piloting the games with 35 children between the ages of 8 and 15, and considered the impact of a number of factors (group size, age range, game type and interaction style) on gameplay. We present lessons learned from prototyping and pilot testing tangible+gestural multi-display systems, and describe the resulting refinements made in our design and implementation.

We present results from a study examining how the physical layout of a project room and task affect the cognitive maps acquired of a connected virtual environment during mixed-presence collaboration. Results indicate that a combination of physical layout and task impacts cognitive maps of the virtual space. Participants did not form a strong model of how different physical work regions were situated relative to each other in the virtual world when the tasks performed in each region differed. Egocentric perspectives of multiple displays enforced by different furniture arrangements encouraged cognitive maps of the virtual world that reflected these perspectives, when the displays were used for the same task. These influences competed or coincided with document-based, audiovisual and interface cues, influencing collaboration. We consider the implications of our findings on WYSIWIS mappings between real and virtual for mixed-presence collaboration.

This workshop aims to discuss and develop ideas on how healthcare services, mobile technologies, and visual analytics techniques can be leveraged and contribute to new ways of mobile healthcare supportive system designs. Designing contemporary mobile support systems for healthcare support requires a clear understanding of information requirements, behaviors and basic needs of users. Design must take into account the challenges of human-device interactions in the healthcare environment; the extension of the care environment beyond the institutional setting and the engagement of patients, facility residents and families in an extended circle of care; and issues of formal and informal data sharing and privacy. This workshop invites researchers and designers working in relevant fields to discuss, compare, and demonstrate effective design approaches that can be adopted to improve the designs of mobile support systems for interactive visualization in healthcare.

We explore whole-body interaction with Tweetris, a game where two players competitively race to form Tetris shapes (tetrominos) with their body. We debuted Tweetris at an all-night, public art event, collecting 6000 winning body shapes made by more than 270 players. Tweetris employs a novel form of interaction cue we call a discretized silhouette: the mapping from physical continuous input is discretized to create a virtual body representation. Discretization creates an interesting set of properties: notably, players have a great deal of flexibility in how they create a given shape with their body. We classify and analyze successful player strategies as design input for whole body interaction, and present results showing how small differences in environment impacted player behaviour. We argue that our approach to eliciting and analyzing interaction in Tweetris has general utility to researchers and designers and we formalize it as the LoFi (Low-Fidelity) Elicitation Protocol.

We present findings from a qualitative study of the spatial practices of biological fieldwork. We argue that these fieldwork practices inform a vision of decentralized spatial annotation in which a variety of motivations, needs, and perspectives coexist, and may support each other synergistically. We contrast this with current and past designs of mobile spatial annotation systems in the literature. From our analysis we identify three guidelines for mobile annotation systems design in biological fieldwork that we argue also extend to other domains: allowing the management of space through user control over annotation processes, promoting structured but flexible annotation through user-defined annotation formats, and providing robust and comprehensive integration of disparate data sources to allow ad hoc, exploratory queries.

We explore how expert First Person Shooter (FPS) players coordinate actions using a shared voice channel. Our findings emphasize the importance of the temporality and spatiality of these tactical verbal communications ("call-outs"). From here, we outline potential designs to mitigate problems in the production/interpretation of call-outs to better support coordination.

We present Tweetris, a full-body interactive Tetris game with extended audience participation. Snapshots of players making correct tetrominos are tweeted, and this feed is used by a mobile and web-based Tetris game, which can be played from anywhere in real-time.

"Limber" refers to an interconnected system of wearable, sensor-enabled components and game-like desktop software, designed to reduce repetitive stress injury among knowledge workers. Regular bodily movement/exercise and maintenance of good posture is rewarded with within a game experience during the course of a workday. We present two design and evaluation iterations that illustrate the utility of the system and identify key design challenges.

In this paper, we introduce approaches to navigating and manipulating objects in a Collaborative Virtual Environment (CVE) that engage tangible objects and an interactive table interface. We also identify three design concerns that are common to physical-virtual connectivity for interaction with CVE systems. At last, we propose solutions to these issues within the context of CVEs.

We introduce TwinSpace, a flexible software infrastructure for combining interactive workspaces and collaborative virtual worlds. Its design is grounded in the need to support deep connectivity and flexible mappings between virtual and real spaces to effectively support collaboration. This is achieved through a robust connectivity layer linking heterogeneous collections of physical and virtual devices and services, and a centralized service to manage and control mappings between physical and virtual. In this paper we motivate and present the architecture of TwinSpace, discuss our experiences and lessons learned in building a generic framework for collaborative cross-reality, and illustrate the architecture using two implemented examples that highlight its flexibility and range, and its support for rapid prototyping.

We present results from a study examining the sensitivity of group navigation strategies to changes in route presentation on a shared mobile device. Two content-equivalent interfaces are compared. An interface providing textual instructions linked to regions on a route map yields reliance on text primarily, encouraging route planning and a divide-and-conquer strategy we term 'navigator and scout'. An interface combining text instructions with map segments on individual pages yields less planning, still permits nav/scout, and sees an increase in an ad-hoc 'sync and go' strategy involving more gathering around the device. Finally, when the route map is used without text, the frequency of the nav/scout strategy drops markedly as sync and go increases.

The results presented in this paper illustrate how a specific map visualization technique is sensitive to setting: a comparative evaluation of the technique gives conflicting results depending on where it takes place. While prior research has explored the impact of factors other than basic visual perception on visualization techniques, relatively little attention has been directed toward the physical setting in which the technique is used. We present results from a study involving 120 participants, comparing the effectiveness of two different geovisualization techniques in promoting recall of map layout. Recall was shown to be sensitive to setting, such that one technique in particular was more effective in a noisy public space than in a controlled, 'white-room' environment. The results have implications for the validation and measurement of information visualization techniques as a whole, and in particular for those employing motion as a communicative attribute.

Groups collaborating around a large wall display can do so in a variety of arrangements, positioning themselves at different distances from the display and from each other. We examined the impact of proximity on the effectiveness and enjoyment of co-located collaboration. Our results revealed collaborative benefits when participants were positioned close together, and interaction with the display was felt to be more effective when participants were close to the display. However, clear tradeoffs were evident for these configurations. When at a distance to the display, the choice of direct versus indirect interaction revealed that interactions were easier when using direct input but the effectiveness of the collaboration was compromised.

We present preliminary results from two studies examining the selection techniques suitable for paper maps using handheld computers or cellphones as interaction devices. An informal mockup exploration indicated a strong tendency toward point-and-click style interaction when participants were asked to envision how a range of queries might be expressed. A subsequent study involving a functional prototype and a short training session showed that participants were receptive to other interaction styles, including tracing paths, circling regions, constraining queries with paper menus, and selecting multiple non-adjacent map icons. The contrasting results underline the importance of using a range of design evaluation techniques when developing applications involving handheld devices as interactors.