Miniature representations, like CAD and blueprints, are useful for designing a larger physical space. While experts are trained to use these methods, non-experts often lack this training. Nonetheless, non-experts can benefit from designing with miniature representations, yet their interactions with these tools are not well understood. In our work, we observed participants designing two rooms using three tools: an online planner, pen and paper, and Lego. We collected and analyzed data from the Desirability Toolkit, a semi-structured interview, and observations of their design sessions. Our findings suggest that participants found each tool engaging and satisfying for different reasons, but paper more empowering and Lego more familiar, efficient, and unconventional. Participants also suggested that these tools had value at different design stages. We also identified that participants often had difficulty scaling objects to match realistic expectations in the paper and Lego miniature representations.

Task-unrelated thoughts (TUTs), colloquially referred to as mind-wandering or daydreaming, are phenomena that can interfere with attention and focus, but are also associated with mental health, creativity, and learning. In digital games, it is unclear how players experience game-unrelated thoughts (GUTs), whether GUTs should be encouraged by game designers, or how it may impact player experience. We ran an initial study to confirm whether GUTs are common (50 of 100 participants reported experiencing them). We then collected 840 minutes of gameplay data from 12 participants playing games they: (1) found relaxing, (2) lost track of time in, and (3) spent most hours playing. Eye-tracking data and experience sampling were used to contextualize a phenomenological analysis of gameplay data. We identified four themes encompassing gameplay, GUTs, and gaze behaviour: these provide a foundation for future research and game design incorporating GUTs.

The field of eXplainable artificial intelligence (XAI) has produced a plethora of methods (e.g., saliency-maps) to gain insight into artificial intelligence (AI) models, and has exploded with the rise of deep learning (DL). However, human-participant studies question the efficacy of these methods, particularly when the AI output is wrong. In this study, we collected and analyzed 156 human-generated text and saliency-based explanations collected in a question-answering task (N = 40) and compared them empirically to state-of-the-art XAI explanations (integrated gradients, conservative LRP, and ChatGPT) in a human-participant study (N = 136). Our findings show that participants found human saliency maps to be more helpful in explaining AI answers than machine saliency maps, but performance negatively correlated with trust in the AI model and explanations. This finding hints at the dilemma of AI errors in explanation, where helpful explanations can lead to lower task performance when they support wrong AI predictions.

This paper explores the concept of placemaking within small, constrained spaces, challenging implicit assumptions in past human-computer interaction work that creating meaningful “places” requires large spaces. Our study focuses on individuals living in settings like van homes, co-living spaces, and other environments characterized by their limited physical dimensions and the creative use of technology within these confines.

Many employees prefer to work from home, yet struggle to squeeze their office into an already fully-utilized space. Virtual Reality (VR) seemingly offered a solution with its ability to transform even modest physical spaces into spacious, productive virtual offices, but hardware challenges—such as low resolution—have prevented this from becoming a reality. Now that hardware issues are being overcome, we are able to investigate the suitability of VR for daily work. To do so, we (1) studied the physical space that users typically dedicate to home offices and (2) conducted an exploratory study of users working in VR for one week. For (1) we used digital ethnography to study 430 self-published images of software developer workstations in the home, confirming that developers faced myriad space challenges. We used speculative design to re-envision these as VR workstations, eliminating many challenges. For (2) we asked 10 developers to work in their own home using VR for about two hours each day for four workdays, and then interviewed them. We found that working in VR improved focus and made mundane tasks more enjoyable. While some subjects reported issues—annoyances with the fit, weight, and umbilical cord of the headset—the vast majority of these issues seem to be addressable. Together, these studies show VR technology has the potential to address many key problems with home workstations, and, with continued improvements, may become an integral part of creating an effective workstation in the home.

While multiple studies suggest that female-identified participants are more likely to experience cybersickness in virtual reality (VR), our systematic review of 71 eligible VR publications (59 studies and 12 surveys) pertaining to gender and cybersickness reveals a number of confounding factors in study design (e.g., a variety of technical specifications, tasks, content), a lack of demographic data, and a bias in participant recruitment. Our review shows an ongoing need within VR research to more consistently include and report on women’s experiences in VR to better understand the gendered possibility of cybersickness. Based on the gaps identified in our systematic review, we contribute study design recommendations for future work, arguing that gender considerations are necessary at every stage of VR study design, even when the study is not ‘about’ gender.

Research on how lived experiences with technology intersect with home and work are core themes within HCI. Prior work has primarily focused on conventional life and work in Western countries. However, the unconventional is becoming conventional—several rising subcultures are coming into prominence due to socio-economic pressures, aided by social media. One example—#vanlife—is now practised by an estimated three million people in North America. #vanlife combines travel, home, and work by their occupants (vanlifers (vanlifers)) living full-time in cargo vans that they usually convert themselves into living spaces. We present a portrait of vanlifers’ current technology practices gleaned through ~200 hours of fieldwork and interviews. Following a thematic analysis of our data, we identified unique opportunities for integrating technology across culture, design, homesteading, offline organization, and gaming. We have distilled these opportunities into eleven provocations to inspire critical design and informed inquiry for technological interventions for #vanlife.

It is our great pleasure to welcome you to this issue of the Proceedings of the ACM on Human-Computer Interaction, the second to focus on the contributions from the research community Interactive Surfaces and Spaces (ISS). Interactive Surfaces and Spaces increasingly pervade our everyday life, appearing in various sizes, shapes, and application contexts, offering a rich variety of ways to interact. This diverse research community explores the design, development, and use of new and emerging interactive surface technologies and interactive spaces. The call for articles for this issue on ISS attracted 77 submissions, from all over the world. This issue has 23 papers, 4 submitted in February 2021 and 19 submitted in July 2021. After the winter round, 4 (total of 19 articles, 21.1%) articles were accepted and 5 (26.3%) articles required major revisions. After the summer round, 19 (total of 58 articles, 32.8%) articles were accepted, and 18 (31,0%) articles required major revisions. The editorial committee worked hard over the two iterations of the review process, winter and summer rounds, to arrive at final decisions. In total, counting both the winter and the summer rounds, 23 articles (total of 77 articles, 29.9%) were accepted. All authors of the accepted articles are invited to present at the ISS conference from November 14--17, 2021. This issue exists because of the dedicated volunteer effort of 31 senior editors who served as Associate Chairs (ACs), 105 expert reviewers in the winter round, and 206 expert reviewers in the summer round to ensure high quality and insightful reviews for all articles. Reviewers and committee members were kept constant for papers that submitted to both rounds. The Editorial Board is presented here: https://iss.acm.org/2021/organization/editorial_board

When interacting with virtual reality (VR) applications like CAD and open-world games, people may want to use gestures as a means of leveraging their knowledge from the physical world. However, people may prefer physical props over handheld controllers to input gestures in VR. We present an elicitation study where 21 participants chose from 95 props to perform manipulative gestures for 20 CAD-like and open-world game-like referents. When analyzing this data, we found existing methods for elicitation studies were insufficient to describe gestures with props, or to measure agreement with prop selection (i.e., agreement between sets of items). We proceeded by describing gestures as context-free grammars, capturing how different props were used in similar roles in a given gesture. We present gesture and prop agreement scores using a generalized agreement score that we developed to compare multiple selections rather than a single selection. We found that props were selected based on their resemblance to virtual objects and the actions they afforded; that gesture and prop agreement depended on the referent, with some referents leading to similar gesture choices, while others led to similar prop choices; and that a small set of carefully chosen props can support multiple gestures.

Games hold the potential to help many address health-related issues such as chronic stress. We investigated the use of biophilia, an affective response to nature grounded in the psychology literature, as indirect physiological input for biofeedback games. We designed and developed a non-violent exploration game, and conducted an empirical study that examined affective and physiological responses to gameplay in virtual nature and urban settings. Our results did not identify a difference in stress levels experienced by players between these two settings, but point to improved attention when playing in nature settings. We discuss implications of these findings, and discuss both difficulties in and potential future strategies for applying biophilia to the design of biofeedback games.

Open offices are cost-effective and continue to be popular. However, research shows that these environments, brimming with distractions and sensory overload, frequently hamper productivity. Our research investigates the use of virtual reality (VR) to mitigate distractions in an open office setting and improve one's ability to be in flow. In a lab study, 35 participants performed visual programming tasks in four combinations of physical (open or closed office) and virtual environments (beach or virtual office). While participants both preferred and were in flow more in a closed office without VR, in an open office, the VR environments outperformed the no VR condition in all measures of flow, performance, and preference. Especially considering the recent rapid advancements in VR, our findings illustrate the potential VR has to improve flow and satisfaction in open offices.

Virtual reality (VR) strives to replicate the sensation of the physical environment by mimicking people's perceptions and experience of being elsewhere. These experiences are of-ten mediated by the objects and tools we interact with in the virtual world (e.g., a controller). Evidence from psychology posits that when using the tool proficiently, it becomes em-bodied (i.e., an extension of one's body). There is little work,however, on how to measure this phenomenon in VR, andon how different types of tools and controllers can affect the experience of interaction. In this work, we leverage cognitive psychology and philosophy literature to construct the Locus-of-Attention Index (LAI), a measure of tool embodiment. We designed and conducted a study that measures readiness-to-hand and unreadiness-to-hand for three VR interaction techniques: hands, a physical tool, and a VR controller. The study shows that LAI can measure differences in embodiment with working and broken tools and that using the hand directly results in more embodiment than using controllers.

Social play can have numerous health benefits but research has shown that not all multiplayer games are effective at promoting social engagement. Asymmetric cooperative games have shown promise in this regard but the design and dynamics of this unique style of play is not yet well understood. To address this, we present the results of two player experience studies using our custom prototype game Beam Me 'Round, Scotty! 2: the first comparing symmetric cooperative play (e.g., where players have the same interface, goals, mechanics, etc.) to asymmetric cooperative play (e.g., where players have differing roles, abilities, interfaces, etc.) and the second comparing the effect of increasing degrees of interdependence between play partners. Our results not only indicate that asymmetric cooperative games may enhance players' perceptions of connectedness, social engagement, immersion, and comfort with a game's controls, but also demonstrate how to further improve these outcomes via deliberate mechanical design changes, such as changes in cooperative action timing and direction of dependence.

Complex virtual reality (VR) tasks, like 3D solid modelling, are challenging with standard input controllers. We propose exploiting the affordances and input capabilities when using a 3D-tracked multi-touch tablet in an immersive VR environment. Observations gained during semi-structured interviews with general users, and those experienced with 3D software, are used to define a set of design dimensions and guidelines. These are used to develop a vocabulary of interaction techniques to demonstrate how a tablet's precise touch input capability, physical shape, metaphorical associations, and natural compatibility with barehand mid-air input can be used in VR. For example, transforming objects with touch input, "cutting" objects by using the tablet as a physical "knife", navigating in 3D by using the tablet as a viewport, and triggering commands by interleaving bare-hand input around the tablet. Key aspects of the vocabulary are evaluated with users, with results validating the approach.

Recent efforts to diversify participation in STEM (Science, Technology, Engineering & Math) activities through informal learning environments, such as hackathons and makerspaces, confirm a real desire for inclusion among potential female participants. However, understanding factors that may contribute to longer-term, sustainable diversification of such groups remains a challenge. In this paper, we present the results of a mixed-methods study of two microcosms of making: game development, and quilting. Our findings reveal parallel structures within these groups despite being highly skewed towards male or female participation, respectively. Our results shed light on attitudes, behaviours, and experiences indicating that similar desires for wider community support among other factors exist in both groups, but these needs are not satisfied in the STEM context. We conclude by discussing the implications of our findings as opportunities for rethinking how we design the environments that are meant to support design itself, considering the role of technology in these spaces, and prioritizing nurturing the development of the maker community beyond the maker space.

New interaction techniques, like multi-touch, tangible interaction, and mid-air gestures often promise to be more intuitive and natural; however, there is little work on how to measure these constructs. One way is to leverage the phenomenon of tool embodiment---when a tool becomes an extension of one's body, attention shifts to the task at hand, rather than the tool itself. In this work, we constructed a framework to measure tool embodiment by incorporating philosophical and psychological concepts. We applied this framework to design and conduct a study that uses attention to measure readiness-to-hand with both a physical tool and a virtual tool. We introduce a novel task where participants use a tool to rotate an object, while simultaneously responding to visual stimuli both near their hand and near the task. Our results showed that participants paid more attention to the task than to both kinds of tool. We also discuss how this evaluation framework can be used to investigate whether novel interaction techniques allow for this kind of tool embodiment.

In the domain of literary criticism, many critics practice close reading , annotating by hand while performing a detailed analysis of a single text. Often this process employs the use of external resources to aid analysis. In this article, we present a study and subsequent tool design focused on leveraging a critic’s annotations as implicit interactions for initiating context-specific computational support that automatically searches external resources. We observed 14 poetry critics performing a close reading, revealing a set of cognitive practices supported through free-form annotation that have not previously been discussed in this context. We used guidelines derived from our study to design a tool, Metatation, which uses a pen-and-paper system with a peripheral display to utilize reader annotations as underspecified interactions to augment close reading. By turning paper-based annotations into implicit queries, Metatation provides relevant supplemental information in a just-in-time manner and acts as a bridge between close and distant reading.

We introduce Flexible Trees, a sketch-based layout adjustment technique. Although numerous tree layout algorithms exist, these algorithms are usually bound to fit within standard shapes such as rectangles, circles and triangles. In order to provide the possibility of interactively customizing a tree layout, we offer a free-form sketch-based interaction through which one can re-define the boundary constraints for the tree layouts by combining ray-line intersection and line segment intersection. Flexible Trees offer topology preserving adjustments; can be used with a variety of tree layouts; and offer a simple way of authoring tree layouts for infographic purposes.

Manipulating a virtual object with appropriate passive haptic cues provides a satisfying sense of presence in virtual reality. However, scaling such experiences to support multiple virtual objects is a challenge as each one needs to be accompanied with a precisely-located haptic proxy object. We propose a solution that overcomes this limitation by hacking human perception. We have created a framework for repurposing passive haptics, called haptic retargeting, that leverages the dominance of vision when our senses conflict. With haptic retargeting, a single physical prop can provide passive haptics for multiple virtual objects. We introduce three approaches for dynamically aligning physical and virtual objects: world manipulation, body manipulation and a hybrid technique which combines both world and body manipulation. Our study results indicate that all our haptic retargeting techniques improve the sense of presence when compared to typical wand-based 3D control of virtual objects. Furthermore, our hybrid haptic retargeting achieved the highest satisfaction and presence scores while limiting the visible side-effects during interaction.

Manipulating a virtual object with appropriate passive haptic cues provides a satisfying sense of presence in virtual reality. However, scaling such experiences to support multiple virtual objects is a challenge as each one needs to be accompanied with a precisely-located haptic proxy object. We showcase a solution that overcomes this limitation by hacking human perception. Our framework for repurposing passive haptics, called haptic retargeting, leverages the dominance of vision when our senses conflict. With haptic retargeting, a single physical prop can provide passive haptics for multiple virtual objects. We introduce three approaches for dynamically aligning physical and virtual objects: body manipulation, world manipulation and a hybrid technique which combines both world and body manipulation. This demo has been presented previously at CHI 2016.

We present Tunnel Divisions, an interactive musical installation designed for ephemeral interaction in public spaces. Informed by concepts from cultural and media studies along with existing literature on interactive public displays, we designed the installation as an intervention meant for the monotonous parts of everyday life. Our demonstration uses low-cost sensors and musical theory to create a generative sound composition unique to the particular space and the people moving through it. By experiencing the installation, we hope to promote critical thinking about the nature and form of interaction with public spaces.

Displaying 3D objects on horizontal displays can cause problems in the way that the virtual scene is presented on the 2D surface; inappropriate choices in how 3D is represented can lead to distorted images and incorrect object interpretations. We present four experiments that test 3D perception. We varied projection geometry in three ways: type of projection (perspective/parallel), separation between the observer's point of view and the projection's center (discrepancy), and the presence of motion parallax (with/without parallax). Projection geometry had strong effects different for each task. Reducing discrepancy is desirable for orientation judgments, but not for object recognition or internal angle judgments. Using a fixed center of projection above the table reduces error and improves accuracy in most tasks. The results have far-reaching implications for the design of 3D views on tables, in particular, for multi-user applications where projections that appear correct for one person will not be perceived correctly by another.

Interacting with nature is beneficial to a person's mental-state, but it can sometimes be difficult to find environments that will induce positive affect (e.g., when planning a run). In this paper, we describe EnviroPulse-a system for auto-matically determining and communicating the expected affective valence (EAV) of environments to individuals. We describe a prototype that allows this to be used in real-time on a smartphone, but EnviroPulse could easily be incorporated into GPS systems, mapping services, or image-based systems. Our work differs from existing work in af-fective computing in that, rather than detecting a user's affect directly, we automatically determine the EAV of the environment through visual analysis. We present results that suggest our system can determine the EAV of envi-ronments. We also introduce real-time affective visual feedback of the calculated EAV of the images, and present results from an informal study suggesting that real-time visual feedback can be used for induction of affect.

Over the past decade, multi-touch surfaces have become commonplace, with many researchers and practitioners describing the benefits of their natural, physical-like interactions. We present a pair of studies that empirically investigates the psychophysical effects of direct interaction with both physical and virtual artefacts. We use the phenomenon of Kinesthetic Figural After Effects-a change in understanding of the physical size of an object after a period of exposure to an object of different size. Our studies show that, while this effect is robustly reproducible when using physical artefacts, this same effect does not manifest when manipulating virtual artefacts on a direct, multi-touch tabletop display. We contribute quantitative evidence suggesting a psychophysical difference in our response to physical vs. virtual objects, and discuss future research directions to explore measurable phenomena to evaluate the presence of physical-like changes from virtual on-screen objects.

Overview+Detail interfaces can be used to examine the details of complex data while retaining the data's overall context. Dynamic data introduce challenges for these interfaces, however, as moving objects may exit the detail view, as well as a person's field of view if they are working at a large interactive surface. To address this "off-view" problem, we propose a new information visualization technique, called Canyon. This technique attaches a small view of an off-view object, including some surrounding context, to the external boundary of the detail view. The area between the detail view and the region containing the off-view object is virtually "folded" to conserve space. A comparison study was conducted contrasting the benefits and limitations of Canyon to an established technique, called Wedge. Canyon was more accurate across a number of tasks, especially more complex tasks, and was comparably efficient.

We investigate behaviours on, and around, large vertical displays during concurrent usage. Using an observational field study, we identify fundamental patterns of how people use existing public displays: their orientation, positioning, group identification, and behaviour within and between social groups just-before, during, and just-after usage. These results are then used to motivate a controlled experiment where two individuals or two pairs of individuals complete tasks concurrently on a simulated large vertical display. Results from our controlled study demonstrates that vertical surface territories are similar to those found in horizontal tabletops in function, but their definitions and social conventions are different. In addition, the nature of use-while-standing systems results in more complex and dynamic physical territories around the display. We show that the anthropological notion of personal space must be slightly refined for application to vertical displays.

We present the iterative design of a virtual sandtray application for a tabletop display. The purpose of our prototype is to support sandtray therapy, a form of art therapy typically used for younger clients. A significant aspect of this therapy is the insight gained by the therapist as they observe the client interact with the figurines they use to create a scene in the sandtray. In this manner, the therapist can gain increased understanding of the client's psyche. We worked with three sandtray therapists throughout the evolution of our prototype. We describe the details of the three phases of this design process: initial face-to-face meetings, iterative design and development via distance collaboration, and a final face-to-face feedback session. This process revealed that our prototype was sufficient for therapists to gain insight about a person's psyche through their interactions with the virtual sandtray.

On traditional tables, people frequently use the third dimension to pile, sort and store objects. However, while effective and informative for organization, this use of the third dimension does not usually extend far above the table. To enrich interaction with digital tables, we present the concept of shallow-depth 3D -- 3D interaction with limited depth. Within this shallow-depth 3D environment several common interaction methods need to be reconsidered. Starting from any of one, two and three touch points, we present interaction techniques that provide control of all types of 3D rotation coupled with translation (6DOF) on a direct-touch tabletop display. The different techniques exemplify a wide range of interaction possibilities: from the one-touch technique, which is designed to be simple and natural, but inherits a degree of imprecision from its simplicity; through to three-touch interaction, which allows precise bimanual simultaneous control of multiple degrees of freedom, but at the cost of simplicity. To understand how these techniques support interaction in shallow-depth 3D, we present a user study that examines the efficiency of, and preferences for, the techniques developed. Results show that users are fastest and most accurate when using the three-touch technique and that their preferences were also strongly in favour of the expressive power available from three-touch.

The rapid development of large interactive wall displays has been accompanied by research on methods that allow people to interact with the display at a distance. The basic method for target acquisition is by ray casting a cursor from one's pointing finger or hand position; the problem is that selection is slow and error-prone with small targets. A better method is the bubble cursor that resizes the cursor's activation area to effectively enlarge the target size. The catch is that this technique's effectiveness depends on the proximity of surrounding targets: while beneficial in sparse spaces, it is less so when targets are densely packed together. Our method is the speech-filtered bubble ray that uses speech to transform a dense target space into a sparse one. Our strategy builds on what people already do: people pointing to distant objects in a physical workspace typically disambiguate their choice through speech. For example, a person could point to a stack of books and say "the green one". Gesture indicates the approximate location for the search, and speech 'filters' unrelated books from the search. Our technique works the same way; a person specifies a property of the desired object, and only the location of objects matching that property trigger the bubble size. In a controlled evaluation, people were faster and preferred using the speech-filtered bubble ray over the standard bubble ray and ray casting approach.

Teams that are geographically distributed often share information both in real-time and asynchronously. When such sharing is through groupware, change conflicts can arise when people pursue parallel and competing actions on the same information. This leads to problems in how the systems and its users maintain a consistent view of shared information across distance and time. We explore change awareness of conflicts in a three-dimensional distributed shared space. Our user study compares the use of visual feedback to an optimistic concurrency control strategy for both synchronous and asynchronous distributed groupware. Our feedback provides a means for synchronous users to recognize and resolve real-time changes, and for asynchronous users to view and resolve changes when switching from an offline to online mode of work. Results of our study suggest that the visual feedback serves as a useful feedthrough mechanism in the synchronous case, but that asynchronous users may be overwhelmed by the quantity of changes if they come online after many changes have been made.