Autistic livestreamers use platforms like Twitch for social connection, self-expression, and community, but these spaces also impose ongoing social and emotional demands. Prior work has documented these experiences, but less is known about what autistic creators themselves envision for the tools and platforms they use. We address this gap through a Research through Design (RtD) co-design study with three autistic Twitch streamers, using speculative artefacts as discussion prompts to explore how participants reasoned about potential livestreaming technologies. Across three co-design activities, we identify three overarching tensions shaping autistic streaming practice: Expression versus Misinterpretation and Harm; Public Participation versus Control and Moderation; and Sustainable Practice versus Finite Energy. Our co-designers surfaced constraints, trade-offs, boundaries, and hesitations toward automation, and concerns about managing viewer expectations. This work contributes an account of co-design as a reflective practice with autistic livestreamers, showing how speculative artefacts can surface values, limits, and design reasoning around livestreaming technology.

Conversational agents that participate in or mediate group interaction introduce challenges that extend beyond supporting individual users, raising new questions about how agents participate in and influence groups. To characterise this emerging design space, we present a systematic review of 53 peer-reviewed studies on group conversational agents (GCAs). We analyse how GCAs intervene in group-level processes, including participation regulation, conflict mediation, task alignment, and execution support. Using concepts from group research as an analytic lens, we organise prior GCA work around recurring group interactional challenges (orientation, conflict, alignment, and execution), and examine the roles agents are designed to play in addressing these challenges. We find that GCAs are predominantly designed as short-term, role-bounded interventions targeting isolated challenges in bounded interactional contexts. We further identify recurring structural tensions in GCA design, including tradeoffs between visibility and discretion, proactivity and group autonomy, and agent authority and group ownership. Together, these findings clarify how current GCAs are positioned within group interaction, surface the implicit assumptions embedded in their designs, and outline open questions for future research on conversational agents as group-level interventions.

Intergenerational conversations often break down when differences in tone, language, or expectations lead participants to feel dismissed or misunderstood. In this work, we explore how people envision AI-driven chatbot interventions for addressing communication problems in text-based intergenerational family chat. We conducted a scenario-based design interview with 10 pairs of family members from different generations, in which participants designed chatbot interventions that varied in intervention target and timing. Our findings show that participants expect chatbots to perform multiple themes of intervention, including mediating understanding, providing emotional support, offering evaluative commentary, and guiding interaction through behavioral suggestions. These expectations varied systematically across intervention contexts, giving rise to distinct chatbot roles such as neutral mediators, message coaches, repair facilitators, and emotion regulators. Across these roles, participants positioned chatbots as moral advisors that evaluate communicative appropriateness and exercise varying degrees of moral authority. Rather than prescribing specific system behaviors, this work offers a conceptual and exploratory account of AI-mediated intervention in intergenerational communication, and articulates key design tensions that arise when chatbots are imagined as socially and morally involved actors in intimate family interactions.

A common collocated group setting in mixed-reality (MR) collaboration is a person wearing a MR headset (HMD user) and presenting MR contents to audiences who are not provided with such specialized devices (Non-HMD users). In this setting, while Non-HMD users can view the MR environment shown on a large physical display, it still remains challenging for the HMD user to interpret their pointing gesture when they spatially refer to objects in the MR environment. To address this, we designed and evaluated two pointing techniques—SCREEN and SCREEN+SPACE—that support Non-HMD users in referring to MR content. Screen pointing allows users to refer to MR objects by pointing at their representation on the display, while Screen+Space pointing additionally enables direct spatial references toward the HMD user’s environment. In a study involving 36 participants (18 pairs) performing an expert-novice collaboration task, both techniques improved communication fluency and reduced cognitive workload. Notably, Screen+Space pointing was preferred over Screen pointing, where Non-HMD users often pointed relative to the HMD user’s physical space rather than the screen. This suggests that people naturally treat MR content as embedded in shared space, and that space pointing enables more natural and effective communication in asymmetric MR collaborations.

We characterize 16 challenges faced by those investigating and developing remote and synchronous collaborative experiences around visualization. Our work reflects the perspectives and prior research efforts of an international group of 29 experts from across human-computer interaction and visualization sub-communities. The challenges are anchored around five collaborative activities that exhibit a centrality of visualization and multimodal communication. These activities include exploratory data analysis, creative ideation, visualization-rich presentations, joint decision making grounded in data, and real-time data monitoring. The challenges also reflect the changing dynamics of these activities in the face of recent advances in extended reality (XR) and artificial intelligence (AI). As an organizing scheme for future research at the intersection of visualization and computer-supported cooperative work, we align the challenges with a sequence of four sets of research and development activities: technological choices, social factors, AI assistance, and evaluation.

Conversational agents are increasingly used in education for learning support. An application is “learning by explaining”, where learners explain their understanding to an agent. However, existing research focuses on single roles, leaving it unclear how different pedagogical roles influence learners’ interaction patterns, learning outcomes and experiences. We conducted a between-subjects study (N=96) comparing agents with three pedagogical roles (Tutee, Peer, Challenger) and a control condition while learning an economics concept. We found that different pedagogical roles shaped learning dynamics, including interaction patterns and experiences. Specifically, the Tutee agent elicited the most cognitive investment but led to high pressure. The Peer agent fostered high absorption and interest through collaborative dialogue. The Challenger agent promoted cognitive and metacognitive acts, enhancing critical thinking with moderate pressure. The findings highlight how agent roles shape different learning dynamics, guiding the design of educational agents tailored to specific pedagogical goals and learning phases.

Nudging participants with text-based reflective nudges enhances deliberation quality on online deliberation platforms. The effectiveness of multimodal reflective nudges, however, remains largely unexplored. Given the multi-sensory nature of human perception, incorporating diverse modalities into self-reflection mechanisms has the potential to better support various reflective styles. This paper explores how presenting reflective nudges of different types (direct: persona and indirect: storytelling) in different modalities (text, image, video and audio) affects deliberation quality. We conducted two user studies with 20 and 200 participants respectively. The first study identifies the preferred modality for each type of reflective nudges, revealing that text is most preferred for persona and video is most preferred for storytelling. The second study assesses the impact of these modalities on deliberation quality. Our findings reveal distinct effects associated with each modality, providing valuable insights for developing more inclusive and effective online deliberation platforms.

Foundation models are rapidly improving the capability of robots in performing everyday tasks autonomously such as meal preparation, yet robots will still need to be instructed by humans due to model performance, the difficulty of capturing user preferences, and the need for user agency. Robots can be instructed using various methods-natural language conveys immediate instructions but can be abstract or ambiguous, whereas end-user programming supports longer-horizon tasks but interfaces face difficulties in capturing user intent. In this work, we propose using direct manipulation of images as an alternative paradigm to instruct robots, and introduce a specific instantiation called ImageInThat which allows users to perform direct manipulation on images in a timeline-style interface to generate robot instructions. Through a user study, we demonstrate the efficacy of ImageInThat to instruct robots in kitchen manipulation tasks, comparing it to a text-based natural language instruction method. The results show that participants were faster with ImageInThat and preferred to use it over the text-based method. Supplementary material including code can be found at: https://image-in-that.github.io/.

Mixed reality (MR) presentations often involve a presenter wearing a head-mounted display (HMD) and an audience watching via a large display, making it difficult for audiences to perceive spatial relationships between the presenter and virtual objects. We report two experiments testing three design variations: (1) scene camera placement (audience-aligned vs. opposite), (2) overlaying the presenter’s first-person view, and (3) highlighting objects in the presenter’s view. Results show that audience-aligned cameras and object highlighting improve spatial understanding, while combining third- and first-person views can further aid perception. We derive design guidelines for configuring MR presentations to better support audience comprehension.

Student feedback is critical for improving teaching, yet instructors often avoid reading evaluations due to emotional burden and information overload. We present a systematic exploration of how language models can distill and transform student evaluations into adaptive, actionable insights. Through a systematic design space exploration combining 4 feedback strategies (removing harmful content, paraphrasing criticism, sandwiching negatives, adding constructive suggestions) with 4 presentation formats (themes, cards, letters, chatbots), we created six AI-augmented prototypes of teaching evaluations. Interviews with 16 post-secondary instructors revealed that effective use of AI in feedback processing should: (1) support action formation through focused views and divergent thinking, (2) reduce emotional costs while enabling celebration and sharing, (3) facilitate longitudinal engagement and re-contextualization across terms, and (4) maintain transparency and preserve access to original context to build trust. Our work provides design guidelines for AI-augmented feedback systems and demonstrates how language models can adaptively process and present information based on feedback receivers' specific needs and contexts.

Generative AI tools can provide people with the ability to create virtual environments and scenes with natural language prompts. Yet, how people will formulate such prompts is unclear—particularly when they inhabit the environment that they are designing. For instance, it is likely that a person might say, “Put a chair here,” while pointing at a location. If such linguistic and embodied features are common to people’s prompts, we need to tune models to accommodate them. In this work, we present a Wizard of Oz elicitation study with 22 participants, where we studied people’s implicit expectations when verbally prompting such programming agents to create interactive VR scenes. Our findings show when people prompted the agent, they had several implicit expectations of these agents: (1) they should have an embodied knowledge of the environment; (2) they should understand embodied prompts by users; (3) they should recall previous states of the scene and the conversation, and that (4) they should have a commonsense understanding of objects in the scene. Further, we found that participants prompted differently when they were prompting in situ (i.e. within the VR environment) versus ex situ (i.e. viewing the VR environment from the outside). To explore how these lessons could be applied, we designed and built Ostaad, a conversational programming agent that allows non-programmers to design interactive VR experiences that they inhabit. Based on these explorations, we outline new opportunities and challenges for conversational programming agents that create VR environments.

To enhance interactions in VR, many techniques introduce offsets between the virtual and real-world position of users’ hands. Nevertheless, such hand redirection (HR) techniques are only effective as long as they go unnoticed by users—not disrupting the VR experience. While several studies consider how much unnoticeable redirection can be applied, these focus on mid-air floating hands that are disconnected from users’ bodies. Increasingly, VR avatars are embodied as being directly connected with the user’s body, which provide more visual cue anchoring, and may therefore reduce the unnoticeable redirection threshold. In this work, we studied more complete avatars and their effect on the sense of embodiment and the detectability of HR. We found that higher avatar completeness increases embodiment, and we provide evidence for the absence of practically relevant effects on the detectability of HR.

This is an accepted manuscript of a conference paper published by ACM.

In this work, we compare the designs of two fashion-tech garments that communicate the anxiety felt when walking alone. While the two garments share a common vision, they are designed to be worn in two radically different settings and to communicate to different audiences: one directly communicates an empathetic experience to its wearer; the other a model wears at a runway show and must share its story to a general audience. We used Research Through Design (RtD) methods to design both fashion-tech garments. Then, we recorded and analyzed the design process for both garments via an annotated portfolio to compare how the audience and setting influenced the design approach and the final wearable. In this paper, we describe both fashion-tech designs and a comparison of their respective annotated portfolios. Our analysis highlights how wearable technology must respond to context to continue communicating its story to its intended audience.

Many illusion and interaction techniques in Virtual Reality (VR) rely on Hand Redirection (HR), which has proved to be effective as long as the introduced offsets between the position of the real and virtual hand do not noticeably disturb the user experience. Yet calibrating HR offsets is a tedious and time-consuming process involving psychophysical experimentation, and the resulting thresholds are known to be affected by many variables—limiting HR’s practical utility. As a result, there is a clear need for alternative methods that allow tailoring HR to the perceptual boundaries of individual users. We conducted an experiment with 18 participants combining movement, eye gaze and EEG data to detect HR offsets Below, At, and Above individuals’ detection thresholds. Our results suggest that we can distinguish HR At and Above from no HR. Our exploration provides a promising new direction with potentially strong implications for the broad field of VR illusions.

We present perspectives from 10 autistic Twitch streamers regarding their experiences as livestreamers and how autism uniquely colors their experiences. Livestreaming offers a social online experience distinct from in-person, face-to-face communication, where autistic people tend to encounter challenges. Our reflexive thematic analysis of interviews with 10 participants showcases autistic livestreamers’ perspectives in their own words. Our findings center on the importance of having streamers establishing connections with other, sharing autistic identities, controlling a space for social interaction, personal growth, and accessibility challenges. In our discussion, we highlight the crucial value of having a medium for autistic representation, as well as design opportunities for streaming platforms to onboard autistic livestreamers and to facilitate livestreamers communication with their audience.

Live and pre-recorded video tutorials are an effective means for teaching physical skills such as cooking or prototyping electronics. A dedicated cameraperson following an instructor’s activities can improve production quality. However, instructors who do not have access to a cameraperson’s help often have to work within the constraints of static cameras. We present Stargazer, a novel approach for assisting with tutorial content creation with a camera robot that autonomously tracks regions of interest based on instructor actions to capture dynamic shots. Instructors can adjust the camera behaviors of Stargazer with subtle cues, including gestures and speech, allowing them to fluidly integrate camera control commands into instructional activities. Our user study with six instructors, each teaching a distinct skill, showed that participants could create dynamic tutorial videos with a diverse range of subjects, camera framing, and camera angle combinations using Stargazer.

Fashion is driven by a narrative, i.e. a story or idea that the designer wants to convey to the audience. Fashion-tech now adds another dimension to this narrative through dynamically changing aspects of the garments. Many factors of presentation in a runway show affect how fashion-tech garments communicate a story to the audience. In this pictorial, we review a set of twenty-eight storytelling fashion-tech garments. We identify, catalogue, and categorize the factors designers used to convey stories to the audience from the runway. The design space consists of three levels: (1) the artifact-level, (2) the viewer-level, and (3) the context-level. The design space addresses how designers show their story through fashion-tech garments and how audience members see and know those messages. Our work contributes a list of considerations that fashion-tech designers must address early in their design process to effectively convey their story during a runway show presentation.

Experiencing virtual environments is often limited to abstract interactions with objects. Physical proxies allow users to feel virtual objects, but are often inaccessible. We present the VoxelHap toolkit which enables users to construct highly functional proxy objects using Voxels and Plates. Voxels are blocks with special functionalities that form the core of each physical proxy. Plates increase a proxy’s haptic resolution, such as its shape, texture or weight. Beyond providing physical capabilities to realize haptic sensations, VoxelHap utilizes VR illusion techniques to expand its haptic resolution. We evaluated the capabilities of the VoxelHap toolkit through the construction of a range of fully functional proxies across a variety of use cases and applications. In two experiments with 24 participants, we investigate a subset of the constructed proxies, studying how they compare to a traditional VR controller. First, we investigated VoxelHap’s combined haptic feedback and second, the trade-offs of using ShapePlates. Our findings show that VoxelHap’s proxies outperform traditional controllers and were favored by participants.

Rendering haptic feedback for interactions with virtual objects is an essential part of effective virtual reality experiences. In this work, we explore providing haptic feedback for rotational manipulations, e.g., through knobs. We propose the use of a Pseudo-Haptic technique alongside a physical proxy knob to simulate various physical resistances. In a psychophysical experiment with 20 participants, we found that designers can introduce unnoticeable offsets between real and virtual rotations of the knob, and we report the corresponding detection thresholds. Based on these, we present the Pseudo-Haptic Resistance technique to convey physical resistance while applying only unnoticeable pseudo-haptic manipulation. Additionally, we provide a first model of how C/D gains correspond to physical resistance perceived during object rotation, and outline how our results can be translated to other rotational manipulations. Finally, we present two example use cases that demonstrate the versatility and power of our approach.

Hand redirection is effective so long as the introduced offsets are not noticeably disruptive to users. In this work we investigate the use of physiological and interaction data to detect movement discrepancies between a user's real and virtual hand, pushing towards a novel approach to identify discrepancies which are too large and therefore can be noticed. We ran a study with 22 participants, collecting EEG, ECG, EDA, RSP, and interaction data. Our results suggest that EEG and interaction data can be reliably used to detect visuo-motor discrepancies, whereas ECG and RSP seem to suffer from inconsistencies. Our findings also show that participants quickly adapt to large discrepancies, and that they constantly attempt to establish a stable mental model of their environment. Together, these findings suggest that there is no absolute threshold for possible non-detectable discrepancies; instead, it depends primarily on participants' most recent experience with this kind of interaction.

We interviewed 10 autistic livestreamers to understand their motivations for livestreaming on Twitch. Our participants explained that streaming helped them fulfill social desires by: supporting them in making meaningful social connections with others; giving them a safe space to practice social skills like “small talk”; and empowering them to be autistic role models and to share their true selves. This work offers an early report on how autistic individuals leverage livestreaming as a beneficial social platform while struggling with audience expectations.

Visuo-haptic illusions are a method to expand proxy-based interactions in VR by introducing unnoticeable discrepancies between the virtual and real world. Yet how different design variables affect the illusions with proxies is still unclear. To unpack a subset of variables, we conducted two user studies with 48 participants to explore the impact of (1) different grasping types and movement trajectories, and (2) different grasping types and object masses on the discrepancy which may be introduced. Our Bayes analysis suggests that grasping types and object masses (≤ 500 g) did not noticeably affect the discrepancy, but for movement trajectory, results were inconclusive. Further, we identified a significant difference between (un)restricted movement trajectories. Our data shows considerable differences in participants’ proprioceptive accuracy, which seem to correlate with their prior VR experience. Finally, we illustrate the impact of our key findings on the visuo-haptic illusion design process by showcasing a new design workflow.

Learnability is a core aspect of software usability. Video games are not an exception, as game designers need to teach players how to play their creations. We analyzed 40 contemporary video games to identify how video games approach learning experiences. We found that games have advanced far beyond using simple tutorials or demonstration screens and adopt a range of repeatable and reusable design strategies using visual cues to facilitate learning. We provide a detailed descriptive framework of these design strategies, elucidating how and when they can be used, and describing how the visual cues are used to build them. Our research can be useful for both general HCI researchers and practitioners seeking to tap into the rich ideas from video game learnability design looking for practical solutions for their work.

People are typically involved in different activities while eating, particularly when eating alone, such as watching television or playing games on their phones. Previous research in Human-Food Interaction (HFI) has primarily focused on studying people’s motivation and analyzing of the media content watched while eating. However, their impact on human behavioral and cognitive processes, particularly flavor perception and its attributes, remains underexplored. We present a user study to investigate the influence of six types of videos, including mukbang – a new food video genre, on flavor perceptions (taste sensations, liking, and emotions) while eating plain white rice. Our findings revealed that participants perceived positive emotional changes and reported significant differences in their augmented taste sensations (e.g., spicy and salty) with different food-based videos. Our findings provided insights into using our approach to promote digital commensality and healthier eating (digital augmentation without altering the food), highlighting the scope for future research.

Remote teleoperation is an important robot control method when they cannot operate fully autonomously. Yet, teleoperation presents challenges to effective and full robot utilization: controls are cumbersome, inefficient, and the teleoperator needs to actively attend to the robot and its environment. Inspired by end-user programming, we propose a new interaction paradigm to support robot teleoperation for combinations of repetitive and complex movements. We introduce Mimic, a system that allows teleoperators to demonstrate and save robot trajectories as templates, and re-use them to execute the same action in new situations. Templates can be re-used through (1) macros—parametrized templates assigned to and activated by buttons on the controller, and (2) programs—sequences of parametrized templates that operate autonomously. A user study in a simulated environment showed that after initial set up time, participants completed manipulation tasks faster and more easily compared to traditional direct control.

Wilderness search and rescue (WSAR) is a command-and-control activity where a Command team manages field teams scattered across a large area looking for a lost person. The challenge is that it can be difficult for Command to maintain awareness of field teams and the conditions of the field. We designed RescueCASTR, an interface that explores the idea of deploying field teams with wearable cameras that stream live video or sequential photos periodically to Command that aid contextual awareness. We ran a remote user study with WSAR managers to understand the opportunities and challenges of such a system. We found that the awareness provided by the footage could give additional confidence and comfort to Command, while reducing the need for explicit communications. However, it could also impact workers' traditional roles and shift the burden of responsibility toward Command. We conclude that, while wearable-camera footage could be beneficial to Command, they need to have the tools and means to narrow their focus within the abundance of information provided. Furthermore, camera streams should not be thought of as a replacement for more direct communications, but rather as another tool available to help Command supplement their understanding of events in the field and help them narrow their focus.

Collaborative exploration of 360 videos with contemporary interfaces is challenging because collaborators do not have awareness of one another's viewing activities. Tourgether360 enhances social exploration of 360° tour videos using a pseudo-spatial navigation technique that provides both an overhead "context" view of the environment as a minimap, as well as a shared pseudo-3D environment for exploring the video. Collaborators are embodied as avatars along a track depending on their position in the video timeline and can point and synchronize their playback. We evaluated the Tourgether360 concept through two studies: first, a comparative study with a simplified version of Tourgether360 with collaborator embodiments and a minimap versus a conventional interface; second, an exploratory study where we studied how collaborators used Tourgether360 to navigate and explore 360° environments together. We found that participants adopted the Tourgether360 approach with ease and enjoyed the shared social aspects of the experience. Participants reported finding the experience similar to an interactive social video game.

Many instructors in computing and HCI disciplines use hands-on activities for teaching and training new skills. Beyond simply teaching hands-on skills like sketching and programming, instructors also use these activities so students can acquire tacit skills. Yet, current video-conferencing technologies may not effectively support hands-on activities in online teaching contexts. To develop an understanding of the inadequacies of current video-conferencing technologies for hands-on activities, we conducted 15 interviews with university-level instructors who had quickly pivoted their use of hands-on activities to an online context during the early part of the COVID-19 pandemic. Based on our analysis, we uncovered four pedagogical goals that instructors have when using hands-on activities online and how instructors were unable to adequately address them due to the technological limitations of current video-conferencing tools. Our work provides empirical data about the challenges that many instructors experienced, and in so doing, the pedagogical goals we identify provide new requirements for video-conferencing systems to better support hands-on activities.

Providing haptic feedback when manipulating virtual objects is an essential part of immersive virtual reality experiences; however, it is challenging to replicate all of an object's properties and characteristics. We propose the use of visuo-haptic illusions alongside physical proxies to enhance the scope of proxy-based interactions with virtual objects. In this work, we focus on two manipulation techniques, linear translation and stretching across different distances, and investigate how much discrepancy between the physical proxy and the virtual object may be introduced without participants noticing. In a study with 24 participants, we found that manipulation technique and travel distance significantly affect the detection thresholds, and that visuo-haptic illusions impact performance and accuracy. We show that this technique can be used to enable functional proxy objects that act as stand-ins for multiple virtual objects, illustrating the technique through a showcase VR-DJ application.

Empathy tools are experiences designed to evoke empathetic responses by placing the user in another’s lived and felt experience. The problem is that designers do not have a common vocabulary to describe empathy tool experiences; consequently, it is difficult to compare/contrast empathy tool designs or to think about their efficacy. To address this problem, we analyzed 26 publications on empathy tools to develop a descriptive framework for designers of empathy tools. Based on our analysis, we found that empathy tools can be described along three dimensions: (i) the amount of agency the tool allows, (ii) the user’s perspective while using the tool, and (iii) the type of sensations that are experienced. We show that this framework can be used to describe a wide variety of empathy tools and provide recommendations for empathy tool designers, as well as techniques for measuring the efficacy of an empathy tool experience.

An increasingly popular way of experiencing remote places is by viewing 360° virtual tour videos, which show the surrounding view while traveling through an environment. However, finding particular locations in these videos can be difficult because current interfaces rely on distorted frame previews for navigation. To alleviate this usability issue, we propose Route Tapestries, continuous orthographic-perspective projection of scenes along camera routes. We first introduce an algorithm for automatically constructing Route Tapestries from a 360° video, inspired by the slit-scan photography technique. We then present a desktop video player interface using a Route Tapestry timeline for navigation. An online evaluation using a target-seeking task showed that Route Tapestries allowed users to locate targets 22% faster than with YouTube-style equirectangular previews and reduced the failure rate by 75% compared to a more conventional row-of-thumbnail strip preview. Our results highlight the value of reducing visual distortion and providing continuous visual contexts in previews for navigating 360°virtual tour videos.

Head-mounted displays (HMDs) increase immersion into virtual worlds. The problem is that this limits headset users' awareness of bystanders: headset users cannot attend to bystanders' presence and activities. We call this the HMD boundary. We explore how to make the HMD boundary permeable by comparing different ways of providing informal awareness cues to the headset user about bystanders. We adapted and implemented three visualization techniques (Avatar View, Radar and Presence++) that share bystanders' location and orientation with headset users. We conducted a hybrid user and simulation study with three different types of VR content (high, medium, low interactivity) with twenty participants to compare how these visualization techniques allow people to maintain an awareness of bystanders, and how they affect immersion (compared to a baseline condition). Our study reveals that a see-through avatar representation of bystanders was effective, but led to slightly reduced immersion in the VR content. Based on our findings, we discuss how future awareness visualization techniques can be designed to mitigate the reduction of immersion for the headset user.

We present Activity River, a personal visualization tool which enables individuals to plan, log, and reflect on their self-defined activities. We are interested in supporting this type of reflective practice as prior work has shown that reflection can help people plan and manage their time effectively. Hence, we designed Activity River based on five design goals (visualize historical and contextual data, facilitate comparison of goals and achievements, engage viewers with delightful visuals, support authorship, and enable flexible planning and logging) which we distilled from the Information Visualization and Human-Computer Interaction literature. To explore our approach's strengths and limitations, we conducted a qualitative study of Activity River using a role-playing method. Through this qualitative exploration, we illustrate how our participants envisioned using our visualization to perform dynamic and continuous reflection on their activities. We observed that they were able to assess their progress towards their plans and adapt to unforeseen circumstances using our tool.

We present a mixed-methods study of viewers on their practices and motivations around watching mukbang — video streams of people eating large quantities of food. Viewers' experiences provide insight on future technologies for multisensorial video streams and technology-supported commensality (eating with others). We surveyed 104 viewers and interviewed 15 of them about their attitudes and reflections on their mukbang viewing habits, their physiological aspects of watching someone eat, and their perceived social relationship with mukbangers. Based on our findings, we propose design implications for remote commensality, and for synchronized multisensorial video streaming content.

Mobile augmented reality - where a mobile device is used to view and interact with virtual objects displayed in the real world - is becoming more common. Target selection is the main method of interaction in mobile AR, but is particularly difficult because targets in AR can have challenging characteristics such as being moving or occluded (by digital or real world objects). Because target selection is particularly difficult and error prone in mobile AR, we conduct a comparative study of target assistance techniques. We compared four different cursor-based selection techniques against the standard touch-to-select interaction, finding that a newly adapted Bubble Cursor-based technique performs consistently best across five different target characteristics. Our work provides new findings demonstrating the promise of cursorbased target assistance in mobile AR.

Exploring and manipulating complex virtual objects is challenging due to limitations of conventional controllers and free-hand interaction techniques. We present the TanGi toolkit which enables novices to rapidly build physical proxy objects using Composable Shape Primitives. TanGi also provides Manipulators allowing users to build objects including movable parts, making them suitable for rich object exploration and manipulation in VR. With a set of different use cases and applications we show the capabilities of the TanGi toolkit and evaluate its use. In a study with 16 participants, we demonstrate that novices can quickly build physical proxy objects using the Composable Shape Primitives and explore how different levels of object embodiment affect virtual object exploration. In a second study with 12 participants we evaluate TanGi's Manipulators and investigate the effectiveness of embodied interaction. Findings from this study show that TanGi's proxies outperform traditional controllers and were generally favored by participants.

We built a chatbot system–Audience Bot–that simulates an audience for novice live streamers to engage with while streaming. New live streamers on platforms like Twitch are expected to perform and talk to themselves, even while no one is watching. We ran an observational lab study on how Audience Bot assists novice live streamers as they acclimate to multitasking–simultaneously playing a video game while performing for a (simulated) audience.

We focus on the problem of simulating the haptic infrastructure of a virtual environment (i.e. walls, doors). Our approach relies on multiple ZoomWalls---autonomous robotic encounter-type haptic wall-shaped props---that coordinate to provide haptic feedback for room-scale virtual reality. Based on a user's movement through the physical space, ZoomWall props are coordinated through a predict-and-dispatch architecture to provide just-in-time haptic feedback for objects the user is about to touch. To refine our system, we conducted simulation studies of different prediction algorithms, which helped us to refine our algorithmic approach to realize the physical ZoomWall prototype. Finally, we evaluated our system through a user experience study, which showed that participants found that ZoomWalls increased their sense of presence in the VR environment. ZoomWalls represents an instance of autonomous mobile reusable props, which we view as an important design direction for haptics in VR.

Wilderness search and rescue (WSAR) requires careful communication between workers in different locations. To understand the contexts from which WSAR workers communicate and the challenges they face, we interviewed WSAR workers and observed a mock-WSAR scenario. Our findings illustrate that WSAR workers face challenges in maintaining a shared mental model. This is primarily done through distributed communication using two-way radios and cell phones for text and photo messaging; yet both implicit and explicit communication suffer. WSAR workers send messages for various reasons and share different types of information with varying levels of urgency. This warrants the use of multiple communication modalities and information streams. However, bringing in more modalities introduces the risk of information overload, and thus WSAR workers today still primarily communicate remotely via the radio. Our work demonstrates opportunities for technology to provide implicit communication and awareness remotely, and to help teams maintain a shared mental model even when synchronous realtime communication is sparse. Furthermore, technology should be designed to bring together multiple streams of information and communication while making sure that they are presented in ways that aid WSAR workers rather than overwhelming them.

Drawing upon multiple disciplines, avant-garde fashion-tech teams push the boundaries between fashion and technology. Many are well trained in envisioning aesthetic qualities of garments, but few have formal training on designing and fabricating technologies themselves. We introduce Mannequette, a prototyping tool for fashion-tech garments that enables teams to experiment with interactive technologies at early stages of their design processes. Mannequette provides an abstraction of light-based outputs and sensor-based inputs for garments through a DJ mixer-like interface that allows for dynamic changes and recording/playback of visual effects. The base of Mannequette can also be incorporated into the final garment, where it is then connected to the final components. We conducted an 8-week deployment study with eight design teams who created new garments for a runway show. Our results revealed Mannequette allowed teams to repeatedly consider new design and technical options early in their creative processes, and to communicate more effectively across disciplinary backgrounds.

Wind simulations are typically one-off implementations for specific applications. We introduce WindyWall, a platform for creative design and exploration of wind simulations. WindyWall is a three-panel 90-fan array that encapsulates users with 270? of wind coverage. We describe the design and implementation of the array panels, discussing how the panels can be re-arranged, where various wind simulations can be realized as simple effects. To understand how people perceive "wind" generated from WindyWall, we conducted a pilot study of wind magnitude perception using different wind activation patterns from WindyWall. Our findings suggest that: horizontal wind activations are perceived more readily than vertical ones, and that people's perceptions of wind are highly variable-most individuals will rate airflow differently in subsequent exposures. Based on our findings, we discuss the importance of developing a method for characterizing wind simulations, and provide design directions for others using fan arrays to simulate wind.

Because gesture design for augmented reality (AR) remains idiosyncratic, people cannot necessarily use gestures learned in one AR application in another. To design discoverable gestures, we need to understand what gestures people expect to use. We explore how the scale of AR affects the gestures people expect to use to interact with 3D holograms. Using an elicitation study, we asked participants to generate gestures in response to holographic task referents, where we varied the scale of holograms from desktop-scale to room-scale objects. We found that the scale of objects and scenes in the AR experience moderates the generated gestures. Most gestures were informed by physical interaction, and when people interacted from a distance, they sought a good perspective on the target object before and during the interaction. These results suggest that gesture designers need to account for scale, and should not simply reuse gestures across different hologram sizes.

Based on an analysis of 49 popular contemporary video games, we develop a descriptive framework of visual interaction cues in video games. These cues are used to inform players what can be interacted with, where to look, and where to go within the game world. These cues vary along three dimensions: the purpose of the cue, the visual design of the cue, and the circumstances under which the cue is shown. We demonstrate that this framework can also be used to describe interaction cues for augmented reality applications. Beyond this, we show how the framework can be used to generatively derive new design ideas for visual interaction cues in augmented reality experiences.

Remote collaborators working together on physical objects have difficulty building a shared understanding of what each person is talking about. Conventional video chat systems are insufficient for many situations because they present a single view of the object in a flattened image. To understand how this limited perspective affects collaboration, we designed the Remote Manipulator (ReMa), which can reproduce orientation manipulations on a proxy object at a remote site. We conducted two studies with ReMa, with two main findings. First, a shared perspective is more effective and preferred compared to the opposing perspective offered by conventional video chat systems. Second, the physical proxy and video chat complement one another in a combined system: people used the physical proxy to understand objects, and used video chat to perform gestures and confirm remote actions.

People often enjoy sharing outdoor activities together such as walking and hiking. However, when family and friends are separated by distance it can be difficult if not impossible to share such activities. We explore this design space by investigating the benefits and challenges of using a telepresence robot to support outdoor leisure activities. In our study, participants participated in the outdoor activity of geocaching where one person geocached with the help of a remote partner via a telepresence robot. We compared a wide field of view (WFOV) camera to a 360° camera. Results show the benefits of having a physical embodiment and a sense of immersion with the 360° view. Yet challenges related to a lack of environmental awareness, safety issues, and privacy concerns resulting from bystander interactions. These findings illustrate the need to design telepresence robots with the environment and public in mind to provide an enhanced sensory experience while balancing safety and privacy issues resulting from being amongst the general public.

In the physical world, teammates develop situation awareness about each other's location, status, and actions through cues such as gaze direction and ambient noise. To support situation awareness, distributed multiplayer games provide awareness cues - information that games automatically make available to players to support cooperative gameplay. The design of awareness cues can be extremely complex, impacting how players experience games and work with teammates. Despite the importance of awareness cues, designers have little beyond experiential knowledge to guide their design. In this work, we describe a design framework for awareness cues, providing insight into what information they provide, how they communicate this information, and how design choices can impact play experience. Our research, based on a grounded theory analysis of current games, is the first to provide a characterization of awareness cues, providing a palette for game designers to improve design practice and a starting point for deeper research into collaborative play.

We present a concept-based world building approach, realized in a system called VR Safari Park, which allows users to rapidly create and manipulate a world simulation. Conventional world building tools focus on the manipulation and arrangement of entities to set up the simulation, which is time consuming as it requires frequent view and entity manipulations. Our approach focuses on a far simpler mechanic, where users add virtual blocks which represent world entities (e.g. animals, terrain, weather, etc.) to a World Tree, which represents the simulation. In so doing, the World Tree provides a quick overview of the simulation, and users can easily set up scenarios in the simulation without having to manually perform fine-grain manipulations on world entities. A preliminary user study found that the proposed interface is effective and usable for novice users without prior immersive VR experience.

We designed a videochat experience where one participant can experience a remote environment from a 360° camera. This allows the remote user to view and explore the environment without necessitating interaction from the local participant. We designed and conducted an observational study to understand the experience, and the challenges that people might encounter. In a study with 32 participants (16 pairs), we found that remote participants could actively participate in the experience with the environment in ways that are not possible with current mobile video chat. However, we also found that participants had challenges in communicating location and orientation information because many of common communication resources we rely on in collocated chat are not available. Based on these findings, we discuss how future mobile video chat systems need to balance immersion with interaction ease.

360° videos are made using omnidirectional cameras that capture a sphere around the camera. Viewers get an immersive experience by freely changing their field of view around the sphere. The problem is that current interfaces are designed for a single user, and we do not know what challenges groups of people will have when viewing these videos together. We report on the findings of a study where 16 pairs of participants watched 360° videos together in a "guided tour" scenario. Our findings indicate that while participants enjoyed the ability to view the scene independently, this caused challenges establishing joint references, leading to breakdowns in conversation. We conclude by discussing how gaze awareness widgets and gesturing mechanisms may support smoother collaborative interaction around collaborative viewing of 360° videos.

Groupware research has long focused on representing gestures as a means to facilitate collaboration. However, this work has not led to wide support of gesturing in commercial groupware systems. In contrast, Dota 2, a popular MOBA game, provides two frequently-used gesturing tools: annotations - freely drawn lines on top of the gamespace - and pings - a combination of animation and sound indicating a point of interest. While gesturing tools are important for quickly coordinating with teammates in Dota 2, there is little information about how and why people use them. To gather this information, we performed two complementary studies: an interaction analysis of eight game replays, and a survey of 167 experienced players. Our findings include: six distinct motivations for the use of gesturing tools; when and how frequently gesture motivations occur during games; and, that players find pings an essential tool for winning, but not annotations. Our findings provide new directions for the design of gesturing tools in groupware and online games.

People are increasingly using mobile video to communicate, collaborate, and share experiences while on the go. Yet this presents challenges in adequately sharing camera views with remote users. In this paper, we study the use of semi-autonomous drones for video conferencing, where an outdoor user (using a smartphone) is connected to a desktop user who can explore the environment from the drone's perspective. We describe findings from a study where pairs collaborated to complete shared navigation and search tasks. We illustrate the benefits of providing the desktop user with a view that is elevated, manipulable, and decoupled from the outdoor user. In addition, we articulate how participants overcame challenges in communicating environmental information and navigational cues, negotiated control of the view, and used the drone as a tool for sharing experiences. This provides a new way of thinking about mobile video conferencing where cameras that are decoupled from both users play an integral role in communication, collaboration, and sharing experiences.

Mobile devices have begun to raise questions around the potential for overuse when in the presence of family or friends. As such, we conducted a diary and interview study to understand how people use mobile devices in the presence of others at home, and how this shapes their behavior and household dynamics. Results show that family members become frustrated when others do non-urgent activities on their phones in the presence of others. Yet people often guess at what others are doing because of the personal nature of mobile devices. In some cases, people developed strategies to provide a greater sense of activity awareness to combat the problem. Mobile phone usage was sometimes perceived as beneficial by providing a mechanism for needed disengagement from family members. These findings suggest several opportunities for redesigning mobile device software to mitigate emergent frustrations, and open up new opportunities for nurturing social interactions among family members.

Recent mobile technology has provided new opportunities for creating remote assistance systems. However, mobile support systems present a particular challenge: both the camera and display are held by the user, leading to shaky video. When pointing or drawing annotations, this means that the desired target often moves, causing the gesture to lose its intended meaning. To address this problem, we investigate annotation stabilization techniques, which allow annotations to stick to their intended location. We studied two annotation systems, using three different forms of annotations, with both tablets and head-mounted displays. Our analysis suggests that stabilized annotations and head-mounted displays are only beneficial in certain situations. However, the simplest approach of automatically freezing video while drawing annotations was surprisingly effective in facilitating the completion of remote assistance tasks.

Dramatic advances in sensor and computing miniaturization for personal data collection are making Personal Informatics (PI) tools a reality. Yet, advances in data collection have not been matched with similar advances in tools to promote, support, and facilitate reflection on this data. This gap leaves people with large swaths of data, but very little understanding of how to make sense of the data or to derive actionable insights. In this work, we explore a process called shared reflection, where individuals are paired with other data collectors, and asked (through prompts) to reflect on one another?s data. Based on a six-week study where 15 participants collected different kinds of personal data and engaged in a shared reflection process, we show that participants gained transformative insights from others' reflections on their data. While this was promising, we discuss practical challenges in deploying this idea into real world personal informatics tools. In particular, while shared reflection can be appropriated to effectively bootstrap reflection on one's data, this needs to be balanced against privacy and control concerns.

Software is typically developed incrementally and released in stages. Planning these releases involves deciding which features of the system should be implemented for each release. This is a complex planning process involving numerous trade-offs-constraints and factors that often make decisions difficult. Since the success of a product depends on this plan, it is important to understand the trade-offs between different release plans in order to make an informed choice. We present STRATOS, a tool that simultaneously visualizes several software release plans. The visualization shows several attributes about each plan that are important to planners. Multiple plans are shown in a single layout to help planners find and understand the trade-offs between alternative plans. We evaluated our tool via a qualitative study and found that STRATOS enables a range of decision-making processes, helping participants decide on which plan is most optimal.

Mobile video conferencing, where one or more participants are moving about in the real world, enables entirely new interaction scenarios (e.g., asking for help to construct or repair an object, or showing a physical location). While we have a good understanding of the challenges of video conferencing in office or home environments, we do not fully understand the mechanics of camera work-how people use mobile devices to communicate with one another-during mobile video calls. To provide an understanding of what people do in mobile video collaboration, we conducted an observational study where pairs of participants completed tasks using a mobile video conferencing system. Our analysis suggests that people use the camera view deliberately to support their interactions-for example, to convey a message or to ask questions-but the limited field of view, and the lack of camera control can make it a frustrating experience.

We present results from a study examining how the physical layout of a project room and task affect the cognitive maps acquired of a connected virtual environment during mixed-presence collaboration. Results indicate that a combination of physical layout and task impacts cognitive maps of the virtual space. Participants did not form a strong model of how different physical work regions were situated relative to each other in the virtual world when the tasks performed in each region differed. Egocentric perspectives of multiple displays enforced by different furniture arrangements encouraged cognitive maps of the virtual world that reflected these perspectives, when the displays were used for the same task. These influences competed or coincided with document-based, audiovisual and interface cues, influencing collaboration. We consider the implications of our findings on WYSIWIS mappings between real and virtual for mixed-presence collaboration.

Physiotherapy patients exercising at home alone are at risk of re-injury since they do not have corrective guidance from a therapist. To explore solutions to this problem, we designed Physio@Home, a prototype that guides people through prerecorded physiotherapy exercises using real-time visual guides and multi-camera views. Our design addresses several aspects of corrective guidance, including: plane and range of movement, joint positions and angles, and extent of movement. We evaluated our design, com-paring how closely people could follow exercise movements under various feedback conditions. Participants were most accurate when using our visual guide and multi-views. We provide suggestions for exercise guidance systems drawn from qualitative findings on visual feedback complexity.

Data surrounds each and every one of us in our daily lives, ranging from exercise logs, to archives of our interactions with others on social media, to online resources pertaining to our hobbies. There is enormous potential for us to use these data to understand ourselves better and make positive changes in our lives. Visualization (Vis) and visual analytics (VA) offer substantial opportunities to help individuals gain insights about themselves, their communities and their interests; however, designing tools to support data analysis in non-professional life brings a unique set of research and design challenges. We investigate the requirements and research directions required to take full advantage of Vis and VA in a personal context. We develop a taxonomy of design dimensions to provide a coherent vocabulary for discussing personal visualization and personal visual analytics. By identifying and exploring clusters in the design space, we discuss challenges and share perspectives on future research. This work brings together research that was previously scattered across disciplines. Our goal is to call research attention to this space and engage researchers to explore the enabling techniques and technology that will support people to better understand data relevant to their personal lives, interests, and needs.

If visualization is to be democratized, we need to provide means for non-experts to create visualizations that allow them to engage directly with datasets. We present constructive visualization a new paradigm for the simple creation of flexible, dynamic visualizations. Constructive visualization is simple-in that the skills required to build and manipulate the visualizations are akin to kindergarten play; it is expressive in that one can build within the constraints of the chosen environment, and it also supports dynamics -- in that these constructed visualizations can be rebuilt and adjusted. We de- scribe the conceptual components and processes underlying constructive visualization, and present real-world examples to illustrate the utility of this approach. The constructive visualization approach builds on our inherent understanding and experience with physical building blocks, offering a model that enables non-experts to create entirely novel visualizations, and to engage with datasets in a manner that would not have otherwise been possible.

Children engage in free play for emotional, physical and social development; researchers have explored supporting free play between physically remote playmates using videoconferencing tools. We show that the configuration of the video conferencing setup affects play. Specifically, we show that a shared visual scene configuration promotes fundamentally active forms of engaged, co-operative play.

We present the design, and evaluation of WaaZam, a video mediated communication system designed to support creative play in customized environments. Users can interact together in virtual environments composed of digital assets layered in 3D space. The goal of the project is to support creative play and increase social engagement during video sessions of geographically separated families. We try to understand the value of customization for individual families with children ages 6-12. We present interviews with creativity experts, a pilot study and a formal evaluation of families playing together in four conditions: separate windows, merged windows, digital play sets, and customized digital environments. We found that playing in the same video space enables new activities and increases social engagement for families. Customization allows families to modify scenes for their needs and support more creative play activities that embody the imagination of the child.

Our research explores the use of mobile video chat in public spaces by people participating in parallel experiences, where both a local and remote person are doing the same activity together at the same time. We prototyped a wearable video chat experience and had pairs of friends and family members participate in 'shared geocaching' over distance. Our results show that video streaming works best for navigation tasks but is more challenging to use for fine-grained searching tasks. Video streaming also creates a very intimate experience with a remote partner, but this can lead to distraction from the 'real world' and even safety concerns. Overall, privacy concerns with streaming from a public space were not typically an issue; however, people tended to rely on assumptions of what were acceptable. The implications are that designers should consider appropriate feedback, user disembodiment, and asymmetry when designing for parallel experiences.

Gestures are a ubiquitous part of human communication over tables, but when tables are distributed, gestures become difficult to capture and represent. There are several problems: extracting arm images from video, representing the height of the gesture, and making the arm embodiment visible and understandable at the remote table. Current solutions to these problems are often expensive, complex to use, and difficult to set up. We have developed a new toolkit - KinectArms - that quickly and easily captures and displays arm embodiments. KinectArms uses a depth camera to segment the video and determine gesture height, and provides several visual effects for representing arms, showing gesture height, and enhancing visibility. KinectArms lets designers add rich arm embodiments to their systems without undue cost or development effort, greatly improving the expressiveness and usability of distributed tabletop groupware.

We deployed the ShareTable - a system that provides easy-to-initiate videochat and a shared tabletop task space - in four divorced households. Throughout the month of its use, the families employed the ShareTable to participate in shared activities, share emotional moments, and communicate closeness through metaphorical touch. The ShareTable provided a number of advantages over the phone and was easier to use than standard videoconferencing. However, it did also introduce concerns over privacy and new sources of conflict about appropriate calling practices. We relate our findings to the larger research landscape and present implications for future work.

Researchers have developed interaction concepts based on mobile projectors. Yet pursuing work in this area - particularly in building projector-based interactions techniques within an application - is cumbersome and time-consuming. To mitigate this problem, we contribute ProjectorKit, a flexible open-source toolkit that eases rapid prototyping mobile projector interaction techniques.

Portable paper calendars (i. e., day planners and organizers) have greatly influenced the design of group electronic calendars. Both use time units (hours/days/weeks/etc.) to organize visuals, with useful information (e.g., event types, locations, attendees) usually presented as - perhaps abbreviated or even hidden - text fields within those time units. The problem is that, for a group, this visual sorting of individual events into time buckets conveys only limited information about the social network of people. For example, people's whereabouts cannot be read 'at a glance' but require examining the text. Our goal is to explore an alternate visualization that can reflect and illustrate group members' calendar events. Our main idea is to display the group's calendar events as spatiotemporal activities occurring over a geographic space animated over time, all presented on a highly interactive public display. In particular, our Spalendar (Spatial Calendar) design animates people's past, present and forthcoming movements between event locations as well as their static locations. Detail of people's events, their movements and their locations is progressively revealed and controlled by the viewer's proximity to the display, their identity, and their gestural interactions with it, all of which are tracked by the public display.

During a live sports event, many sports fans use social media as a part of their viewing experience, reporting on their thoughts on the event as it unfolds. In this work, we use this information stream to semantically annotate live broadcast sports games, using these annotations to select video highlights from the game. We demonstrate that this approach can be used to select highlights specific for fans of each team, and that these clips reflect the emotions of a fan during a game. Further, we describe how these clips differ from those seen on nightly sportscasts.

We explore how expert First Person Shooter (FPS) players coordinate actions using a shared voice channel. Our findings emphasize the importance of the temporality and spatiality of these tactical verbal communications ("call-outs"). From here, we outline potential designs to mitigate problems in the production/interpretation of call-outs to better support coordination.

Eating with others, or commensality, is an enjoyable activity that serves many important social functions; however, many individuals eat meals alone due to life circumstances, meaning that they miss out on these social benefits. We developed and deployed a simple technology probe providing social awareness around mealtimes to explore how social systems might help alleviate the loneliness of solitary dining. Our findings suggest that these systems can convey a sense of connectedness around a meal; further, our analysis revealed three themes relevant to systems of this type: that contextually-located peripheral awareness engenders connectedness; that such tools can foster a feeling of shared social presence, and that they can be a catalyst for other forms of communication around the meal. These findings suggest that "remote commensality" is not only possible, but that it may take on forms entirely different to that which we are accustomed.

Modern mobile devices allow a rich set of multi-finger interactions that combine modes into a single fluid act, for example, one finger for panning blending into a two-finger pinch gesture for zooming. Such gestures require the use of both hands: one holding the device while the other is interacting. While on the go, however, only one hand may be available to both hold the device and interact with it. This mostly limits interaction to a single-touch (i.e., the thumb), forcing users to switch between input modes explicitly. In this paper, we contribute the Fat Thumb interaction technique, which uses the thumb's contact size as a form of simulated pressure. This adds a degree of freedom, which can be used, for example, to integrate panning and zooming into a single interaction. Contact size determines the mode (i.e., panning with a small size, zooming with a large one), while thumb movement performs the selected mode. We discuss nuances of the Fat Thumb based on the thumb's limited operational range and motor skills when that hand holds the device. We compared Fat Thumb to three alternative techniques, where people had to precisely pan and zoom to a predefined region on a map and found that the Fat Thumb technique compared well to existing techniques.

Modern mobile devices rely on the screen as a primary input modality. Yet the small screen real-estate limits interaction possibilities, motivating researchers to explore alternate input techniques. Within this arena, our goal is to develop Body-Centric Interaction with Mobile Devices: a class of input techniques that allow a person to position and orient her mobile device to navigate and manipulate digital content anchored in the space on and around the body. To achieve this goal, we explore such interaction in a bottom-up path of prototypes and implementations. From our experiences, as well as by examining related work, we discuss and present three recurring themes that characterize how these interactions can be realized. We illustrate how these themes can inform the design of Body-Centric Interactions by applying them to the design of a novel mobile browser application. Overall, we contribute a class of mobile input techniques where interactions are extended beyond the small screen, and are instead driven by a person's movement of the device on and around the body.

In this paper, we introduce approaches to navigating and manipulating objects in a Collaborative Virtual Environment (CVE) that engage tangible objects and an interactive table interface. We also identify three design concerns that are common to physical-virtual connectivity for interaction with CVE systems. At last, we propose solutions to these issues within the context of CVEs.

Applications that provide location-based experiences are an increasingly viable design space given the proliferation of GPS-enabled mobile devices. However, these applications are in their infancy, and we do not yet know what design factors will contribute to their success. For this reason, we have studied the well-established location-based experience of geocaching. We report on the results of a survey of geocachers along with observations from our own in-depth geocaching activities. Our findings illustrate that geocaching permits users to create a range of experiences for others within a permeable yet restricted culture of norms. Once created, geocaches are maintained by the community of geocachers through a well-designed groupware system. Here maintenance acts can be performed “in the small, ” given their lightweight and well-defined nature, and become less about maintenance and more about personal participation. These findings provide insight into how community and groupware can be leveraged to support applications for location-based experiences.

We explore the design of a system for three-way collaboration over a shared visual workspace, specifically in how to support three channels of communication: person, reference, and task-space. In two studies, we explore the implications of extending designs intended for dyadic collaboration to three-person groups, and the role of each communication channel. Our studies illustrate the utility of multiple configurations of users around a distributed workspace, and explore the subtleties of traditional notions of identity, awareness, spatial metaphor, and corporeal embodiments as they relate to three-way collaboration.

In this paper, we explore the practice of using a whiteboard for multiple tasks, and specifically how users employ whiteboards to smoothly transition between related sets of tasks. Our study underscores several basic, but important affordances of whiteboards that support this practice, including visual persistence, flexibility of interaction primitives, and their situated physicality. We discuss the implications of these findings for the design of large display applications.

Video slicing---a variant of slit scanning in photography---extracts a scan line from a video frame and successively adds that line to a composite image over time. The composite image becomes a time line, where its visual patterns reflect changes in a particular area of the video stream. We extend this idea of video slicing by allowing users to draw marks anywhere on the source video to capture areas of interest. These marks, which we call slit-tears, are used in place of a scan line, and the resulting composite timeline image provides a much richer visualization of the video data. Depending on how tears are placed, they can accentuate motion, small changes, directional movement, and relational patterns.

To design information visualization tools for collaborative use, we need to understand how teams engage with visualizations during their information analysis process. We report on an exploratory study of individuals, pairs, and triples engaged in information analysis tasks using paper-based visualizations. From our study results, we derive a framework that captures the analysis activities of co-located teams and individuals. Comparing this framework with existing models of the information analysis process suggests that information visualization tools may benefit from providing a flexible temporal flow of analysis actions.

In this paper, we reflect on the design and deployment process of MAGICBoard, a public display deployed in a university setting that solicits the electronic votes and opinions of bystanders on trivial but amusing topics. We focus on the consequences of our design choices with respect to encouraging bystanders to interact with the public display. Bystanders are individuals around the large display who may never fully engage with the application itself, but are potential contributors to the system. Drawing on our recent experiences with MAGICBoard, we present a classification of bystanders, and then discuss three design themes relevant to the design of systems for bystander use: graduated proximal engagement, lowering barriers for interaction and supporting covert engagement.

We introduce MultiPresenter, a novel presentation system designed to work on very large display spaces (multiple displays or physically large high-resolution displays). MultiPresenter allows presenters to organize and present pre-made and dynamic presentations that take advantage of a very large display space accessed from a personal laptop. Presenters can use the extra space to provide long-term persistency of information to the audience. Our design deliberately separates content generation (authoring) from the presentation of content. We focus on supporting presentation flow and a variety of presentation styles, ranging from automated, scripted sequences of pre-made slides to highly dynamic ad-hoc, and non-linear content. By providing smooth transition between these styles, presenters can easily alter the flow of content during a presentation to adapt to an audience or to change emphasis in response to emerging interests. We describe our goals, rationale and the design process, providing a detailed description of the current version of the system, and discuss our experience using it throughout a one-semester first year computer science course.

We introduce Shadow Reaching, an interaction technique that makes use of a perspective projection applied to a shadow representation of a user. The technique was designed to facilitate manipulation over large distances and enhance understanding in collaborative settings. We describe three prototype implementations that illustrate the technique, examining the advantages of using shadows as an interaction metaphor to support single users and groups of collaborating users. Using these prototypes as a design probe, we discuss how the three components of the technique (sensing, modeling, and rendering) can be accomplished with real (physical) or computed (virtual) shadows, and the benefits and drawbacks of each approach.

Designing collaborative interfaces for tabletops remains difficult because we do not fully understand how groups coordinate their actions when working collaboratively over tables. We present two observational studies of pairs completing independent and shared tasks that investigate collaborative coupling, or the manner in which collaborators are involved and occupied with each other's work. Our results indicate that individuals frequently and fluidly engage and disengage with group activity through several distinct, recognizable states with unique characteristics. We describe these states and explore the consequences of these states for tabletop interface design.

Email use in the context of everyday work practices, or email flow, has not been heavily studied. We present the results of a pair of studies examining how users interlace email with their day-to-day, ongoing work processes. We demonstrate that our subjects use email as a tool for managing moment-to-moment attention and task focus. We also provide a model of this workflow that builds upon an existing model by Venolia et al. Finally, we provide specific design recommendations to enhance the usability of email clients in support of these modes of interaction.

Previous research has shown that rotation and orientation of items plays three major roles during collaboration: comprehension, coordination and communication. Based on these roles of orientation and advice from kinesiology research, we have designed the Rotate'N Translate (RNT) interaction mechanism, which provides integrated control of rotation and translation using only a single touch-point for input. We present an empirical evaluation comparing RNT to a common rotation mechanism that separates control of rotation and translation. Results of this study indicate RNT is more efficient than the separate mechanism and better supports the comprehension, coordination and communication roles of orientation.

Visual information overload is a threat to the interpretation of displays presenting large data sets or complex application environments. To combat this problem, researchers have begun to explore how haptic feedback can be used as another means for information transmission. In this paper, we show that people can perceive and accurately process haptically rendered ordinal data while under cognitive workload. We evaluate three haptic models for rendering ordinal data with participants who were performing a taxing visual tracking task. The evaluation demonstrates that information rendered by these models is perceptually available even when users are visually busy. This preliminary research has promising implications for haptic augmentation of visual displays for information visualization.