Via Research through Design (RtD), we explore the potential of headphones as a general-purpose input device for both foreground motion-gestures as well as background sensing of user activity. As a familiar wearable device, headphones offer a compelling site for head-situated interaction and sensing. Using emerging sensing modalities such as inertial motion, capacitive touch sensing, and depth cameras, our implemented prototypes explore sensing and interaction techniques that offer a range of compelling capabilities.

We present Escapement, a video prototyping tool that introduces a powerful new concept for prototyping screen-based interfaces by flexibly mapping sensor values to dynamic playback control of videos. This recasts the time dimension of video mock-ups as sensor-mediated interaction.

Documentation can support design work and create opportunities for learning and reflection. We explore how a novel documentation tool for a remote interaction design course provides insight into design process and integrates strategies from expert practice to support studio-style collaboration and reflection. Using Research through Design, we develop and deploy Kaleidoscope, an online tool for documenting design process, in an upper-level HCI class during the COVID-19 pandemic, iteratively developing it in response to student feedback and needs. We discuss key themes from the real-world deployment of Kaleidoscope, including: tensions between documentation and creation; effects of centralizing discussion; privacy and visibility in shared spaces; balancing evidence of achievement with feelings of overwhelm; and the effects of initial perceptions and incentives on tool usage. These successes and challenges provide insights to guide future tools for design documentation and HCI education that scaffold learning process as an equal partner to execution.

Creative practice often requires persevering through moments of ambiguity, where the outcome of a process is unclear. Creative practitioners intentionally manage this process, for example by developing strategies to break out of creative ruts, or stay motivated through uncertainty. Understanding the way experts engage with and manage these creativity-relevant processes represents a rich source of foundational knowledge for designers of Creativity Support Tools. These strategies represent an opportunity for CST research: to create CSTs that embody emotional and process-focused strategies and techniques. Through interviews with expert practitioners in diverse domains including performance, craft, engineering, and design, we identify four strategies for managing process: Strategic Forgetting, Mode Switching, Embodying Process, and Aestheticizing. Understanding tool- and domain-agnostic creative strategies used by experts to manage their own creative process can inform the design of future CSTs that amplify the benefits of successful strategies and scaffold new techniques.

To better ground technical (systems) investigation and interaction design of cross-device experiences, we contribute an in-depth survey of existing multi-device practices, including fragmented workflows across devices and the way people physically organize and configure their workspaces to support such activity. Further, this survey documents a historically significant moment of transition to a new future of remote work, an existing trend dramatically accelerated by the abrupt switch to work-from-home (and having to contend with the demands of home-at-work) during the COVID-19 pandemic. We surveyed 97 participants, and collected photographs of home setups and open-ended answers to 50 questions categorized in 5 themes. We characterize the wide range of multi-device physical configurations and identify five usage patterns, including: partitioning tasks, integrating multi-device usage, cloning tasks to other devices, expanding tasks and inputs to multiple devices, and migrating between devices. Our analysis also sheds light on the benefits and challenges people face when their workflow is fragmented across multiple devices. These insights have implications for the design of multi-device experiences that support people’s fragmented workflows.

We introduce Friendscope, an instant, in-the-moment experience sharing system for lightweight commercial camera glasses. Friendscope explores a new concept called a shared camera. This concept allows a wearer to share control of their camera with a remote friend, making it possible for both people to capture photos/videos from the camera in the moment. Through a user study with 48 participants, we found that users felt connected to each other, describing the shared camera as a more intimate form of livestreaming. Moreover, even privacy-sensitive users were able to retain their sense of privacy and control with the shared camera. Friendscope's different shared camera configurations give wearers ultimate control over who they share the camera with and what photos/videos they share. We conclude with design implications for future experience sharing systems.

Version control systems are powerful tools for managing history information and shaping personal and collaborative processes. While many complex tools exist for software engineering, and basic functionality for capturing versions is often found in collaborative applications such as text editors and design layout tools, these systems are not attuned to the needs and behaviors of creative practitioners within those domains, and fail to support creative practitioners in many others. Through 18 semi-structured interviews across diverse domains of creativity, we investigate how creative practitioners use version histories in their process. With the familiar paradigms and features of software version control as an organizing structure, we discuss how these creative practitioners embrace, challenge, and complicate uses of version histories in four ways: using versions as a palette of materials, providing confidence and freedom to explore, leveraging low-fidelity version capture, and reflecting on and reusing versions across long time scales. We discuss how the themes present across this wide range of mediums and domains can provide insight into future designs and uses of version control systems to support creative process.

Work combining live performance and technology often involves incorporating technology directly into the performance as it occurs onstage, including interactive costumes, or performer-controlled sets, lighting or sound. We invert this common approach, developing technology-mediated experiences outside the temporal and spatial confines of a live theatre production. We describe the 4-month co-design process with expert theatre practitioners, and detail how the process 1) shaped our design guidelines and 2) expands the discussion around existing best practices for cross-disciplinary collaboration. In the style of research through design, we present three annotated prototypes: the Augmented Playbill, the Prayer Wheel, and Tarot Cards as well as accompanying AR applications to convey the decisions we made and the philosophy we iteratively developed throughout the project. These artifacts also embody our six design guidelines: resonant affordances, extended narrative, reflective interaction, selective reveal, personalized experience, and privileged access.

AirConstellations supports a unique semi-fixed style of cross-device interactions via multiple self-spatially-aware armatures to which users can easily attach (or detach) tablets and other devices. In particular, AirConstellations affords highly flexible and dynamic device formations where the users can bring multiple devices together in-air — with 2–5 armatures poseable in 7DoF within the same workspace — to suit the demands of their current task, social situation, app scenario, or mobility needs. This affords an interaction metaphor where relative orientation, proximity, attaching (or detaching) devices, and continuous movement into and out of ad-hoc ensembles can drive context-sensitive interactions. Yet all devices remain self-stable in useful configurations even when released in mid-air.

An exciting, expanding palette of hybrid materials is emerging that can be programmed to actuate by a range of external and internal stimuli. However, there exists a dichotomy between the physicality of the actuators and the intangible computational signal that is used to program them. For material practitioners, this lack of physical cues limits their ability to engage in a "conversation with materials" (CwM). This paper presents a creative workstation for supporting this epistemological style by bringing a stronger physicality to the computational signal and balance the conversation between physical and digital actors. The station utilizes a streaming architecture to distribute control across multiple devices and leverage the rich spatial cognition that a physical space affords. Through a formal user study, we characterize the actuation design practice supported by the CwM workstation and discuss opportunities for tangible interfaces to hybrid materials.

Failure, whether it be "complete-and-utter" or "a minor setback", occurs in a variety of different creative practices, yet how it is perceived, handled, and recovered from is a lesser explored design space. Failing to address these perceptions of failure can have psychological repercussions, discourage users from continuing a practice, and form cultural stigma such as those associated with STEM fields. However, mediating practices to develop a culture of resiliency and perseverance is key to sustaining a (lifelong) practice and reshaping pedagogical strategies. In this work, we outline the design space of "guardians", or elements of a creative practice that mitigate the psychological effects of failure. Through contextual inquiry, we contribute an inventory of failure-mitigation strategies from a variety of creative disciplines. We synthesize guidelines for the design of new guardians and present a preliminary exploration of guardians for the lasercutting practice -- effigies and test tags.

As the landscape of wearable devices continues to expand, power remains a major issue for adoption, usability, and miniaturization. Users are faced with an increasing number of personal devices to manage, charge, and care for. In this work, we argue that power constraints limit the design space of wearable devices. We present AlterWear: an architecture for new wearable devices that implement a batteryless design using electromagnetic induction via NFC and bistable e-ink displays. Although these displays are active only when in proximity to an NFC-enabled device, this unique combination of hardware enables both quick, dynamic and long-term interactions with persistent visual displays. We demonstrate new wearables enabled through AlterWear with dynamic, fashion-forward, and expressive displays across several form factors, and evaluate them in a user study. By forgoing the need for onboard power, AlterWear expands the ecosystem of functional and fashionable wearable technologies.

Human hair is a cultural material, with a rich history displaying individuality, cultural expression and group identity. It is malleable in length, color and style, highly visible, and embedded in a range of personal and group interactions. As wearable technologies move ever closer to the body, and embodied interactions become more common and desirable, hair presents a unique and little-explored site for novel interactions. In this paper, we present an exploration and working prototype of hair as a site for novel interaction, leveraging its position as something both public and private, social and personal, malleable and permanent. We develop applications and interactions around this new material in HäirIÖ: a novel integration of hair-based technologies and braids that combine capacitive touch input and dynamic output through color and shape change. Finally, we evaluate this hair-based interactive technology with users, including the integration of HäirIÖ within the landscape of existing wearable and mobile technologies.

Recent digital fabrication tools have enabled new form-giving using a wide range of physical materials. However, light as a first class creative material has been largely ignored within the design of our electronic objects. Our work expands the illumination design space by treating light as a physical material. We introduce a digital design tool that simulates and visualizes physical light interactions with a variety of materials for creating custom luminaires. We further develop a computational design and fabrication process for creating custom secondary optics elements (SOEs), which provides additional handles for users to physically shape and redirect light to compose, fill, and evenly diffuse planar and volumetric geometries. Through a workshop study with novice electronic designers, we show how incorporating physical techniques to shape light alters how users view the role and function of LEDs and electronics. We produce example pieces that showcase how our approach expands the electronics aesthetic and discuss how viewing light as material can engender novel, expressive artifacts.