Creativity support tools have begun to incorporate GenAI for exploring ideas. However, our preliminary study with nine designers showed that current GenAI tools lack explicit support for iteratively evolving, reflecting upon and tracking design alternatives. We developed DesignTrace, an early-stage GenAI design tool that allows designers to experiment with semantically relevant visual variations in an interactive design space. Its representation captures the progression of designers’ visual and semantic ideas through command histories, state tracking, and an interactive branching structure. A study of twelve professional designers shows that DesignTrace’s palette helps express, explore, and reflect on design intentions. Its interactive branching structure helps them maintain visual consistency across design iterations; remember and revisit earlier design decisions; and see connections across ideas. Our work shows how re-envisioning GenAI-based interfaces around explicit design traces enable designers to benefit from generative capabilities while maintaining control as they explore design variants.

Current hospital medication management systems generate fragmented clinical workflows, forcing clinicians to improvise repairs by creating external artifacts. We argue that instead of formalizing their workarounds into new electronic health record (EHR) features, future systems should explicitly support improvisation capabilities that help clinicians better handle unpredictable breakdowns. In a field study (144 hours) with 11 clinicians, we observed how resource constraints require local optimizations — “expert hacks” by clinicians — at the expense of global consistency. Subsequent group interviews with 20 clinicians highlighted three distinct issues: one clinician’s prescribing shortcut often becomes another clinician’s roadblock; medication notes fail to distinguish between norms and deviations; and clinicians often reify personal strategies to translate medication data across systems. We argue that simply adding new electronic features will not suffice. Instead, we propose a novel design approach that focuses on helping clinicians create personal tools that let them successfully manage information breakdowns in their particular context.

International audience

International audience

International audience

Although HCI researchers often generate and compare new design concepts, they lack an established method for rigorously conducting qualitative assessments . We define and characterize Comparative Structured Observation as a qualitative research method that takes advantage of the structure of controlled experiments to generate comparable, ecologically relevant experiences with two or more design variants, often implemented as medium-fidelity prototypes. Researchers observe users and ask them to compare and reflect on each variant. We identify criteria for creating a successful Comparative Structured Observation study and illustrate variations of the method by analyzing four published studies. We also examine six additional studies (three “near” and three “far”) to clarify the boundary between what should and should not be considered a Comparative Structured Observation . We discuss the benefits and limitations of the method and argue that gathering comparative reflections about design variants can help researchers assess and advance their design concepts.

Visually oriented designers often struggle to create effective generative AI (GenAI) prompts. A preliminary study identified specific issues in composing and fine-tuning prompts, as well as needs in accurately translating intentions into rich input. We developed DesignPrompt, a moodboard tool that lets designers combine multiple modalities — images, color, text — into a single GenAI prompt and tweak the results. We ran a comparative structured observation study with 12 professional designers to better understand their intent expression, expectation alignment and transparency perception using DesignPrompt and text input GenAI. We found that multimodal prompt input encouraged designers to explore and express themselves more effectively. Designer’s interaction preferences change according to their overall sense of control over the GenAI and whether they are seeking inspiration or a specific image. Designers developed innovative uses of DesignPrompt, including developing elaborate multimodal prompts and creating a multimodal prompt pattern to maximize novelty while ensuring consistency.

This paper examines prevailing understandings of creativity in creative computing research through the lens of feminist epistemology. We analyze “creativity support” as a construct that encodes different definitions of creative work. Drawing on existing literature and practices, the paper surfaces four views about creative work that underpin current creative technologies and HCI research: problem-solving, cognitive emergence, embodied action, and tool-mediated expert activity. Each view makes different claims about the role of computing in creative work and the creative subject assumed. We articulate the attendant politics of each view and illustrate how critical feminist epistemology can serve as an analytical tool to reason about the trade-offs of various creativity definitions. The paper concludes with recommendations for integrating feminist values into creativity-oriented HCI research.

Although video is extremely useful for expressing interaction, post-hoc editing makes it impractical for rapid prototyping. We present an early-stage video-based design method — “editing-in-the-camera” — where title cards guide video capture and label video clips. This method lets designers easily create video prototypes that can be discussed within the same design session, without further editing. We present VideoClipper, a mobile app that incorporates this method by transforming sequences of title cards into an interactive storyboard that designers can shoot into directly. VideoClipper offers simple special effects to better illustrate user interaction with paper prototypes, including ghosting and stop-motion animation. We also present Collaborative VideoClipper, which was created during the COVID-19 pandemic to support multi-user, multi-device rapid prototyping with remote participants. We describe evaluation results of both applications and describe our experiences in diverse educational and professional settings, including brainstorming, interviewing, video prototyping, user studies and participatory design workshops.

How can we let users adapt video-based meetings as easily as they rearrange furniture in a physical meeting room? We describe a design space for video conferencing systems that includes a five-step “ladder of tailorability,” from minor adjustments to live reprogramming of the interface. We then present Mirrorverse and show how it applies the principles of computational media to support live tailoring of video conferencing interfaces to accommodate highly diverse meeting situations. We present multiple use scenarios, including a virtual workshop, an online yoga class, and a stand-up team meeting to evaluate the approach and demonstrate its potential for new, remote meetings with fluid transitions across activities.

Game masters (GMs) are creative practitioners who plan and orchestrate tabletop role-playing games. Through an interview study, we investigate how eight expert game masters adapt everyday technologies and materials as creativity support tools (CSTs) for improvisational and collaborative play. We integrate theories of improvisational and distributed creativity with the human-artifact model, which provides an activity-theoretical vocabulary for analyzing the mediating relationships between specialist practitioners and their tools. We show how GMs prepare and deploy readymade artifacts: analog and digital CSTs that flexibly mediate recurring creative tasks in their practice, such as improvising narrative elements, facilitating smooth play, and creating aesthetic effects. We find that GMs demonstrate designerly thinking as they create, share, and refine repertoires of readymade artifacts. We argue that our theoretical approach can inform future studies of IT-mediated creativity, and that readymade artifacts can be an analytical and generative concept for the design of novel creativity support tools.

A key aspect of knowledge work is the analysis and manipulation of sets of related documents. We conducted interviews with 12 patent examiners and 12 scientists and found that all face difficulties using specialized tools for managing text from multiple documents across interconnected activities, including searching, collecting, annotating, organizing, writing and reviewing, while manually tracking their provenance. We introduce Passages, interactive objects that reify text selections and can then be manipulated, reused, and shared across multiple tools. Passages directly supports the above-listed activities as well as fluid transitions among them. Two user studies show that participants found Passages both elegant and powerful, facilitating their work practices and enabling greater reuse and novel strategies for analyzing and composing documents. We argue that Passages offers a general approach applicable to a wide variety of text-based interactions.

Most current dance support technologies focus on dancers, teachers or choreographers who are engaged in a single activity. We are interested in creating tools that support professional dancers over longer periods of time, as their careers and personal practices evolve. We interviewed 12 professional and pre-professional dancers about a critical moment in their careers: the transition to a new dance style due to shifting interests, ageing or injury. We identify three key challenges—overcoming habits, learning new forms of movement, transitioning over time—and their strategies for addressing them. We argue that successful tools must help dancers change their mentality about new movement styles, rather than focusing solely on movement mechanics. We suggest three possible implications for design: develop “movement substrates” that handle multiple movement representations; integrate learning and reflection in a single session; and create movement definitions through movement. We conclude with a discussion of directions for future research.

Machine Learning models can output confident but incorrect predictions. To address this problem, ML researchers use various techniques to reliably estimate ML uncertainty, usually performed on controlled benchmarks once the model has been trained. We explore how the two types of uncertainty—aleatoric and epistemic—can help non-expert users understand the strengths and weaknesses of a classifier in an interactive setting. We are interested in users’ perception of the difference between aleatoric and epistemic uncertainty and their use to teach and understand the classifier. We conducted an experiment where non-experts train a classifier to recognize card images, and are tested on their ability to predict classifier outcomes. Participants who used either larger or more varied training sets significantly improved their understanding of uncertainty, both epistemic or aleatoric. However, participants who relied on the uncertainty measure to guide their choice of training data did not significantly improve classifier training, nor were they better able to guess the classifier outcome. We identified three specific situations where participants successfully identified the difference between aleatoric and epistemic uncertainty: placing a card in the exact same position as a training card; placing different cards next to each other; and placing a non-card, such as their hand, next to or on top of a card. We discuss our methodology for estimating uncertainty for Interactive Machine Learning systems and question the need for two-level uncertainty in Machine Teaching.

With the increasing interest in movement sonification and expressive gesture-based interaction, it is important to understand which factors contribute to movement learning and how. We explore the effects of movement sonification and users’ musical background on motor variability in complex gesture learning. We contribute an empirical study in which musicians and non-musicians learn two gesture sequences over three days, with and without movement sonification. Results show the interlaced interaction effects of these factors and how they unfold in the three-day learning process. For gesture 1, which is fast and dynamic with a direct “action-sound” sonification, movement sonification induces higher variability for both musicians and non-musicians on day 1. While musicians reduce this variability to a similar level as no auditory feedback condition on day 2 and day 3, non-musicians remain to have significantly higher variability. Across three days, musicians also have significantly lower variability than non-musicians. For gesture 2, which is slow and smooth with an “action-music” metaphor, there are virtually no effects. Based on these findings, we recommend future studies to take into account participants’ musical background, consider longitudinal study to examine these effects on complex gestures, and use awareness when interpreting the results given a specific design of gesture and sound.

Dance making is often a highly idiosyncratic, collaborative endeavour between a choreographer and a group of dancers that constitutes a rich context for designers of creativity-support tools (CSTs). However, long-term, ecologically valid studies of collaboration in dance making are rare, especially when mediated by digital tools. We present a 5-month field study in the frame of a dance course, where a choreographer and six students used a CST originally designed for choreographic writing. We contrast our findings with our initial assumptions about the role of the tool to mediate a diversity of notating styles and hierarchical roles. We highlight the value of and the challenges behind this in-the-wild study in uncovering needs and roles as they emerged over time.

Our goal is to help creative writers make procrastination productive. Interviews with eight creative writers highlighted two key practices: Over Criticizing, where perfectionism and negative self-appraisal demotivates them and reduces their output; and Creative Voicing, where speaking their text aloud promotes reflection and inspires new possibilities. A structured observation study compared writers’ perception of their own, pre-recorded text versus computer-generated voices. The latter distances them from their text and offers new perspectives. We developed SonAmi, an interactive coaster that voices selected dialog whenever the author lifts their mug. Two creative writers said SonAmi made them feel they were listening to “someone else’s text” or “a podcast”, which helped them identify and improve writing issues. We show how tangible creativity support tools can build upon authors’ existing strategies i.e. voicing their own words, and take advantage of naturally occurring events e.g., taking a sip of coffee, to support productive procrastination without interfering with creative flow.

Aerial hoops are circular, hanging devices for both acrobatic exercise and artistic performance that let us explore the role of interactive sonification in physical activity. We present SonicHoop, an augmented aerial hoop that generates auditory feedback via capacitive touch sensing, thus becoming a digital musical instrument that performers can play with their bodies. We compare three sonification strategies through a structured observation study with two professional aerial hoop performers. Results show that SonicHoop fundamentally changes their perception and choreographic processes: instead of translating music into movement, they search for bodily expressions that compose music. Different sound designs affect their movement differently, and auditory feedback, regardless of type of sound, improves movement quality. We discuss opportunities for using SonicHoop as an aerial hoop training tool, as a digital musical instrument, and as a creative object; as well as using interactive sonification in other acrobatic practices to explore full-body vertical interaction.

Co-customizations are collaborative customizations in messaging apps that all conversation members can view and change, e.g. the color of chat bubbles on Facebook Messenger. Co-customizations grant new opportunities for expressing intimacy; however, most apps offer private customizations only. To investigate how people in close relationships integrate co-customizations into their established communication app ecosystems, we built DearBoard: an Android keyboard that allows two people to co-customize its color theme and a toolbar of expression shortcuts (emojis and GIFs). In a 5-week field study with 18 pairs of couples, friends, and relatives, participants expressed their shared interests, history, and knowledge of each other through co-customizations that served as meaningful decorations, interface optimizations, conversation themes, and non-verbal channels for playful, affectionate interactions. The co-ownership of the co-customizations invited participants to negotiate who customizes what and for whom they customize. We discuss how co-customizations mediate intimacy through place-making efforts and suggest design opportunities.

Players in Massively Multiplayer Online Role-Playing Games (MMORPGs) generate long-standing histories with their characters, but cannot express or see traces of their adventures in the game worlds. We are interested in designing game systems where players shape and contribute their own narratives as game content. Study one designed and playtested three Virtual Tabletop Role-Playing Game (VTTRPG) prototypes where we found that structured, graphical representations of players' traces support co-design and narrative analysis. We also identified four categories of traces: environment, build, memory and object. We introduce Play Traces, a novel analysis method for representing and co-designing with players and their narratives. A structured observation including Play Traces studied 17 players over 16 four-hour sessions in the third VTTRPG prototype. We found that players successfully (and enjoyably) co-designed novel narratives. We identified three themes for how traces can affect and support players in shaping new interactive narratives. We present four design implications describing how player-created narratives in MMORPGs should first Reveal & Pull Attention from other players, Invite & Push further exploration, Guide & Assist toward endings, and optionally Show & Hide traces. Finally, we discuss how treating players as co-designers offers a promising approach for developing the next generation of MMORPGs.

Preparing a new dance performance involves more than learning individual steps. We are interested in understanding how dancers collaborate as they rehearse a new dance piece, with a particular emphasis on how they use physical and digital artifacts to support this process. We conducted a 12-month longitudinal observational study with a dance company that re-staged a dance piece, taken from the contemporary repertoire and unknown to the dancers. The study focused on the role that artifacts play in shaping the learning of a dance piece. We showed how dancers produced an ecology of artifacts with the aim of analyzing the choreographic ideas behind the dance and sharing them with other learners. We showed that sharing these artifacts was challenging because they are idiosyncratic and embody their creator perspective and vocabulary. We then illustrated how dancers overcome this challenge by compiling artifacts and distributing the learning task among the group in order to create a common knowledge of the piece which improves the learning process. We conclude with design opportunities for technologies supporting long-term dance learning processes.

Machine learning systems became pervasive in modern interactive technology but provide users with little, if any, agency with respect to how their models are trained from data. In this paper, we are interested in the way novices handle learning algorithms, what they understand from their behavior and what strategy they may use to "make it work". We developed a web-based sketch recognition algorithm based on Deep Neural Network (DNN), called Marcelle-Sketch, that end-users can train incrementally. We present an experimental study that investigate people's strategies and (mis)understandings in a realistic algorithm-teaching task. Our study involved 12 participants who performed individual teaching sessions using a think-aloud protocol. Our results show that participants adopted heterogeneous strategies in which variability affected the model performances. We highlighted the importance of sketch sequencing, particularly at the early stage of the teaching task. We also found that users' understanding is facilitated by simple operations on drawings, while confusions are caused by certain inherent properties of DNN. From these findings, we propose implications for design of IML systems dedicated to novices and discuss the socio-cultural aspect of this research.

Although Human–Computer Interaction research has developed various theories and frameworks for analyzing new and existing interactive systems, few address thegenerationof novel technological solutions, and new technologies often lack theoretical foundations. We introduceGenerative Theories of Interaction, which draw insights from empirical theories about human behavior in order to define specific concepts and actionable principles, which, in turn, serve as guidelines for analyzing, critiquing, and constructing new technological artifacts. After introducing and defining Generative Theories of Interaction, we present three detailed examples from our own work: Instrumental Interaction, Human–Computer Partnerships, and Communities & Common Objects. Each example describes the underlying scientific theory and how we derived and applied HCI-relevant concepts and principles to the design of innovative interactive technologies. Summary tables offer sample questions that help analyze existing technology with respect to a specific theory, critique both positive and negative aspects, and inspire new ideas for constructing novel interactive systems.

A key challenge HCl researchers face when designing a controlled experiment is choosing the appropriate number of participants, or sample size. A priori power analysis examines the relationships among multiple parameters, including the complexity associated with human participants, e.g., order and fatigue effects, to calculate the statistical power of a given experiment design. We created Argus, a tool that supports interactive exploration of statistical power: Researchers specify experiment design scenarios with varying confounds and effect sizes. Argus then simulates data and visualizes statistical power across these scenarios, which lets researchers interactively weigh various trade-offs and make informed decisions about sample size. We describe the design and implementation of Argus, a usage scenario designing a visualization experiment, and a think-aloud study.

Designers create inspirational mood boards to express their design ideas visually, through collages of images and text. They find appropriate images and reflect on them as they explore emergent design concepts. After presenting the results of a participatory design workshop and a survey of professional designers, we introduce SemanticCollage, a digital mood board tool that attaches semantic labels to images by applying a state-of-the-art semantic labeling algorithm. SemanticCollage helps designers to 1) translate vague, visual ideas into search terms; 2) make better sense of and communicate their designs; while 3) not disrupting their creative flow. A structured observation with 12 professional designers demonstrated how semantic labels help designers successfully guide image search and find relevant words that articulate their abstract, visual ideas. We conclude by discussing how SemanticCollage inspires new uses of semantic labels for supporting creative practice.

Writing technical documents frequently requires following constraints and consistently using domain-specific terms. We interviewed 12 legal professionals and found that they all use a standard word processor, but must rely on their memory to manage dependencies and maintain consistent vocabulary within their documents. We introduce Textlets, interactive objects that reify text selections into persistent items. We show how Textlets help manage consistency and constraints within the document, including selective search and replace, word count, and alternative wording. Eight participants tested a search-and-replace Textlet as a technology probe. All successfully interacted directly with the Textlet to perform advanced tasks; and most (6/8) spontaneously generated a novel replace-all-then-correct strategy. Participants suggested additional ideas, such as supporting collaborative editing over time by embedding a Textlet into the document to flag forbidden words. We argue that Textlets serve as a generative concept for creating powerful new tools for document editing.

Players in modern Massively Multiplayer Online Role-Playing Games progress through ambitiously designed narratives, but have no real influence on the game, since only their characters’ data, not the game environment, persists. Although earlier games supported player influence by persisting changes in the world, they relied on players’ capacity to form their own stories and lacked guidance for character progression. We explore how persistence and narrative emergence let us build upon players’ influence rather than restrict it. We ran four studies and found that players highly value first-time and unique events, and often externalize their experiences to the Web to collaborate and socialize, but unintentionally also disrupt some aspects of in-game play. We introduce Narrative Substrates, a theoretical framework for designing game architectures that represent, manage, and persist traces of player activity as unique, interactive content. To illustrate and test the theory, we developed the game We Ride and deployed it as a two-phase technology probe over one year. We identify key benefits and challenges of our approach, and argue that reification of emergent narratives offers new design opportunities for creating truly interactive games.

This joint panel between UIST and CSCW brings together leading researchers at the intersection of the conferences-systems researchers in collaborative and social computing-to engage in a discussion and retrospective. Pairs of panelists will represent each decade since the founding of the conferences, sharing a brief retrospective that surveys the most influential papers of that decade, the zeitgeist of the problems that were popular that decade and why, and what each decade's work has to say to the decades that came before and after. The panel is intended as a space to celebrate advances in the field, and reflect on the burdens and opportunities that it faces ahead.

Professional designers create mood boards to explore, visualize, and communicate hard-to-express ideas. We present ImageCascade, an intelligent, collaborative ideation tool that combines individual and shared work spaces, as well as collaboration with multiple forms of intelligent agents. In the collection phase, ImageCascade offers fluid transitions between serendipitous discovery of curated images via ImageCascade, combined text- and image-based Semantic search, and intelligent AI suggestions for finding new images. For later composition and reflection, ImageCascade provides semantic labels, generated color palettes, and multiple tag clouds to help communicate the intent of the mood board. A study of nine professional designers revealed nuances in designers' preferences for designer-led, system-led, and mixed-initiative approaches that evolve throughout the design process. We discuss the challenges in creating effective human-computer partnerships for creative activities, and suggest directions for future research.

Our objective is to explore distributed forms of creativity that arise in play to help guide and foster supportive research, game design, and technology. This workshop seeks to bring together researchers, game designers, and others to examine theories of creativity and play, game design practices, methods for studying creativity in play, and creative play experiences. Participants will present work, video prototype, discuss topics, and contribute to outcomes.

Touchstone2 offers a direct-manipulation interface for generating and examining trade-offs in experiment designs. Based on interviews with experienced researchers, we developed an interactive environment for manipulating experiment design parameters, revealing patterns in trial tables, and estimating and comparing statistical power. We also developed TSL, a declarative language that precisely represents experiment designs. In two studies, experienced HCI researchers successfully used Touchstone2 to evaluate design trade-offs and calculate how many participants are required for particular effect sizes. We discuss Touchstone2's benefits and limitations, as well as directions for future research.

Couples exhibit special communication practices, but apps rarely offer couple-specific functionality. Research shows that sharing streams of contextual information (e.g. location, motion) helps couples coordinate and feel more connected. Most studies explored a single, ephemeral stream; we study how couples' communication changes when sharing multiple, persistent streams. We designed Lifelines, a mobile-app technology probe that visualizes up to six streams on a shared timeline: closeness to home, battery level, steps, media playing, texts and calls. A month-long study with nine couples showed that partners interpreted information mostly from individual streams, but also combined them for more nuanced interpretations. Persistent streams allowed missing data to become meaningful and provided new ways of understanding each other. Unexpected patterns from any stream can trigger calls and texts, whereas seeing expected data can replace direct communication, which may improve or disrupt established communication practices. We conclude with design implications for mediating awareness within couples.

Kinaesthetic creativity refers to the body's ability to generate alternate futures in activities such as role-playing in participatory design workshops. This has relevance not only to the design of methods for inspiring creativity but also to the design of systems that promote engaging experiences via bodily interaction. This paper probes this creative process by studying how dancers interact with technology to generate ideas. We developed a series of parameterized interactive visuals and asked dance practitioners to use them in generating movement materials. From our study, we define a taxonomy that comprises different relationships and movement responses dancers form with the visuals. Against this taxonomy, we describe six types of interaction patterns and demonstrate how dance creativity is driven by the ability to shift between these patterns. We then propose a set of interaction design qualities to support kinaesthetic creativity.

Our observations of landscape architecture students revealed a new phenomenon-interstices. Their bimanual interactions with a pen and touch surface involved various sustained hand gestures, interleaved between their regular commands. Positioning of the non-preferred hand indicates anticipated actions, including: sustained hovering near the surface; pulled back but still floating above the surface; and resting in their laps. We ran a second study with 14 landscape architect students which confirmed our observations, and uncovered a new interstice i.e. stabilizing the preferred hand while handwriting. We conclude with directions for future research and challenges for designers and researchers.

We present Videostrates, a concept and a toolkit for creating real-time collaborative video editing tools. Videostrates supports both live and recorded video composition with a declarative HTML-based notation, combining both simple and sophisticated editing tools that can be used collaboratively. Videostrates is programmable and unleashes the power of the modern web platform for video manipulation. We demonstrate its potential through three use scenarios: collaborative video editing with multiple tools and devices; orchestration of multiple live streams that are recorded and broadcast to a popular streaming platform; and programmatic creation of video using WebGL and shaders for blue screen effects. These scenarios only scratch the surface of Videostrates' potential, which opens up a design space for novel collaborative video editors with fully programmable interfaces.

The growing adoption of emojis, stickers and GIFs suggests a corresponding demand for rich, personalized expression in messaging apps. Some people customize apps to enable more personal forms of expression, yet we know little about how such customizations shape everyday communication. Since people increasingly communicate via multiple apps side-by-side, we are also interested in how customizing one app influences communication via other apps. We created a taxonomy of customization options based on interviews with 15 "extreme users" of communication apps. We found that participants tailored their apps to express their identities, organizational culture, and intimate bonds with others. They also experienced expression breakdowns: frustrations around barriers to transferring personal forms of expression across apps, which inspired inventive workarounds to maintain cross-app habits of expression, such as briefly switching apps to generate and export content for a particular conversation. We conclude with implications for personalized expression in ecosystems of communication apps.

The stereotype of creativity as an isolated, individual activity has given way to a more nuanced understanding of creativity as a social process. Our interview study of 23 contemporary music composers and choreographers focuses on the role that artifacts play in shaping creative collaborations with performers. We found that creators and performers form relationships where the creator acts as an author, a curator, a planner, or a researcher, and the performer acts as an interpreter, a creator, an improvisor, or an informant. Furthermore, we found that creators sculpt, layer, remix artifacts, moving fluidly across these different forms of interaction throughout the creative process. We conclude that the slippages that occur at the seams between roles and interactions drive creativity forward by opening up pathways into the future.

Our goal is to understand how dancers learn complex dance phrases. We ran three workshops where dancers learned dance fragments from videos. In workshop 1, we analyzed how dancers structure their learning strategies by decomposing movements. In workshop 2, we introduced MoveOn, a technology probe that lets dancers decompose video into short, repeatable clips to support their learning. This served as an effective analysis tool for identifying the changes in focus and understanding their decomposition and recomposition processes. In workshop 3, we compared the teacher's and dancers' decomposition strategies, and how dancers learn on their own compared to teacher-created decompositions. We found that they all ungroup and regroup dance fragments, but with different foci of attention, which suggests that teacher-imposed decomposition is more effective for introductory dance students, whereas personal decomposition is more suitable for expert dancers. We discuss the implications for designing technology to support analysis, learning and teaching of dance through movement decomposition.

Professional designers and developers often struggle when transitioning between the design and implementation of an interactive system. We conducted three studies that focused on the design of custom interactions to understand the mismatches between their processes, tools, and representations. We found that current practices induce unnecessary rework and cause discrepancies between design and implementation. We identified three recurring types of breakdowns: omitting critical details, ignoring edge cases, and disregarding technical limitations. We propose four design principles to create tools that mitigate these problems: Provide multiple viewpoints , maintain a single source of truth , reveal the invisible, and support design by enaction . We applied these principles to create E NACT , a live environment for prototyping touch-based interactions. We conducted two studies to assess E NACT and to compare designer–developer collaboration with E NACT versus current tools. Results suggest that E NACT helps participants detect more edge cases, increases designers’ participation and provides new opportunities for co-creation.

Contemporary choreographers often interact directly with dancers when exploring their ideas, but lack adequate tools for capturing and documenting their work. Although our first study of choreographers and dancers revealed diverse strategies for recording choreographic fragments, we found that they all worked in terms of constraints, which they represented via spatial diagrams, as movement qualities or with their own personal notation system. This led to the design of Knotation, a mobile pen-based tool that lets choreographers sketch their own representations of choreographic ideas and render them interactive. In study two, Knotation served as a technology probe to support the contrasting practices of three professional choreographers. We revised Knotation based on their input, and ran a third structured observation study with six professional choreographers. Knotation easily supported both dance-then-record and record-then-dance strategies. Participants used and appropriated Knotation's advanced features, including the combination of interactive timelines and floorplan diagrams, to represent and explore complex choreographic structures.

We introduce BIGFile, a new fast file retrieval technique based on the Bayesian Information Gain framework. BIGFile provides interface shortcuts to assist the user in navigating to a desired target (file or folder). BIGFile's split interface combines a traditional list view with an adaptive area that displays shortcuts to the set of file paths estimated by our computationally efficient algorithm. Users can navigate the list as usual, or select any part of the paths in the adaptive area. A pilot study of 15 users informed the design of BIGFile, revealing the size and structure of their file systems and their file retrieval practices. Our simulations show that BIGFile outperforms Fitchett et al.'s AccessRank, a best-of-breed prediction algorithm. We conducted an experiment to compare BIGFile with ARFile (AccessRank instantiated in a split interface) and with a Finder-like list view as baseline. BIGFile was by far the most efficient technique (up to 44% faster than ARFile and 64% faster than Finder), and participants unanimously preferred the split interfaces to the Finder.

Makers often create both physical and digital prototypes to explore a design, taking advantage of the subtle feel of physical materials and the precision and power of digital models. We introduce ShapeMe, a novel smart material that captures its own geometry as it is physically cut by an artist or designer. ShapeMe includes a software toolkit that lets its users generate customized, embeddable sensors that can accommodate various object shapes. As the designer works on a physical prototype, the toolkit streams the artist's physical changes to its digital counterpart in a 3D CAD environment. We use a rapid, inexpensive and simple-to-manufacture inkjet printing technique to create embedded sensors. We successfully created a linear predictive model of the sensors' lengths, and our empirical tests of ShapeMe show an average accuracy of 2 to 3 mm. We present two application scenarios for modeling multi-object constructions, such as architectural models, and 3D models consisting of multiple layers stacked one on top of each other. ShapeMe demonstrates a novel technique for integrating digital and physical modeling, and suggests new possibilities for creating shape-aware materials.

Remote collaboration across wall-sized displays creates a key challenge: how to support audio-video communication among users as they move in front of the display. We present CamRay, a platform that uses camera arrays embedded in wall-sized displays to capture video of users and present it on remote displays according to the users' positions. We investigate two settings: in Follow-Remote, the position of the video window follows the position of the remote user; in Follow-Local, the video window always appears in front of the local user. We report the results of a controlled experiment showing that with Follow-Remote, participants are faster, use more deictic instructions, interpret them more accurately, and use fewer words. However, some participants preferred the virtual face-to-face created by Follow-Local when checking for their partners' understanding. We conclude with design recommendations to support remote collaboration across wall-sized displays.

Although users accomplish ever more tasks on touch-enabled mobile devices, gesture-based interaction remains limited and almost never customizable by users. Our goal is to help users create gestures that are both personally memorable and reliably recognized by a touch-enabled mobile device. We address these competing requirements with two dynamic guides that use progressive feedforward to interactively visualize the "negative space" of unused gestures: the Pathward technique suggests four possible completions to the current gesture, and the Fieldward technique uses color gradients to reveal optimal directions for creating recognizable gestures. We ran a two-part experiment in which 27 participants each created 42 personal gesture shortcuts on a smartphone, using Pathward, Fieldward or No Feedforward. The Fieldward technique best supported the most common user strategy, i.e. to create a memorable gesture first and then adapt it to be recognized by the system. Users preferred the Fieldward technique to Pathward or No Feedforward, and remembered gestures more easily when using the technique. Dynamic guides can help developers design novel gesture vocabularies and support users as they design custom gestures for mobile applications.

Traditional graphic design tools emphasize the grid for structuring layout. Interviews with professional graphic designers revealed that they use surprisingly sophisticated structures that go beyond the grid, which we call graphical substrates. We present a framework to describe how designers establish graphical substrates based on properties extracted from concepts, content and context, and use them to compose layouts in both space and time. We developed two technology probes to explore how to embed graphical substrates into tools. Contextify lets designers tailor layouts according to each reader's intention and context; while Linkify lets designers create dynamic layouts based on relationships among content properties. We tested the probes with professional graphic designers, who all identified novel uses in their current projects. We incorporated their suggestions into StyleBlocks, a prototype that reifies CSS declarations into interactive graphical substrates. Graphical substrates offer an untapped design space for tools that can help graphic designers generate personal layout structures.

Today's users communicate via multiple apps, even when they offer almost identical functionality. We studied how and why users distribute their contacts within their app ecosystem. We found that the contacts in an app affect a user's conversations with other contacts, their communication patterns in the app, and the quality of their social relationships. Users appropriate the features and technical constraints of their apps to create idiosyncratic communication places, each with its own recursively defined membership rules, perceived purposes, and emotional connotations. Users also shift the boundaries of their communication places to accommodate changes in their contacts' behaviour, the dynamics of their relationships, and the restrictions of the technology. We argue that communication apps should support creating multiple communication places within the same app, relocating conversations across apps, and accessing functionality from other apps.

Users must manage frequent software and operating system upgrades across multiple computing devices. While current research focuses primarily on the security aspect, we investigate the user's perspective of upgrading software. Our first study (n=65) found that users delay major upgrades by an average of 80 days. We then ran a field study (n=14), beginning with in-depth observations during an operating system upgrade, followed by a four-week diary study. Very few participants prepared for upgrades (e.g., backing up files), and over half had negative reactions to the upgrade process and other changes (e.g., bugs, lost settings, unwanted features). During the upgrade process, waiting times were too long, feedback was confusing or misleading, and few had clear mental models of what was happening. Users almost never mentioned security as a concern or reason for upgrading. By contrast, interviews (n=3) with technical staff responsible for one organization's upgrades focused only on security and licensing, not user interface changes. We conclude with recommendations to improve the user's upgrade experience.

Professional interaction designers and software developers have different trainings and skills, yet they need to closely collaborate to create interactive systems. We conducted three studies to understand the mismatches between their processes, tools and representations. Based on 16 interviews, we found that current practices induce unnecessary rework and cause discrepancies between the original design and the implementation. We identified three key design breakdowns where designers omitted critical details, ignored the presence of edge cases or disregarded technical limitations. We next observed two face-to-face meetings between designers and developers. We found that early involvement of the developer helped to mitigate potential design breakdowns but new ones emerged as the project unfolded. Finally, we ran a participatory design session with two designer/developer pairs. Both pairs had difficulty representing and communicating pre-existing interactions. Creating complete interaction descriptions required iterating from individual examples to rule-based representations. We conclude with implications for designing collaborative tools that facilitate the designer's ability to express and the developer's ability to implement complex interactive systems.

CommandBoard offers a simple, efficient and incrementally learnable technique for issuing gesture commands from a soft keyboard. We transform the area above the keyboard into a command-gesture input space that lets users draw unique command gestures or type command names followed by execute. Novices who pause see an in-context dynamic guide, whereas experts simply draw. Our studies show that CommandBoard's inline gesture shortcuts are significantly faster (almost double) than markdown symbols and significantly preferred by users. We demonstrate additional techniques for more complex commands, and discuss trade-offs with respect to the user's knowledge and motor skills, as well as the size and structure of the command space.

We are interested in how to create digital tools to support informal sketching and prototyping of physical objects by novices. Achieving this goal first requires a deeper understanding of how non-professional designers generate, explore, and communicate design ideas with traditional tools, i.e., sketches on paper and hands-on prototyping materials. We describe a study framed around two all-day design charrettes where participants perform a complete design process: ideation sketching, concept development and presentation, fabrication planning documentation and collaborative fabrication of hand-crafted prototypes. This structure allows us to control key aspects of the design process while collecting rich data about creative tasks, including sketches on paper, physical models, and videos of collaboration discussions. Participants used a variety of drawing techniques to convey 3D concepts. They also extensively manipulated physical materials, such as paper, foam, and cardboard, both to support concept exploration and communication with design partners. Based on these observations, we propose design guidelines for CAD tools targeted at novice crafters.

Gesture-typing is an efficient, easy-to-learn, and errortolerant technique for entering text on software keyboards. Our goal is to "recycle" users' otherwise-unused gesture variation to create rich output under the users' control, without sacrificing accuracy. Experiment 1 reveals a high level of existing gesture variation, even for accurate text, and shows that users can consciously vary their gestures under different conditions. We designed an Expressive Keyboard for a smart phone which maps input gesture features identified in Experiment 1 to a continuous output parameter space, i.e. RGB color. Experiment 2 shows that users can consciously modify their gestures, while retaining accuracy, to generate specific colors as they gesture-type. Users are more successful when they focus on output characteristics (such as red) rather than input characteristics (such as curviness). We designed an app with a dynamic font engine that continuously interpolates between several typefaces, as well as controlling weight and random variation. Experiment 3 shows that, in the context of a more ecologically-valid conversation task, users enjoy generating multiple forms of rich output. We conclude with suggestions for how the Expressive Keyboard approach can enhance a wide variety of gesture recognition applications.

Aligning and distributing graphical objects is a common, but cumbersome task. In a preliminary study (six graphic designers, six non-designers), we identified three key problems with current tools: lack of persistence, unpredictability of results, and inability to 'tweak' the layout. We created StickyLines, a tool that treats guidelines as first-class objects: Users can create precise, predictable and persistent interactive alignment and distribution relationships, and 'tweaked' positions can be maintained for subsequent interactions. We ran a [2x2] within-participant experiment to compare StickyLines with standard commands, with two levels of layout difficulty. StickyLines performed 40% faster and required 49% fewer actions than traditional alignment and distribution commands for complex layouts. In study three, six professional designers quickly adopted StickyLines and identified novel uses, including creating complex compound guidelines and using them for both spatial and semantic grouping.

Recent advances in materials science research allow production of highly stretchable sensors and displays. Such technologies, however, are still not accessible to non-expert makers. We present a novel and inexpensive fabrication method for creating Stretchis, highly stretchable user interfaces that combine sensing capabilities and visual output. We use Polydimethylsiloxan (PDMS) as the base material for a Stretchi and show how to embed stretchable touch and proximity sensors and stretchable electroluminescent displays. Stretchis can be ultra-thin (≈ 200μm), flexible, and fully customizable, enabling non-expert makers to add interaction to elastic physical objects, shape-changing surfaces, fabrics, and the human body. We demonstrate the usefulness of our approach with three application examples that range from ubiquitous computing to wearables and on-skin interaction.

Creative processes involve a repertoire of digital devices ranging from mobile phones over tablets and desktop computers to electronic whiteboards and wall-sized displays. While some integration across multiple devices is supported more sophisticated kinds of integration that connect devices and amplify their potential are limited. Many creative practices also rely on physical materials and tools. This will workshop investigate how the combination of physical and digital artifacts can support creative work practices. In this context, we propose to examine: Individual and social creative activities; Creativity methods; Emergence and transformation of design ideas; Generative design materials; Design constraints.

Although ubiquitous, color pickers have remained largely unchanged for 25 years. Based on contextual interviews with artists and designers, we created the Color Portraits design space to characterize five key color manipulation activities: sampling and tweaking individual colors, manipulating color relationships, combining colors with other elements, revisiting previous color choices, and revealing a design process through color. We found similar color manipulation requirements with scientists and engineers. We designed novel color interaction tools inspired by the design space, and used them as probes to identify specific design requirements, including: interactive palettes for sampling colors and exploring relationships; color composites for blending and decomposing colors with other elements; interactive histories to enable reuse of previous color choices; and providing color as a way to reveal underlying processes. We argue that color tools should allow users to interact with colors, not just pick or sample them.

Makers participate in remixing culture by drawing inspiration from, combining, and adapting designs for physical objects. To examine how makers remix each others' designs on a community scale, we analyzed metadata from over 175,000 digital designs from Thingiverse, the largest online design community for digital fabrication. Remixed designs on Thingiverse are predominantly generated designs from Customizer a built-in web app for adjusting parametric designs. However, we find that these designs do not elicit subsequent user activity and the authors who generate them tend not to contribute additional content to Thingiverse. Outside of Customizer, influential sources of remixing include complex assemblies and design primitives, as well as non-physical resources posing as physical designs. Building on our findings, we discuss ways in which online maker communities could become more than just design repositories and better support collaborative remixing.

Interactive paper is a form of tangible computing that allows people to take full advantage of their existing cognitive and physical skills, and encourages us, as designers, to explore novel strategies for supporting creativity. This talk traces my own history with interactive paper, based on participatory design projects with creative professionals who successfully combine paper and computers. I begin with Pierre Wellner's ground-breaking Digital Desk in the 1990's and continue through to Phillippe LeRoux's Quid Sit Musicus in 2014, which used interactive paper both during composition and as an integral part of the live performance. The insights we gained from these projects has led to a new way of thinking about interaction design. Our concept of co-adaptive instruments is based on instrumental Interaction, which treats interaction as a first class object and co-adaptation, which helps users both learn (adapt to) and appropriate (adapt) interactive systems. By taking cues from how people interact in the physical world, we can create less brittle systems that are easier to learn and appropriate, and support a wide variety of creative activities.

We revisit Alan Kay's early vision of dynamic media that blurs the distinction between documents and applications. We introduce shareable dynamic media that are malleable by users, who may appropriate them in idiosyncratic ways; shareable among users, who collaborate on multiple aspects of the media; and distributable across diverse devices and platforms. We present Webstrates, an environment for exploring shareable dynamic media. Webstrates augment web technology with real-time sharing. They turn web pages into substrates, i.e. software entities that act as applications or documents depending upon use. We illustrate Webstrates with two implemented case studies: users collaboratively author an article with functionally and visually different editors that they can personalize and extend at run-time; and they orchestrate its presentation and audience participation with multiple devices. We demonstrate the simplicity and generative power of Webstrates with three additional prototypes and evaluate it from a systems perspective.

Contemporary music composition is a highly creative and disciplined activity that requires free expression of ideas and sophisticated computer programming. This paper presents a technique for structured observation of expert creative behavior, as well as Polyphony, a novel interface for systematically studying all phases of computer-aided composition. Polyphony is a unified user interface that integrates interactive paper and electronic user interfaces for composing music. It supports fluid transitions between informal sketches and formal computer-based representations. We asked 12 composers to use Polyphony to compose an electronic accompaniment to a 20-second instrumental composition by Anton Webern. All successfully created a complete, original composition in an hour and found the task challenging but fun. The resulting dozen comparable snapshots of the composition process reveal how composers both adapt and appropriate tools in their own way.

The advent of ultra-high resolution wall-size displays and their use for complex tasks require a more systematic analysis and deeper understanding of their advantages and drawbacks compared with desktop monitors. While previous work has mostly addressed search, visualization and sense-making tasks, we have designed an abstract classification task that involves explicit data manipulation. Based on our observations of real uses of a wall display, this task represents a large category of applications. We report on a controlled experiment that uses this task to compare physical navigation in front of a wall-size display with virtual navigation using pan-and-zoom on the desktop. Our main finding is a robust interaction effect between display type and task difficulty: while the desktop can be faster than the wall for simple tasks, the wall gains a sizable advantage as the task becomes more difficult. A follow-up study shows that other desktop techniques (overview+detail, lens) do not perform better than pan-and-zoom and are therefore slower than the wall for difficult tasks.

We review current technical and social barriers to COPPA compliance for popular online services aimed at children. We show that complying with COPPA has proven difficult for developers, even when a genuine attempt was made. We investigate reasons for this lack of compliance and identify common causes: specifically, difficulties obtaining verifiable parental control as well as supply mechanisms for parents to understand, review, grant access and monitor collection of their children's personal data. Unless part of online services, mobile apps do not need to comply with COPPA.

We introduce BodyScape, a body-centric design space that allows us to describe, classify and systematically compare multi-surface interaction techniques, both individually and in combination. BodyScape reflects the relationship between users and their environment, specifically how different body parts enhance or restrict movement within particular interaction techniques and can be used to analyze existing techniques or suggest new ones. We illustrate the use of BodyScape by comparing two free-hand techniques, on-body touch and mid-air pointing, first separately, then combined. We found that touching the torso is faster than touching the lower legs, since it affects the user's balance; and touching targets on the dominant arm is slower than targets on the torso because the user must compensate for the applied force.

Effectively planning a large multi-track conference requires an understanding of the preferences and constraints of organizers, authors, and attendees. Traditionally, the onus of scheduling the program falls on a few dedicated organizers. Resolving conflicts becomes difficult due to the size and complexity of the schedule and the lack of insight into community members' needs and desires. Cobi presents an alternative approach to conference scheduling that engages the entire community in the planning process. Cobi comprises (a) communitysourcing applications that collect preferences, constraints, and affinity data from community members, and (b) a visual scheduling interface that combines communitysourced data and constraint-solving to enable organizers to make informed improvements to the schedule. This paper describes Cobi's scheduling tool and reports on a live deployment for planning CHI 2013, where organizers considered input from 645 authors and resolved 168 scheduling conflicts. Results show the value of integrating community input with an intelligent user interface to solve complex planning tasks.

We present paper substrates, interactive paper components that support the creation and manipulation of complex musical data. Substrates take different forms, from whole pages to movable strips, and contain or control typed data representations. We conducted participatory design sessions with five professional musicians with extensive experience with music creation tools. All generated innovative uses of paper substrates, manipulating their data, linking multiple representation layers and creating modular, reusable paper elements. The substrates reflect the structure of their computer-based data, but in a much more flexible and adaptable form. We use their prototypes to provide concrete examples of substrates, identify their roles, properties and functions. Finally, we explore their physical and interaction design with an interactive prototype.

Augmented Reality (AR) has been proved useful to guide operational tasks in professional domains by reducing the shift of attention between instructions and physical objects. Modern smartphones make it possible to use such techniques in everyday tasks, but raise new challenges for the usability of AR in this context: small screen, occlusion, operation "through a lens". We address these problems by adding real-time feedback to the AR overlay. We conducted a controlled experiment comparing AR with and without feedback, and with standard textual and graphical instructions. Results show significant benefits for mobile AR with feedback and reveals some problems with the other techniques.

Despite the demonstrated benefits of bimanual interaction, most tablets use just one hand for interaction, to free the other for support. In a preliminary study, we identified five holds that permit simultaneous support and interaction, and noted that users frequently change position to combat fatigue. We then designed the BiTouch design space, which introduces a support function in the kinematic chain model for interacting with hand-held tablets, and developed BiPad, a toolkit for creating bimanual tablet interaction with the thumb or the fingers of the supporting hand. We ran a controlled experiment to explore how tablet orientation and hand position affect three novel techniques: bimanual taps, gestures and chords. Bimanual taps outperformed our one-handed control condition in both landscape and portrait orientations; bimanual chords and gestures in portrait mode only; and thumbs outperformed fingers, but were more tiring and less stable. Together, BiTouch and BiPad offer new opportunities for designing bimanual interaction on hand-held tablets.

Very-high-resolution wall-sized displays offer new opportunities for interacting with large data sets. While pointing on this type of display has been studied extensively, higher-level, more complex tasks such as pan-zoom navigation have received little attention. It thus remains unclear which techniques are best suited to perform multiscale navigation in these environments. Building upon empirical data gathered from studies of pan-and-zoom on desktop computers and studies of remote pointing, we identified three key factors for the design of mid-air pan-and-zoom techniques: uni- vs. bimanual interaction, linear vs. circular movements, and level of guidance to accomplish the gestures in mid-air. After an extensive phase of iterative design and pilot testing, we ran a controlled experiment aimed at better understanding the influence of these factors on task performance. Significant effects were obtained for all three factors: bimanual interaction, linear gestures and a high level of guidance resulted in significantly improved performance. Moreover, the interaction effects among some of the dimensions suggest possible combinations for more complex, real-world tasks.

This article introduces runtime toolkit overloading, a novel approach to help third-party developers modify the interaction and behavior of existing software applications without access to their underlying source code. We describe the abstractions provided by this approach as well as the mechanisms for implementing them in existing environments. We describe Scotty, a prototype implementation for Mac OS X Cocoa that enables developers to modify existing applications at runtime, and we demonstrate a collection of interaction and functional transformations on existing off-the-shelf applications. We show how Scotty helps a developer make sense of unfamiliar software, even without access to its source code. We further discuss what features of future environments would facilitate this kind of runtime software development.

We introduce the knotty gesture, a simple yet powerful technique for interacting with paper. Knots are tiny circles that can be added to any gesture. Users can leave subtle marks that permit both immediate interaction in the flow of writing and create rich opportunities for future interaction. We identify diverse applications of knotty gestures and explore alternative techniques for interacting with their traces. We conducted two experiments to evaluate the design and recognition heuristics and demonstrated that people can successfully execute knotty gestures, even without feedback. Knotty gestures provide users with a subtle, in-the-flow-of-writing technique for tagging information and subsequently interacting with the paper.

This paper explores the need for sustainable design with paper: how people really print and how we can take advantage of novel, reusable paper technology. We conducted two studies to investigate user's printing behavior. A key finding of the first study was that users often need an intermediate state between the electronic and physical forms of their documents. The second study examined users' predictions of which types of documents required this intermediate state. We formulate these findings into design guidelines that take into account: examination phase, transitions, cognitive and emotional reasons, and task- and event-relevant documents. Finally, we discuss how the different physical characteristics of reusable paper affect the user interface and could effectively support sustainable design.

Generative Walkthoughs support the redesign phase of an iterative design process, helping designers generate new design alternatives informed by social science principles. Designers first analyze their own scenarios or storyboards with respect to concrete examples drawn from five socio-technical principles: situated action, rhythms & routines, co-adaptive systems, peripheral awareness and distributed cognition. They then walk through the scenario and brainstorm new design alternatives that reflect the design principle in question. This combination of structured walkthroughs with focused brainstorming helps designers, particularly those with little social science background, to generate concrete, actionable ideas that reflect key findings from the social science literature. We taught Generative Walkthroughs in ten courses with over 220 students and found that technically-trained students not only learned these socio-technical principles, but were able to apply them in innovative ways in a variety of design settings.

Many couples are forced to live apart, for work, school or other reasons. This paper describes our study of 13 such couples and what they lack from existing communication technologies. We explored what they wanted to share (presence, mood, environment, daily events and activities), how they wanted to share (simple, lightweight, playful, pleasant interaction), and when they wanted to share ('empty moments' such as waiting, walking, taking a break, waking up, eating, and going to sleep). 'Empty moments' provide a compelling new opportunity for design, requiring subtlety and flexibility to enable participants to share connection without explicit messages. We designed MissU as a technology probe to study empty moments in situ. Similar to a private radio station, MissU shares music and background sounds. Field studies produced results relevant to social science, technology and design: couples with established routines were comforted; characteristics such as ambiguity and 'movable' technology (situated in the home yet portable) provide support. These insights suggest a design space for supporting the sharing of empty moments.

We focus on the creative use of paper in the music composition process, particularly the interaction between paper and end-user programming. When expressing musical ideas, composers draw in a precise way, not just sketch. Working in close collaboration with composers, we designed Musink to provide them with a smooth transition between paper drawings and OpenMusic, a flexible music composition tool. Musink's built-in recognizers handle common needs, such as scoping and annotation. Users can also define new gestures and associate them with their own or predefined software functions. Musink supports semi-structured, delayed interpretation and serves as a customizable gesture browser, giving composers significant freedom to create their own, individualized composition languages and to experiment with music, on-paper and on-line.

We report on our studies of the evolving work practices of biologists and the role paper and electronic lab notebooks play in supporting their individual and collaborative activity. We describe the participatory design and longitudinal field testing of Prism, a hybrid laboratory notebook that lets biologists capture, visualize and interact with cross-linked streams of physical and electronic data. We used Prism as a technology probe that users could adapt to integrate additional activity streams and share information from other biologists. Our key findings include the use of master notebooks, whether paper or electronic, which act as a reference point for handling and organizing the diverse strands of personal activity, and the importance of redundancy, which biologists use to make sense of their data. Prism provides a flexible, extensible tool that supports individual and collaborative reflection in creative work.

We describe OctoPocus, an example of a dynamic guide that combines on-screen feedforward and feedback to help users learn, execute and remember gesture sets. OctoPocus can be applied to a wide range of single-stroke gestures and recognition algorithms and helps users progress smoothly from novice to expert performance. We provide an analysis of the design space and describe the results of two experi-ments that show that OctoPocus is significantly faster and improves learning of arbitrary gestures, compared to con-ventional Help menus. It can also be adapted to a mark-based gesture set, significantly improving input time compared to a two-level, four-item Hierarchical Marking menu.

This paper describes the design and early testing of SketchCam, a camera that lets children take pictures through sketching. Our observations of children's use of digital cameras led to insights about how children appropriate the screen in the picture-taking process. Inspired by their use of these cameras, we designed SketchCam, which enables children to capture images directly by sketching on a touch screen. We describe the results of our observations of children; the design of SketchCam, and the results of a preliminary study of a working prototype with four children aged three to six.

Touchstone is an open-source experiment design platform designed to help establish a solid research foundation for HCI in the area of novel interaction techniques. Touchstone includes a design platform for exploring alternative designs of controlled laboratory experiments, a run platform for running subjects and a limited analysis platform for advice and access to on-line statistics packages. Designed for HCI researchers and their students, Touchstone facilitates the process of creating new experiments, as well as replicating and extending experiments in the research literature. We tested Touchstone by designing two controlled experiments. One illustrates how to create a new experiment from scratch. The other replicates and extends a previous study of multiscale pointing interaction techniques: OrthoZoom was fastest, followed by bi-manual Pan & Zoom; SDAZ and traditional Pan & Zoom were consistently slower.

PageLinker is a browser extension that allows to contextualise navigation by linking web pages together and to navigate through a network of related web pages without prior planning. The design is based on extensive interviews with biologists, which highlighted their difficulties finding previously visited web pages. They found current browser tools inadequate, resulting in poorly organised bookmarks and rarely used history lists. In a four-week controlled field experiment, PageLinker significantly reduced time, page loads and mouse clicks. By presenting links in context, PageLinker facilitates web page revisitation, is less prone to bookmark overload and is highly robust to change.

Panos Markopoulos studied computer science and specialized in human-computer interaction. His research concerns intra-family communication, privacy and interaction design for children. He is an Associate Professor in the Department of Industrial Design of the Eindhoven University of Technology. Boris de Ruyter is an experimental psychologist with an interest in applying user modeling and psychometric techniques in the domains of consumer electronics, personal healthcare and lighting. He is a Principal Scientist at Philips Research leading the Cognitive and Behavioural Insights capability group and the Interactive Healthcare Solutions research programme. Wendy Mackay was trained in experimental psychology and computer science with an interest in multi-media, mixed reality and multidisciplinary design. She is a Research Director at INRIA in France and heads the in|situ| research group at INRIA Futurs and the University of Paris-Sud.

This article presents the design of Telebeads, a conceptual exploration of mobile mnemonic artefacts. Developed together with five 10-14 year olds across two participatory design sessions, we address the problem of social network massification by allowing teenagers to link individuals or groups with wearable objects such as handmade jewelery. We propose different concepts and scenarios using mixed-reality mobile interactions to augment crafted artefacts and describe a working prototype of a bluetooth luminous ring. We also discuss what such communication appliances may offer in the future with respect to interperception, experience networks and creativity analysis.

Communication appliances, intended for home settings, require intuitive forms of interaction. Computer vision offers a potential solution, but is not yet sufficiently accurate.As interaction designers, we need to know more than the absolute accuracy of such techniques: we must also be able to compare how they will work in our design settings, especially if we allow users to collaborate in the interpretation of their actions. We conducted a 2x4 within-subjects experiment to compare two interaction techniques based on computer vision: motion sensing, with EyeToy®-like feedback, and object tracking. Both techniques were 100% accurate with 2 or 5 choices. With 21 choices, object-tracking had significantly fewer errors and took less time for an accurate selection. Participants' subjective preferences were divided equally between the two techniques. This study compares these techniques as they would be used in real-world applications, with integrated user feedback, allowing interface designers to choose the one that best suits the specific user requirements for their particular application.

This paper describes the design and early evaluation of the Tangicam, or tangible camera, a mobile device for children to capture and edit realistic pictures and videos. Our first experimental results show that the affordances of the Tangicam allow imitation learning and free playing in a context of tangible and augmented reality. Our goal is to create a simple and robust observation system that lets children produce narratives based on situated [51] video, audio and sensor data. We also want to explore how these temporal structures may allow children to describe themselves, other children or natural phenomena and how such situated time series may help develop new forms of synaesthetic and intersubjective constructions.

The Interactive Thread is a design method that helps us gather detailed, contextualised data from a large user population while sharing interaction design methods with professional designers from different disciplines. We developed a set of 10-15 minute exercises drawn from design, social and computer science and presented them as a series woven throughout two interaction design conferences. Our goals were to provide an entertaining, interactive conference activity, to teach and share multi-disciplinary design methods, and to gather information that would otherwise be too labour-intensive for us as designers. This paper reflects on our experiences, including what worked and what did not. We discuss how others may reuse this strategy in other settings, including workshops, conferences and corporate retreats and we include an appendix with the specific interactive thread exercises.

We describe a new method for use in the process of co-designing technologies with users called technology probes. Technology probes are simple, flexible, adaptable technologies with three interdisciplinary goals: the social science goal of understanding the needs and desires of users in a real-world setting, the engineering goal of field-testing the technology, and the design goal of inspiring users and researchers to think about new technologies. We present the results of designing and deploying two technology probes, the messageProbe and the videoProbe, with diverse families in France, Sweden, and the U.S. We conclude with our plans for creating new technologies for and with families based on our experiences.

We conducted an experiment that compared three post-WIMP interaction techniques: floating palettes, marking menus and toolglasses, in a real-world Coloured Petri-Net editor, CPN2000. We created six situations in which users performed identical sets of actions with equally-complex nets, but with different cognitive contexts. We found significant differences in performance and preferences across interaction techniques. When a user is in a "copy" context, floating palettes are more efficient. If the user is problem solving, toolglasses or marking menus are preferred. No single interaction technique is clearly superior: each has strengths in different contexts. Since a single application must support different kinds of cognitive tasks, interaction designers should consider integrating multiple interaction techniques, rather than selecting only one.

Using a participatory design process, we created three prototype augmented laboratory notebooks that provide the missing link between paper, physical artifacts and on-line data. The final a-book combines a graphics tablet and a PDA. The tablet captures writing on the paper notebook and the PDA acts as an "interaction lens" or window between physical and electronic documents. Our approach is document-centered, with a software architecture based on layers of physical and electronic information.

This paper presents three design principles to support the development of large-scale applications and take advantage of recent research in new interaction techniques: Reification turns concepts into first class objects, polymorphism permits commands to be applied to objects of different types, and reuse makes both user input and system output accessible for later use. We show that the power of these principles lies in their combination. Reification creates new objects that can be acted upon by a small set of polymorphic commands, creating more opportunities for reuse. The result is a simpler yet more powerful interface.

Air traffic control is a complex, safety-critical activity, with well-established and successful work practices. Yet many attempts to automate the existing system have failed because controllers remain attached to a key work artifact: the paper flight strip. This article describes a four-month intensive study of a team of Paris en-route controllers in order to understand their use of paper flight strips. The article also describes a comparison study of eight different control rooms in France and the Netherlands. Our observations have convinced us that we do not know enough to simply get rid of paper strips, nor can we easily replace the physical interaction between controllers and paper strips.These observationshighlight the benefits of strips, including qualities difficult to quantify and replicate in new computer systems. Current thinking offers two basic alternatives: maintaining the existing strips without computer support and bearing the financial cost of limiting the air traffic, or replacing the strips with automated versions, which offer potential benefits in terms of increased efficiency through automation, but unknown risks through radical change of work practices. We conclude with a suggestion for a third alternative: to maintain the physical strips, but turn them into the interface to the computer. This would allow controllers to build directly upon their existing, safe work practices with paper strips, while offering them a gradual path for incorporating new computer-based functions. Augmented paper flight strips allow us to take advantage of uniquely human skills in the physical world, and allows us to leave the user interface and its subsequent evolution in the hands of the people most responsible, the air traffic controllers themselves.

A revolution in computer interface design is changing the way we think about computers. Rather than typing on a keyboard and watching a television monitor, Augmented Reality lets people use familiar, everyday objects in ordinary ways. The difference is that these objects also provide a link into a computer network. Doctors can examine patients while viewing superimposed medical images; children can program their own LEGO constructions; construction engineers can use ordinary paper engineering drawings to communicate with distant colleagues. Rather than immersing people in an artificially-created virtual world, the goal is to augment objects in the physical world by enhancing them with a wealth of digital information and communication capabilities.

DIVA supports exploratory data analysis of multimedia streams, enabling users to visualize, explore and evaluate patterns in data that change over time. The underlying stream algebra provides the mathematical basis for operating on diverse kinds of streams. The streamer visualization technique provides a smooth transition between spatial and temporal views of the data. Mapping source and presentation streams into a two-dimensional space provides users with a direct manipulation, nontemporal interface for viewing and editing streams. DIVA was developed to help us analyze both qualitative and quantitative data collected in our research with French air traffic controllers, including video of controllers at work, audio records of telephone, radio and other conversations, output from tools such as RADAR, and coded logs based on our observations. Although our emphasis is on exploratory data analysis, DIVA's stream architecture should prove useful for a wide variety of multimedia applications.

This paper describes our exploration of a design space for an augmented reality prototype. We began by observing air traffic controllers and their interactions with paper flight strips. We then worked with a multi-disciplinary team of researchers and controllers over a period of a year to brainstorm and prototype ideas for enhancing paper flight strips, We argue that augmented reality is more promising (and simpler to implement) than the current strategies that seek to replace flight strips with keyboard/monitor interfaces. We also argue that an exploration of the design space, with active participation from the controllers, is essential not only for designing particular artifacts, but also for understanding the strengths and limitations of augmented reality in general.

Videotape has become one of the CHI community's most useful technologies: it allows us to analyze users' interactions with computers, prototype new interfaces, and present the results of our research and technical innovations to others.But video is a double-edged sword.It is often misused, however unintentionally.How can we use it well, without compromising our integrity?This paper presents actual examples of questionable videotaping practices.Next, it explains why we cannot simply borrow ethical guidelines from other professions.It concludes with a proposal for developing usable ethical guidelines for the capture, analysis and presentation of video.

Paper video storyboards are still in use by even very experienced video producers with access to the most advanced video editing software. An analysis of the characteristics of paper and on-line editing provide an overlapping but distinct set of benefits (and problems). Paper provides the user with the ability to lay out various temporal sequences over a large spatial area and the ability to quickly sketch, annotate and rearrange the relevant video clips. On-line editing provides users with the ability to generate and store a variety of video arrangements. Video Mosaic provides users with the ability to combine the best of both worlds: elements of a paper video storyboard are used as input to an on-line video editing system to take advantage of the best aspects of each. We developed a Unix and a Macintosh version of Video Mosaic. This paper describes the design of Video Mosaic, compares alternative approaches to creating this type of application, and suggests directions for future work.

One of the properties of a user interface is that it both guides and constrains the patterns of interaction between the user and the software application.Application software is increasingly designed to be "customizable" by the end user, providing specific mechanisms by which users may specify individual preferences about the software and how they will interact with it over multiple sessions.Users may thus encode and preserve their preferred patterns of use.These customizations, together with choices about which applications to use, make up the unique "softswue environment" for each individual.While it is theoretically possible for each user to carefully evaluate and optimize each possible customization option, this study suggests that most people do not.In facfi since time spent customizing is time spent not working, many people do not take advantage of the customization features at all.I studied the customization behaviorof51 users of a Unix software environment, over a period of four months.This paper describes the process by which users decide to customize and examines the factors that irdluence when and how users make those decisions.These findings have implications for both the design of software and the integration of new software into an organization.

Vi&o is a powerful medium: it can make a point or convince people in ways that other media cannot.Somehow, video seems "real".Yet, perhaps it is too powerful.Just as statements taken out of context can be very damaging, so can video clips misconstrue events or violate the privacy of the subjects involved.As researchers and designers increasingly use video to obtaict information about how people interact with technology, it becomes important to examine the ethical issues involved in the creation and presentation of video.Should there be constraints on recordirtg information?How do we obtain "informed consent"?Under what conditions should video be presente4 and to which audiences?This session provides a forum for discussion: the goal is to identify ethical issues in the use of video, learn horn invited guests about existing practice in other fields and learn horn audience members about the issues that have arisen in their own work.After the discussion, the members of the audience will attempt to determine whether or not the human-computer interaction community needs to develop its own set of guidelines for the ethical use of video.

The act of customizing software is generally viewed as a solitary activity that allows users to express individual preferences. In this study, users at two different research sites, working with two different kinds of customizable software, were found to actively share their customization files with each other. This sharing allowed the members of each organization to establish and perpetuate informally-defined norms of behavior.

The second conference on Computer Supported Cooperative Work has provided focus on use of computers for supporting workers that are at various levels of geographic dispersion. The participants in this panel reported case studies at that conference on group work (1) in face-to-face meetings, (2) in the same building, and (3) distributed across a number of sites. Each panelist therefore brings insight about the communication needs of their research subjects and both the value and limitations of particular technologies for supporting the communication that ties the members of the groups together as geographic distance varies.

The Information Lens provides electronic mail users with the ability to write rules that automatically sort, select, and filter their messages. This paper describes preliminary results from an eighteen-month investigation of the use of this system at a corporate test site. We report the experiences of 13 voluntary users who have each had at least three months experience with the most recent version of the system. We found that:

Article Free Access Share on Video: Data for studying human-computer interaction Authors: W. E. Mackay Massachusetts Istitute of Technology Massachusetts Istitute of TechnologyView Profile , R. Guindon Microelectronics and Computer Technology Corporation Microelectronics and Computer Technology CorporationView Profile , M. M. Mantel Electric Data Systems Corporation Electric Data Systems CorporationView Profile , L. Suchman Xerox Palo Alto Research Center Xerox Palo Alto Research CenterView Profile , D. G. Tatar Xerox Palo Alto Research Center Xerox Palo Alto Research CenterView Profile Authors Info & Claims CHI '88: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsMay 1988 Pages 133–137https://doi.org/10.1145/57167.57189Published:01 May 1988Publication History 15citation544DownloadsMetricsTotal Citations15Total Downloads544Last 12 Months56Last 6 weeks6 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

This paper describes a series of interviews that focus on the ways that professional office workers use electronic mail to manage their daily work. A number of implications for the design of flexible mail systems are discussed.

This paper describes a series of interviews that examine the ways that professional office workers use electronic mail to manage their daily work. The purpose is to generate hypotheses for future research. A number of implications for the design of flexible mail systems are discussed. Two principal claims are made. First, the use of electronic mail is strikingly diverse, although not infinitely so. Individuals vary both in objective measures of mail use and in preferred strategies for managing work electronically. Feelings of control are similarly diverse and are related to the size of the user's inbox, numbers of folders, and subscriptions to distribution lists. This diversity implies that one's own experiences with electronic mail are unlikely to provide sufficient understanding of other's uses of mail. Mail designers should thus seek flexible primitives that capture the important dimensions of use and provide flexibility for a wide range of users. The second claim is that electronic mail is more than just a communication system. Users archive messages for subject retrieval, prioritize messages to sequence work activities, and delegate tasks via mail. A taxonomy of work management is proposed in which mail is used for information management, time management, and task management activities. Directions for future research are suggested.