Advances in emerging technologies, such as on-body mechanical actuators and electrical muscle stimulation, have allowed computers to take control over our bodies. This presents opportunities as well as challenges, raising fundamental questions about agency and the role of our bodies when interacting with technology. To advance this research field as a whole, we brought together expert perspectives in a week-long seminar to articulate the grand challenges that should be tackled when it comes to the design of computers’ control over our bodies. These grand challenges span technical, design, user, and ethical aspects. By articulating these grand challenges, we aim to begin initiating a research agenda that positions bodily control not only as a technical feature but as a central, experiential, and ethical concern for future human–computer interaction endeavors.

We propose a technical method and system to enable an unmodified FDM 3D printer to fasten fabric in a manner analogous to traditional sewing. Rows of printed “needles” pierce layers of fabric with “thread” printed to fasten them together. This recreates traditional kinds of stitches while also enabling new ones. A technical evaluation shows the strength is as good or better than traditionally sewn seams. Using this sewing inspired joinery method, a software system translates traditional sewing patterns into a multi-step workflow. Steps are ordered and grouped to balance automation with traditional sequencing for human fabric handling. Intermittent requests by the system require the user to place fabric pieces aided by printed registration marks. A study suggests people with little experience can use the system and complete necessary fabric handling steps. Finally, enhanced types of seams are presented, such as decorative stitches, rivets, fringes, and print-in-place fasteners like buttons and zippers.

Haptic technology offers new opportunities for interaction, yet remains inaccessible to people unfamiliar with the technology due to challenges in rapid prototyping and the absence of a widely understood vocabulary, making early-stage design communication difficult. To address these challenges, we developed Feel-Play-Imagine (FPI), a method for haptics experts to involve team members and stakeholders in the early stages of design, and explored its use in the context of storytelling. FPI involves introducing people to haptics through experiencing polished haptic experiences in context (Feel) and experimenting with alternative modalities (Play), then engaging in discussions using stories to imagine designed experiences (Imagine). We report on the results of using FPI in an ongoing co-design project and a lab study with 10 expert storytellers from various backgrounds. Our findings include the value of hands-on and playful experiences to learn about haptic technologies, the ability of FPI to support design decisions, the ability of our developed Worksheet to structure discussion in some contexts, and the need to support multimodal and gestural communication when discussing haptic and tangible interaction.

Designed haptic feedback—technology-mediated touch feedback—has the potential to mediate positive and meaningful experiences. These experiences are rich and complex in nature and thus challenging to design. Established User Experience (UX) and Haptic Experience (HX) models describe the design of experiences; however, they are too general and evaluation-focused to inform haptic experience design. We review 104 publications designing haptic experiences and analyse how researchers consider pragmatic, hedonic, and eudaimonic qualities of haptic experience. Our findings show that researchers mainly engage with the pragmatic qualities of the experience. We thus propose a unified model for HX for understanding the design of haptic experiences, combining key elements of UX and HX research to give haptic designers a tool for thinking about the rich and complex haptic experiences elicited by their designs. This raises open questions for haptic experience research, as designing mediated touch experiences through haptic technology remains challenging.

Haptic Experience (HX) is a proposed set of quality criteria useful to haptics, with prior evidence for a 5-factor model with vibrotactile feedback. We report on an ongoing process of scale development to measure HX, and explore whether these criteria hold when applied to more diverse devices, including vibrotactile, force feedback, surface haptics, and mid-air haptics. From an in-person user study with 430 participants, exploratory factor analysis (EFA), and confirmatory factor analysis (CFA), we extract an 11-item and 4-factor model (Realism, Harmony, Involvement, Expressivity) with only a partial overlap to the previous model. We compare this model to the previous vibrotactile model, finding that the new 4-factor model is more generalized and can guide attributes or applications of new haptic systems. Our findings suggest that HX may vary depending on the modalities used in an application, but these four factors are general constructs that might overlap with modality-specific concepts of HX. These factors can inform designers about the right quality criteria to use when designing or evaluating haptic experiences for multiple modalities.

Human supervisors in multi-robot systems are primarily responsible for monitoring robots, but can also be assigned with secondary tasks. These tasks can act as interruptions and can be categorized as either intrinsic, i.e., being directly related to the monitoring task, or extrinsic, i.e., being unrelated. In this paper, we investigate the impact of these two types of interruptions through a user study (N = 39), where participants monitor a number of remote mobile robots while intermittently being interrupted by either a robot fault correction task (intrinsic) or a messaging task (extrinsic). We find that task performance of participants does not change significantly with the interruptions but depends greatly on the number of robots. However, interruptions result in an increase in perceived workload, and extrinsic interruptions have a more negative effect on workload across all NASA-TLX scales. Participants also reported switching between extrinsic interruptions and the primary task to be more difficult compared to the intrinsic interruption case. Statistical significance of these results is confirmed using ANOVA and one-sample t-test. These findings suggest that when deciding task assignment in such supervision systems, one should limit interruptions from secondary tasks, especially extrinsic ones, in order to limit user workload.

We contribute a technical solution to reduce print time and material with unmodified fused deposition modelling printers. The approach uses ad hoc objects inserted by a user during printing as a replacement for printed support of overhanging structures. Examples of objects include household items like books, toy bricks, and custom mechanisms like a screw jack. A software-only system is integrated into existing slicing software to analyze generated support print paths, search a library of objects to find suitable replacements, optimize combinations of replacement objects, and make necessary adjustments to impacted printing layers and paths. During printing, the user is prompted to insert objects with the help of lightweight printed holders to guide placement and prevent movement. Instructions printed on the build-plate help identify and position objects. A technical evaluation measures performance and benefits with different sets of ad hoc objects and different levels of user involvement.

We present TactJam, an end-to-end suite for creating and sharing low fidelity prototypes of on-body vibrotactile feedback. With TactJam, designers can create, record and share vibrotactile patterns online. This opens up new ways of collaboratively designing vibrotactile patterns both in collocated as well as in remote settings. We evaluate TactJam in a two-part distributed online workshop, exploring the design of on-body tactons. Participants were able to successfully use TactJam to learn about tacton design. We present an overview of mappings between tactons and their associated concepts before comparing the results of tactons created using solely a GUI and tactons created through experimenting with placements directly on the body. Conducting both parts of the workshop separately highlighted the importance of designing directly with bodies: less implicit assumptions were made, and designs were guided by personal experience. We reflect on these results and close on deliberations for the future development of TactJam.

Game designers and researchers employ a sophisticated language for producing great player experiences with concepts such as juiciness, which refers to excessive positive feedback. However, much of their discourse excludes the role and value of haptic feedback. In this paper, we adapt terminology from game design to study haptic feedback. Specifically, we define haptic embellishments (HEs) as haptic feedback that reinforce information already provided through other means (e.g., via visual feedback) and juicy haptics as excessive positive haptic feedback with the intention of improving user experience in games and other interactive media. We report two empirical studies of users’ experiences interacting with visuo-haptic content on their phones to 1) study participants’ preferences for ten design principles for HEs and 2) measure the added value of juicy haptics, implemented as HEs, on player experience in a game. Results indicate that juicy haptics can enhance enjoyability, aesthetic appeal, immersion, and meaning.

We present a software system for fused deposition modelling 3D printing that replaces infill material with scrap to reduce material and energy consumption. Example scrap objects include unused 3D prints from prototyping and calibration, household waste like coffee cups, and off-cuts from other fabrication projects. To achieve this, our system integrates into an existing CAD workflow and manages a database of common items, previous prints, and manually entered objects. While modelling in a standard CAD application, the system suggests objects to insert, ranked by how much infill material they could replace. This computation extends an existing nesting algorithm to determine which objects fit, optimize their alignment, and adjust the enclosing mesh geometry. While printing, the system uses custom tool-paths and animated instructions to enable anyone nearby to manually insert the scrap material.

When interacting with virtual reality (VR) applications like CAD and open-world games, people may want to use gestures as a means of leveraging their knowledge from the physical world. However, people may prefer physical props over handheld controllers to input gestures in VR. We present an elicitation study where 21 participants chose from 95 props to perform manipulative gestures for 20 CAD-like and open-world game-like referents. When analyzing this data, we found existing methods for elicitation studies were insufficient to describe gestures with props, or to measure agreement with prop selection (i.e., agreement between sets of items). We proceeded by describing gestures as context-free grammars, capturing how different props were used in similar roles in a given gesture. We present gesture and prop agreement scores using a generalized agreement score that we developed to compare multiple selections rather than a single selection. We found that props were selected based on their resemblance to virtual objects and the actions they afforded; that gesture and prop agreement depended on the referent, with some referents leading to similar gesture choices, while others led to similar prop choices; and that a small set of carefully chosen props can support multiple gestures.

Autonomous vehicles have been rapidly progressing towards full autonomy using fixed driving styles, which may differ from individual passenger preferences. Violating these preferences may lead to passenger discomfort or anxiety. We studied passenger responses to different driving style parameters in a physical autonomous vehicle. We collected galvanic skin response, heart rate, and eye-movement patterns from 20 participants, along with self-reported comfort and anxiety scores. Our results show that the presence and proximity of a lead vehicle not only raised the level of all measured physiological responses, but also exaggerated the existing effect of the longitudinal acceleration and jerk parameters. Skin response was also found to be a significant predictor of passenger comfort and anxiety. By using multiple independent events to isolate different driving style parameters, we demonstrate a method to control and analyze such parameters in future studies.

Haptic technology is maturing, with expectations and evidence that it will contribute to user experience (UX). However, we have very little understanding about how haptic technology can influence people's experience. Researchers and designers need a way to understand, communicate, and evaluate haptic technology's effect on UX. From a literature review and two studies - one with haptics novices, the other with expert hapticians - we developed a theoretical model of the factors that constitute a good haptic experience (HX). We define HX and propose its constituent factors: design parameters of Timeliness, Density, Intensity, and Timbre; the cross-cutting concern of Personalization; usability requirements of Utility, Causality, Consistency, and Saliency; and experiential factors of Harmony, Expressivity, Autotelics, Immersion, and Realism as guiding constructs important for haptic experience. This model will help guide design and research of haptic systems, inform language around haptics, and provide the basis for evaluative instruments, such as checklists, heuristics, or questionnaires.

We present metamaterial textures---3D printed surface geometries that can perform a controlled transition between two or more textures. Metamaterial textures are integrated into 3D printed objects and allow designing how the object interacts with the environment and the user's tactile sense. Inspired by foldable paper sheets ("origami") and surface wrinkling, our 3D printed metamaterial textures consist of a grid of cells that fold when compressed by an external global force. Unlike origami, however, metamaterial textures offer full control over the transformation, such as in between states and sequence of actuation. This allows for integrating multiple textures and makes them useful, e.g., for exploring parameters in the rapid prototyping of textures. Metamaterial textures are also robust enough to allow the resulting objects to be grasped, pushed, or stood on. This allows us to make objects, such as a shoe sole that transforms from flat to treaded, a textured door handle that provides tactile feedback to visually impaired users, and a configurable bicycle grip. We present an editor assists users in creating metamaterial textures interactively by arranging cells, applying forces, and previewing their deformation.

We present a new haptic device that enables blind users to continuously interact with spatial virtual environments that contain moving objects, as is the case in sports or shooter games. Users interact with DualPanto by operating the me handle with one hand and by holding on to the it handle with the other hand. Each handle is connected to a pantograph haptic input/output device. The key feature is that the two handles are spatially registered with respect to each other. When guiding their avatar through a virtual world using the me handle, spatial registration enables users to track moving objects by having the device guide the output hand. This allows blind players of a 1-on-1 soccer game to race for the ball or evade an opponent; it allows blind players of a shooter game to aim at an opponent and dodge shots. In our user study, blind participants reported very high enjoyment when using the device to play (6.5/7).

Social robots must be believable to be effective; but creating believable, affectively expressive robot behaviours requires time and skill. Inspired by the directness with which performers use their voices to craft characters, we introduce Voodle (vocal doodling), which uses the form of utterances -- e.g., tone and rhythm -- to puppet and eventually control robot motion. Voodle offers an improvisational platform capable of conveying hard-to-express ideas like emotion. We created a working Voodle system by collecting a set of vocal features and associated robot motions, then incorporating them into a prototype for sketching robot behaviour. We explored and refined Voodle's expressive capacity by engaging expert performers in an iterative design process. We found that users develop a personal language with Voodle; that a vocalization's meaning changed with narrative context; and that voodling imparts a sense of life to the robot, inviting designers to suspend disbelief and engage in a playful, conversational style of design.

Haptic force feedback systems are unique in their ability to dynamically render physical representations. Although haptic devices have shown promise for supporting learning, prior work mainly describes results of haptic-supported learning without identifying underlying learning mechanisms. To this end, we designed a haptic-supported learning environment and analyzed four students who used it to make connections between two different mathematical representations of sine and cosine: the unit circle, and their graph on the Cartesian plane. We highlight moments where students made connections between the representations, and identify how the haptic feedback supported these moments of insight. We use this evidence in support of a proposed theoretical and design framework for educational haptics. This framework captures four types of haptic representations, and focuses on one -- the haptic bridge -- that effectively scaffolds sense-making with multiple representations.

Social robots that physically display emotion invite natural communication with their human interlocutors, enabling applications like robot-assisted therapy where a complex robot's breathing influences human emotional and physiological state. Using DIY fabrication and assembly, we explore how simple 1-DOF robots can express affect with economy and user customizability, leveraging open-source designs.

Vibrotactile (VT) display is becoming a standard component of informative user experience, where notifications and feedback must convey information eyes-free. However, effective design is hindered by incomplete understanding of relevant perceptual qualities, together with the need for user feedback to be accessed in-situ. To access evaluation streamlining now common in visual design, we introduce proxy modalities as a way to crowdsource VT sensations by reliably communicating high-level features through a crowd-accessible channel. We investigate two proxy modalities to represent a high-fidelity tactor: a new VT visualization, and low-fidelity vibratory translations playable on commodity smartphones. We translated 10 high-fidelity vibrations into both modalities, and in two user studies found that both proxy modalities can communicate affective features, and are consistent when deployed remotely over Mechanical Turk. We analyze fit of features to modalities, and suggest future improvements.

Chairs, wearables, and handhelds have become popular sites for spatial tactile display. Visual animators, already expert in using time and space to portray motion, could readily transfer their skills to produce rich haptic sensations if given the right tools. We introduce the tactile animation object, a directly manipulated phantom tactile sensation. This abstraction has two key benefits: 1) efficient, creative, iterative control of spatiotemporal sensations, and 2) the potential to support a variety of tactile grids, including sparse displays. We present Mango, an editing tool for animators, including its rendering pipeline and perceptually-optimized interpolation algorithm for sparse vibrotactile grids. In our evaluation, professional animators found it easy to create a variety of vibrotactile patterns, with both experts and novices preferring the tactile animation object over controlling actuators individually.

Persuasive technology is now mobile and context-aware. Intelligent analysis of accelerometer signals in smartphones and other specialized devices has recently been used to classify activity (e.g., distinguishing walking from cycling) to encourage physical activity, sustainable transport, and other social goals. Unfortunately, results vary drastically due to differences in methodology and problem domain. The present report begins by structuring a survey of current work within a new framework, which highlights comparable characteristics between studies; this provided a tool by which we and others can understand the current state-of-the art and guide research towards existing gaps. We then present a new user study, positioned in an identified gap, that pushes limits of current success with a challenging problem: the real-time classification of 15 similar and novel gaits suitable for several persuasive application areas, focused on the growing phenomenon of exercise games. We achieve a mean correct classification rate of 78.1% of all 15 gaits with a minimal amount of personalized training of the classifier for each participant when carried in any of 6 different carrying locations (not known a priori). When narrowed to a subset of four gaits and one location that is known, this improves to means of 92.2% with and 87.2% without personalization. Finally, we group our findings into design guidelines and quantify variation in accuracy when an algorithm is trained for a known location and participant.

The Computing Attitudes Survey (CAS) is a newly designed instrument, adapted from the Colorado Learning Attitudes about Science Survey (CLASS), for measuring novice to expert-like perceptions about computer science. In this paper we outline the iterative design process used for the adaptation and present our progress toward establishing the instrument's validity. We present results of think-aloud interviews and discuss procedures used to determine expert consensus for CAS items. We also detail results of a pilot of the instrument with 447 introductory students in Fall 2011 along with a preliminary factor analysis of this data. Findings to date show consistent interpretation of statements by faculty and students, establish expert consensus of opinion and identify eight candidate factors for further analysis.

Awareness of other people's activity is an important part of shared-workspace collaboration, and is typically supported using visual awareness displays such as radar views. These visual presentations are limited in that the user must be able to see and attend to the view in order to gather awareness information. Using audio to convey awareness information does not suffer from these limitations, and previous research has shown that audio can provide valuable awareness in distributed settings. In this paper we evaluate the effectiveness of synthesized dynamic audio information, both on its own and as an adjunct to a visual radar view. We developed a granular-synthesis engine that produces realistic chalk sounds for off-screen activity in a groupware workspace, and tested the audio awareness in two ways. First, we measured people's ability to identify off-screen activities using only sound, and found that people are almost as accurate with synthesized sounds as with real sounds. Second, we tested dynamic audio awareness in a realistic groupware scenario, and found that adding audio to a radar view significantly improved awareness of off-screen activities in situations where it was difficult to see or attend to the visual display. Our work provides new empirical evidence about the value of dynamic synthesized audio in distributed groupware.