Children often experience emotional distress during periods of separation from their parents, while parents may feel guilt about their limited availability. Yet, existing remote parent–child communication technologies rarely accommodate children’s communicative capacities or provide meaningful support for their agency. Thus, we present CuddleConnect, an interactive system that supports child-initiated, affective remote communication. CuddleConnect combines a weighted plush toy for children with a mobile application for parents, enabling children to express emotions and request hugs through light cues, while allowing parents to receive these requests on their smartphones and respond with mediated hugs delivered via haptic feedback and warmth on the plush toy. We report findings from a one-week in-situ study with four parent–child dyads, highlighting both opportunities for fostering closeness and challenges related to ambiguity and responsiveness. These insights inform design implications for future synchronous and asynchronous parent–child communication technologies.

On-body computing systems offer new forms of interaction, but while they are increasingly integrated into everyday contexts, their unique privacy and safety challenges remain understudied. This paper examines these challenges through a two-round interview study with N = 15 experts in human-computer interaction, and privacy and safety, using speculative scenarios and adversarial roleplaying to elicit insights. Our findings reveal risks specific to on-body interactions, including over-collection of sensitive data, unwanted inferences, harm to bystanders, and threats to bodily autonomy and psychological well-being. Importantly, in the on-body context, privacy and safety concerns are deeply interconnected and cannot be addressed in isolation. We contribute an empirically grounded characterization of these entangled challenges and derive eight actionable design guidelines to support safer, more privacy-aware, on-body systems. This work informs future research and design in ubiquitous computing by highlighting the need for proactive and integrated approaches to privacy and safety in trustworthy on-body computing.

Pull-up objects form 3D shapes by pulling a string routed through a 2D material, offering low-cost 2D fabrication and reversible transformation. However, existing approaches rely on origamic folding, which creates faceted, oftentimes rigid surfaces and requires manual pull-up string routing. We introduce EmbroForm, a digital fabrication pipeline for fully soft pull-up objects with organic, higher-fidelity shapes. Instead of folding, EmbroForm forms 3D shapes by seaming the boundaries of a flexible 2D patch unwrapped from the target. To enable this, we contribute a fabrication technique that automates the routing of sliding strings on flexible sheet materials with machine embroidery, which we extend on to design zig-zag lacings for seaming the boundaries. Then we introduce an end-to-end pipeline that, given a 3D mesh, creates an optimized 2D unwrapped patch and generates pull-up string routing paths for fabrication. We provide a design tool for customization and validate our approach with technical experiments and implemented application cases.

Haptic feedback contributes to immersive virtual reality (VR) experiences. However, designing such feedback at scale for all objects within a VR scene remains time-consuming. We present Scene2Hap, an LLM-centered system that automatically designs object-level vibrotactile feedback for entire VR scenes based on the objects’ semantic attributes and physical context. Scene2Hap employs a multimodal large language model to estimate each object’s semantics and physical context, including its material properties and vibration behavior, from multimodal information in the VR scene. These estimated attributes are then used to generate or retrieve audio signals, subsequently converted into plausible vibrotactile signals. For more realistic spatial haptic rendering, Scene2Hap estimates vibration propagation and attenuation from vibration sources to neighboring objects, considering the estimated material properties and spatial relationships of virtual objects in the scene. Three user studies confirm that Scene2Hap successfully estimates the vibration-related semantics and physical context of VR scenes and produces realistic vibrotactile signals.

Human-in-the-loop optimization identifies optimal interface designs by iteratively observing user performance. However, it often requires numerous iterations due to the lack of prior information. While recent approaches have accelerated this process by leveraging previous optimization data, collecting user data remains costly and often impractical. We present a conceptual framework, Human-in-the-Loop Optimization with Model-Informed Priors (HOMI), which augments human-in-the-loop optimization with a training phase where the optimizer learns adaptation strategies from diverse, synthetic user data generated with predictive models before deployment. To realize HOMI, we introduce Neural Acquisition Function+ (NAF+), a Bayesian optimization method featuring a neural acquisition function trained with reinforcement learning. NAF+ learns optimization strategies from large-scale synthetic data, improving efficiency in real-time optimization with users. We evaluate HOMI and NAF+ with mid-air keyboard optimization, a representative VR input task. Our work presents a new approach for more efficient interface adaptation by bridging in situ and in silico optimization processes.

Virtual Reality (VR) depends on haptic feedback to create immersive experiences. Traditional passive proxies align physical props with their virtual counterparts but remain limited in scalability and expressiveness, or require bulky actuators to support reconfiguration. We introduce User-reconfigured Haptics, an approach that utilizes implicit user actions to reconfigure haptic interfaces to extend the gamut of VR haptic experiences. Modular 3D-printed cells are assembled into dynamic interfaces that express diverse haptic properties such as softness and weight. By masking physical reconfigurations with visual (re)mapping, user actions unnoticeably change haptic properties, resulting in user-driven, dynamic haptic experiences. User studies show that our design can provide distinguishable haptic experiences and is perceived as realistic and enjoyable in a VR task. We further showcase four applications: a fishing rod that changes weight and flexibility, a dynamic desktop of pressable buttons, a glove with adjustable squeezing, and a crossbow with variable pulling resistance.

The haptic properties of textile interfaces support eyes-free use well. However, user interfaces often require composing many controls, which complicates orientation by palpation. We investigated three composition concepts that combine textile components in different ways, using a sample smart home scenario: One uses the select-and-control flow of universal remotes, one resembles the users’ environment, and one divides the interface into application sections. Of the latter, we also created a spacious variant to observe the effect of blank space between interface regions. We first explored how well users understand those composition concepts when first using the interface eyes-free. After familiarization, we measured input performance and preferences. We found task completion times much faster than expected from previous recognition studies. While performance was similar for most interfaces, the select-and-control flow was slower, but participants preferred it after familiarization. From our findings, we derive design recommendations for compound textile UIs and individual components.

Supernumerary robotic limbs (SRLs) are robotic structures integrated closely with the user's body, which augment human physical capabilities and necessitate seamless, naturalistic human-machine interaction. For effective assistance in physical tasks, enabling SRLs to hand over objects to humans is crucial. Yet, designing heuristic-based policies for robots is time-consuming, difficult to generalize across tasks, and results in less human-like motion. When trained with proper datasets, generative models are powerful alternatives for creating naturalistic handover motions. We introduce 3HANDS, a novel dataset of object handover interactions between a participant performing a daily activity and another participant enacting a hip-mounted SRL in a naturalistic manner. 3HANDS captures the unique characteristics of SRL interactions: operating in intimate personal space with asymmetric object origins, implicit motion synchronization, and the user's engagement in a primary task during the handover. To demonstrate the effectiveness of our dataset, we present three models: one that generates naturalistic handover trajectories, another that determines the appropriate handover endpoints, and a third that predicts the moment to initiate a handover. In a user study (N=10), we compare the handover interaction performed with our method compared to a baseline. The findings show that our method was perceived as significantly more natural, less physically demanding, and more comfortable.

Exoskeletons open up a unique interaction space that seamlessly integrates users' body movements with robotic actuation. Despite its potential, human-exoskeleton interaction remains an underexplored area in HCI, largely due to the lack of accessible prototyping tools that enable designers to easily develop exoskeleton designs and customized interactive behaviors. We present ExoKit, a do-it-yourself toolkit for rapid prototyping of low-fidelity, functional exoskeletons targeted at novice roboticists. ExoKit includes modular hardware components for sensing and actuating shoulder and elbow joints, which are easy to fabricate and (re)configure for customized functionality and wearability. To simplify the programming of interactive behaviors, we propose functional abstractions that encapsulate high-level human-exoskeleton interactions. These can be readily accessed either through ExoKit's command-line or graphical user interface, a Processing library, or microcontroller firmware, each targeted at different experience levels. Findings from implemented application cases and two usage studies demonstrate the versatility and accessibility of ExoKit for early-stage interaction design.

More and more immersive environments support third-party applications, leading to concerns about the trustworthiness of user interfaces (UIs), which attackers could exploit, endangering users. Although security warnings attempt to safeguard users by highlighting risks, most studies primarily target security indicators as usable intervention for app transitions in virtual reality. Our research broadens this focus by providing a systematic, data-driven investigation of security warnings for third-party applications in immersive environments. We analyzed a decade’s worth of top VR interactions and security conference findings and assessed the top 10 free VR applications from two leading stores each. From our design process, we implemented four warnings. Through two user studies involving 61 participants, we measured their responses to these warnings during virtual object interactions. Our findings indicate that a red glow on an object was the most effective warning, frequently associated with danger, while pop-up warnings were the least effective.

Rehearse 3/10Why are we here, why are we alive?One key reason is that our ancestors on the savannas of Africa were really good at one thing…They weren't bigger than the animals they took down a lot of the time, they weren't faster than the animals they took down a lot of the time, but they were much better at banding together into groups and cooperating.…One key reason… Proactive Gesture Cues Current Slide Hover to Preview and Modify Gestures Presenter Notes with Gesture highlights Edit DeleteWhy were humans at hunting ?• Not bigger or faster than animals • Good at cooperationFigure 1: GestureCoach guides speakers to perform gestures while rehearsing their talk.A gesture recommendation model predicts text segments in the presenter notes that should be emphasized with gestures and retrieves relevant semantic gestures for each segment.During rehearsal, the system highlights the segments and proactively cues a video clip of the gesture by tracking users' speech, allowing them to integrate the gesture smoothly into their talk.Hovering over a segment allows users to preview and modify the associated gesture with alternate suggestions from the model.

Figure 1: We propose proprioceptive motion feedback to align user's physical movements with the expected motion style of their avatar to enhance embodiment in virtual reality (VR).MotionStyler is a proof-of-concept system for designing and rendering such proprioceptive motion styles in real-time in VR with an arm-based exoskeleton.Based on a conceptual space comprising eight motion properties grouped into four key dimensions, an accompanying design tool helps designers to combine individual properties into expressive proprioceptive motion styles; e.g., they might mimic the motion style of a rigid treefolk by combining the sensation of heavy, creaky limbs with a reduced motion speed and range.

Bio-based and bio-degradable materials have shown promising results for sustainable Human-Computer Interaction (HCI) applications, including shape-changing interfaces. However, the diversity of shape-changing behaviors achievable with these materials remains unclear as the fabrication knowledge is scattered across multiple research fields. This paper introduces SoftBioMorph, a fabrication framework that aims to integrate the fabrication know-how of sustainable soft shape-changing interfaces with biopolymers. Based on the example of Sodium Alginate, the framework contributes (1) a set of material synthesis processes that modify the biopolymer’s properties to fulfill different functions; (2) a set of DIY crafting-based assembling techniques that functionalize the material and assembling properties to achieve three primitive types of change in shape; and (3) a series of application cases that demonstrate the versatility of the framework. We further discuss limitations, research questions, and fabrication challenges, presenting a comprehensive approach to sustainable prototyping in HCI.

Parent-child separation can detrimentally affect a child’s well-being. While digital technologies enable remote communication, they often lack effective means for young children to initiate contact, engage physically, and express emotions. Pioneering work on tangible devices for communication offer promising solutions, but current research minimally explores devices tailored to the differing needs of parents and children during separation, and fewer evaluate their designs with functional prototypes. Building on design suggestions from previous work, we implement a functional pair of prototypes for remote parent-child communication: ChiParCo, an interactive plush bunny for children, and ParChiCo, an interactive glove for parents. The pair provides three interactions: waving, hand-holding and emotion-sharing, which combine physical, synchronous and asynchronous, as well as emotive communication. In a qualitative user study with parent-child dyads (n=10), we investigate the form factors of the prototypes, participants’ perceptions of the three interactions and what design implications we can draw from their feedback.

Shape Memory Alloys (SMAs) afford the seamless integration of shape-changing behaviour into textiles, enabling designers to augment apparel with dynamic shaping and styling. However, existing works fall short of providing versatile methods adaptable to varying scales, materials, and applications, curtailing designers’ capacity to prototype customised solutions. To address this, we introduce Flextiles, parameterised SMA design schema that leverage the traditional craft of smocking to integrate planar shape-change seamlessly into diverse textile projects. The conception of Flextiles stems from material experimentation and consultative dialogues with designers, whose insights inspired strategies for customising scale, elasticity, geometry, and actuation of Flextiles. To support the practical implementation of Flextiles, we provide a design tool and experimentally characterise their material properties. Lastly, through a workshop with practitioners, we explore the multifaceted applications and perspectives surrounding Flextiles, and subsequently realise four scenarios that illustrate the creative potential of these modular, customisable patterns.

Compliance, the degree of displacement under applied force, is pivotal in determining the material perception when touching an object. Vibrotactile actuators can be used for creating grain-based virtual compliance, but they have poor spatial resolution and a limiting rigid form factor. We propose a novel electrotactile compliance illusion that renders grains of electrical pulses on an electrode array in response to finger force changes. We demonstrate its ability to render compliance in distinct shapes through a thin, lightweight, and flexible finger-worn interface. Detailed technical parameters and the implementation of our device are provided. A controlled experiment confirms the technique can (1) create virtual compliance; (2) adjust the compliance magnitude with grain and electrode parameters; and (3) render compliance with specific shapes. In three example applications, we present how this illusion can enhance physical objects, elements in graphical user interfaces, and virtual reality experiences.

Machine embroidery is a versatile technique for creating custom and entirely fabric-based patterns on thin and conformable textile surfaces. However, existing machine-embroidered surfaces remain static, limiting the interactions they can support. We introduce Embrogami, an approach for fabricating textile structures with versatile shape-changing behaviors. Inspired by origami, we leverage machine embroidery to form finger-tip-scale mountain-and-valley structures on textiles with customized shapes, bistable or elastic behaviors, and modular composition. The structures can be actuated by the user or the system to modify the local textile surface topology, creating interactive elements like toggles and sliders or textile shape displays with an ultra-thin, flexible, and integrated form factor. We provide a dedicated software tool and report results of technical experiments to allow users to flexibly design, fabricate, and deploy customized Embrogami structures. With four application cases, we showcase Embrogami’s potential to create functional and flexible shape-changing textiles with diverse visuo-tactile feedback.

There is an increasing interest in combining interactive technology with food, leading to a new research area called human-food interaction. While food experiences are increasingly benefiting from interactive technology, for example in the form of food tracking apps, 3D-printed food and projections on dining tables, a more systematic advancement of the field is hindered because, so far, there is no comprehensive articulation of the grand challenges the field is facing. To further and consolidate conversations around this topic, we invited 21 HFI experts to a 5-day seminar. The goal was to review our own and prior work to identify the grand challenges in human-food interaction. The result is an articulation of 10 grand challenges in human-food interaction across 4 categories (technology, users, design and ethics). By presenting these grand challenges, we aim to help researchers move the human-food interaction research field forward.

Mutual-capacitive sensing is the most common technology for detecting multi-touch, especially on flat and simple curvature surfaces. Its extension to a more complex shape is still challenging, as a uniform distribution of sensing electrodes is required for consistent touch sensitivity across the surface. To overcome this problem, we propose a method to adapt the sensor layout of common capacitive multi-touch sensors to more complex 3D surfaces, ensuring high-resolution, robust multi-touch detection. The method automatically computes a grid of transmitter and receiver electrodes with as regular distribution as possible over a general 3D shape. It starts with the computation of a proxy geometry by quad meshing used to place the electrodes through the dual-edge graph. It then arranges electrodes on the surface to minimize the number of touch controllers required for capacitive sensing and the number of input/output pins to connect the electrodes with the controllers. We reach these objectives using a new simplification and clustering algorithm for a regular quad-patch layout. The reduced patch layout is used to optimize the routing of all the structures (surface grooves and internal pipes) needed to host all electrodes on the surface and inside the object's volume, considering the geometric constraints of the 3D shape. Finally, we print the 3D object prototype ready to be equipped with the electrodes. We analyze the performance of the proposed quad layout simplification and clustering algorithm using different quad meshing and characterize the signal quality and accuracy of the capacitive touch sensor for different non-planar geometries. The tested prototypes show precise and robust multi-touch detection with good Signal-to-Noise Ratio and spatial accuracy of about 1mm.

Wearable robotic arms (WRA) open up a unique interaction space that closely integrates the user’s body with an embodied robotic collaborator. This space affords diverse interaction styles, including body movement, hand gestures, or gaze. Yet, it is so-far unexplored which commands are desirable from a user perspective. Contributing findings from an elicitation study (N=14), we provide a comprehensive set of interactions for basic robot control, navigation, object manipulation, and emergency situations, performed when hands are free or occupied. Our study provides insights into preferred body parts, input modalities, and the users’ underlying sources of inspiration. Comparing interaction styles between WRAs and off-body robots, we highlight how WRAs enable a range of interactions specific for on-body robots and how users use WRAs both as tools and as collaborators. We conclude by providing guidance on the design of ad-hoc interaction with WRAs informed by user behavior.

Personal fabrication has mostly focused on handheld tools as embodied extensions of the user, and machines like laser cutters and 3D printers automating parts of the process without intervention. Although interactive digital fabrication has been explored as a middle ground, existing systems have a fixed allocation of user intervention vs. machine autonomy, limiting flexibility, creativity, and improvisation. We explore a new class of devices that combine the desirable properties of a handheld tool and an autonomous fabrication robot, offering a continuum from manual and assisted to autonomous fabrication, with seamless mode transitions. We exemplify the concept of mixed-initiative physical sketching with a working robotic printer that can be handheld for free-hand sketching, can provide interactive assistance during sketching, or move about for computer-generated sketches. We present interaction techniques to seamlessly transition between modes, and sketching techniques benefitting from these transitions to, e.g., extend (upscale, repeat) or revisit (refine, color) sketches. Our evaluation with seven sketchers illustrates that RoboSketch successfully leverages each mode’s strengths, and that mixed-initiative physical sketching makes computer-supported sketching more flexible.

When vibrations are synchronized with our actions, we experience them as material properties. This has been used to create virtual experiences like friction, counter-force, compliance, or torsion. Implementing such experiences is non-trivial, requiring high temporal resolution in sensing, high fidelity tactile output, and low latency. To make this style of haptic feedback more accessible to non-domain experts, we present Haptic Servos: self-contained haptic rendering devices which encapsulate all timing-critical elements. We characterize Haptic Servos’ real-time performance, showing the system latency is <5 ms. We explore the subjective experiences they can evoke, highlighting that qualitatively distinct experiences can be created based on input mapping, even if stimulation parameters and algorithm remain unchanged. A workshop demonstrated that users new to Haptic Servos require approximately ten minutes to set up a basic haptic rendering system. Haptic Servos are open source, we invite others to copy and modify our design.

Wearable robotic limbs (WRLs) augment human capabilities through robotic structures that attach to the user’s body. While WRLs are intensely researched and various device designs have been presented, it remains difficult for non-roboticists to engage with this exciting field. We aim to empower interaction designers and application domain experts to explore novel designs and applications by rapidly prototyping personalized WRLs that are customized for different tasks, different body locations, or different users. In this paper, we present WRLKit, an interactive computational design approach that enables designers to rapidly prototype a personalized WRL without requiring extensive robotics and ergonomics expertise. The body-aware optimization approach starts by capturing the user’s body dimensions and dynamic body poses. Then, an optimized fabricable structure of the WRL is generated for a desired mounting location and workspace of the WRL, to fit the user’s body and intended task. The results of a user study and several implemented prototypes demonstrate the practical feasibility and versatility of WRLKit.

For grasping, tactile stimuli to multiple fingertips are crucial for realistic shape rendering and precise manipulation. Pinching is particularly important in virtual reality since it is frequently used to grasp virtual objects. However, the interaction space of tactile feedback around pinching is underexplored due to a lack of means to provide co-located but different stimulation to finger pads. We propose a double-sided electrotactile device with a thin and flexible form factor to fit within pinched fingerpads, comprising two overlapping 3 × 3 electrode arrays. Using this new tactile interface, we define a new concept of double-sided tactile interactions with three feedback modes: (1) single-sided stimulation, (2) simultaneous double-sided stimulation, and (3) spatiotemporal double-sided stimulation. Through two user studies, we (1) demonstrate that participants can accurately discriminate between single-sided and double-sided stimulation and find a qualitative difference in tactile sensation; and (2) confirm the occurrence of apparent tactile motion between fingers and present optimal parameters for continuous or discrete movements. Based on these findings, we demonstrate five VR applications to exemplify how double-sided tactile interactions can produce spatiotemporal movement of a virtual object between fingers and enrich touch feedback for UI operation.

Living bio-materials are increasingly used in HCI for fabricating objects by growing. However, how to integrate electronics to make these objects interactive still needs to be clarified. This paper presents an exploration of the fabrication design space of Biohybrid Interactive Devices, a class of interactive devices fabricated by merging electronic components and living organisms. From the exploration of this space using bacterial cellulose, we outline a fabrication framework centered on the biomaterials‘ life cycle phases. We introduce a set of novel fabrication techniques for embedding conductive elements, sensors, and output components through biological (e.g. bio-fabrication and bio-assembling) and digital processes. We demonstrate the combinatory aspect of the framework by realizing three tangible, wearable, and shape-changing interfaces. Finally, we discuss the sustainability of our approach, its limitations, and the implications for bio-hybrid systems in HCI.

Gestural interaction with freehands and while grasping an everyday object enables always-available input . To sense such gestures, minimal instrumentation of the user’s hand is desirable. However, the choice of an effective but minimal IMU layout remains challenging, due to the complexity of the multi-factorial space that comprises diverse finger gestures, objects, and grasps. We present SparseIMU , a rapid method for selecting minimal inertial sensor-based layouts for effective gesture recognition. Furthermore, we contribute a computational tool to guide designers with optimal sensor placement. Our approach builds on an extensive microgestures dataset that we collected with a dense network of 17 inertial measurement units (IMUs). We performed a series of analyses, including an evaluation of the entire combinatorial space for freehand and grasping microgestures (393 K layouts), and quantified the performance across different layout choices, revealing new gesture detection opportunities with IMUs. Finally, we demonstrate the versatility of our method with four scenarios.

Human hair opens up new opportunities for embodied interactions that build on its unique physical affordances and location on the body. As hair has high socio-cultural significance, the design of hair interfaces is coupled with social and personal needs. Albeit this makes field investigations indispensable, they are missing from prior work. We present a fabrication approach for gesture-controlled hair interfaces that are robust enough to be deployed in the field. Our approach contributes sensorized feather hair extensions that combine capacitive and piezoresistive sensing. Their tactile properties make the interface blend seamlessly with human hair. We furthermore contribute results from a field experiment where participants gained first-hand experience in various social contexts. These show how hair-based interactions have great potential moving beyond planar touch gestures whilst their social appropriateness is context-sensitive. We synthesize the findings into design implications that ground the future design of usable and socially acceptable hair interfaces.

Skin is a promising interaction medium and has been widely explored for mobile, and expressive interaction. Recent research in HCI has seen the development of Epidermal Computing Devices: ultra-thin and non-invasive devices which reside on the user’s skin, offering intimate integration with the curved surfaces of the body, while having physical and mechanical properties that are akin to skin, expanding the horizon of on-body interaction. However, with rapid technological advancements in multiple disciplines, we see a need to synthesize the main open research questions and opportunities for the HCI community to advance future research in this area. By systematically analyzing Epidermal Devices contributed in the HCI community, physical sciences research and from our experiences in designing and building Epidermal Devices, we identify opportunities and challenges for advancing research across five themes. This multi-disciplinary synthesis enables multiple research communities to facilitate progression towards more coordinated endeavors for advancing Epidermal Computing.

We present Print-A-Sketch, an open-source handheld printer prototype for sketching circuits and sensors. Print-A-Sketch combines desirable properties from free-hand sketching and functional electronic printing. Manual human control of large strokes is augmented with computer control of fine detail. Shared control of Print-A-Sketch supports sketching interactive interfaces on everyday objects – including many objects with materials or sizes which otherwise are difficult to print on. We present an overview of challenges involved in such a system and show how these can be addressed using context-aware, dynamic printing. Continuous sensing ensures quality prints by adjusting inking-rate to hand movement and material properties. Continuous sensing also enables the print to adapt to previously printed traces to support incremental and iterative sketching. Results show good conductivity on many materials and high spatial precision, supporting on-the-fly creation of functional interfaces.

This paper presents XRtic, a prototyping toolkit enabling real-world cloth deformations to be used in novel ways in eXtended Reality (XR) applications. XRtic was developed based on the insights gathered from semi-structured interviews with XR developers. It consists of custom-made actuators that can be attached to regular clothing, a controller bus system, and a controller interface. Using our toolkit, users can design and integrate different cloth deformation types synchronised with virtual content in a plug-and-play manner. Along with a technical analysis of the actuation behaviour of the XRtic actuators, we present the findings gathered from a user study with eight XR developers, focusing on the usability of the system and creative support. Overall, participants found it an easy-to-use toolkit that supports iterative and rapid prototyping, and enables cloth to be deformed in unique ways in synchronisation with XR applications. Based on the findings, we also report limitations and future work relating to our system.

Designers and makers are increasingly interested in leveraging bio-based and bio-degradable ‘do-it-yourself’ (DIY) materials for sustainable prototyping. Their self-produced bioplastics possess compelling properties such as self-adhesion but have so far not been functionalized to create soft interactive devices, due to a lack of DIY techniques for the fabrication of functional electronic circuits and sensors. In this paper, we contribute a DIY approach for creating Interactive Bioplastics that is accessible to a wide audience, making use of easy-to-obtain bio-based raw materials and familiar tools. We present three types of conductive bioplastic materials and their formulation: sheets, pastes and foams. Our materials enable additive and subtractive fabrication of soft circuits and sensors. Furthermore, we demonstrate how these materials can substitute conventional prototyping materials, be combined with off-the-shelf electronics, and be fed into a sustainable material ‘life-cycle’ including disassembly, re-use, and re-melting of materials. A formal characterization of our conductors highlights that they are even on-par with commercially available carbon-based conductive pastes.

We are at a tipping point for digital communication: moving beyond 'ways of seeing' to include 'ways of feeling'.Much as optical technologies transformed sight and the visual (from the telescope and microscope to Google Glass), the rapid expansion in digital touch technologies is set to reconfigure touch and the tactile in significant ways.Advances in haptics, virtual reality, and physiological sensing provide new sensory ways of communicating, as well as new ways to capture the quality of touch.These state-of-the-art digital touch technologies promise to supplement, heighten, extend and reconfigure how people communicate.They are reshaping what and who can be touched, as well as when and how they can be touched, changing existing forms of communication and giving rise to changes in co-located and remote communication between humans, and between humans and robots.These developments sit alongside social discourses of concern and loss, with the digital being associated with the removal of touch from the material sensory landscape (Jewitt et al., 2020).Yet, in our current climate of social distancing and disengaging with touch in our everyday interactions, the promise of the digital becomes increasingly appealing and significant, in re-enabling touch possibilities.As the global population emerges from and comes to terms with their experiences of the COVID-19 pandemic, these questions become all the more significant.The breadth and interdisciplinary interest in this growing field-across designers, artists, computer scientists, engineers, psychologists and social scientists, with interests in robotics and touch, affective computing, wearables, and digital installations -brings attention to the growing need to engage with 'social' aspects of digital touch, moving beyond technologies and physiological foci to engage with touch practices (Jewitt et al., 2021).We argue for the need to think about touch beyond the physiological act of sensing and perceiving to a more rich and nuanced interpretation of touch that takes account of its emotional and psychological significance, the social, cultural and historical evolution of touch practices in human communication, and approaches to touch of the "lived, social body as a site of meaning-making, where the skin acts as both a boundary between and a point of connection with others" (Karpashevich et al.).Given this landscape, this special issue addresses timely and important questions around the need to think about touch in different ways.This provides a new direction in the field that seeks to address the complexity of human touch and interrogates the limitation of today's haptic devices.Papers in this special issue contribute to understanding this gap by engaging with socio-cultural issues around digital touch communication and identifying new ways to study touch, specifically aiming to explore the nuance and subtlety of touch and an expanded view of touch.

We are surrounded by sensing devices. We are accustomed to them, appreciate their benefits, and even create affective bonds and might neglect the implications they might have for our daily life. By presenting Eyecam, an anthropomorphic webcam mimicking a human eye, we challenge conventional relationships with ubiquitous sensing devices and call to re-think how sensing devices might appear and behave. Inspired by critical design, Eyecam is an exaggeration of a familiar sensing device which allows for critical reflections on its perceived functionalities and its impact on human-human and human-device relations. We identify 5 different roles Eyecam can take: Mediator, Observer, Mirror, Presence, and Agent. Contributing design fictions and thinking prompts, we allow for articulation on privacy awareness and intrusion, affect in mediated communication, agency and self-perception along with speculation on potential futures. We envision this work to contribute to a bold and responsible design of ubiquitous sensing devices.

Today’s typical input device is flat, rigid and made of glass. However, advances in sensing technology and interaction design suggest thinking about input on other surface, including soft materials. While touching rigid and soft materials might feel similar, they clearly feel different when pressure is applied to them. Yet, to date, studies only investigated force input on rigid surfaces. We present a first systematic evaluation of the effects of compliance on force input. Results of a visual targeting task for three levels of softness indicate that high force levels appear more demanding for soft surfaces, but that performance is otherwise similar. Performance remained very high (∼ 5% for 20 force levels) regardless of the compliance, suggesting force input was underestimated so far. We infer implications for the design of force input on soft surfaces and conclude that interaction models used on rigid surfaces might be used on soft surfaces.

Emerging research has demonstrated the viability of on-textile actuation mechanisms, however, an easily customizable and versatile on-cloth actuation mechanism is yet to be explored. In this paper, we present ClothTiles along with its rapid fabrication technique that enables actuation of clothes. ClothTiles leverage flexible 3D-printing and Shape-Memory Alloys (SMAs) alongside new parametric actuation designs. We validate the concept of fabric actuation using a base element, and then systematically explore methods of aggregating, scaling, and orienting prospects for extended actuation in garments. A user study demonstrated that our technique enables multiple actuation types applied across a variety of clothes. Users identified both aesthetic and functional applications of ClothTiles. We conclude with a number of insights for the Do-It-Yourself community on how to employ 3D-printing with SMAs to enable actuation on clothes.

In traditional body-art, designs are adjusted to the body as they are applied, enabling creative improvisation and exploration. Conventional design and fabrication methods of epidermal interfaces, however, separate these steps. With BodyStylus we present the first computer-assisted approach for on-body design and fabrication of epidermal interfaces. Inspired by traditional techniques, we propose a hand-held tool that augments freehand inking with digital support: projected in-situ guidance assists creating valid on-body circuits and aesthetic ornaments that align with the human bodyscape, while pro-active switching between inking and non-inking creates error preventing constraints. We contribute BodyStylus’s design rationale and interaction concept along with an interactive prototype that uses self-sintering conductive ink. Results of two focus group explorations showed that guidance was more appreciated by artists, while constraints appeared more useful to engineers, and that working on the body inspired critical reflection on the relationship between bodyscape, interaction, and designs.

Using microgestures, prior work has successfully enabled gestural interactions while holding objects. Yet, these existing methods are prone to false activations caused by natural finger movements while holding or manipulating the object. We address this issue with SoloFinger, a novel concept that allows design of microgestures that are robust against movements that naturally occur during primary activities. Using a data-driven approach, we establish that single-finger movements are rare in everyday hand-object actions and infer a single-finger input technique resilient to false activation. We demonstrate this concept’s robustness using a white-box classifier on a pre-existing dataset comprising 36 everyday hand-object actions. Our findings validate that simple SoloFinger gestures can relieve the need for complex finger configurations or delimiting gestures and that SoloFinger is applicable to diverse hand-object actions. Finally, we demonstrate SoloFinger’s high performance on commodity hardware using random forest classifiers.

Physical Human-Robot-Interaction (pHRI) is beneficial for communication in social interaction or to perform collaborative tasks but is also crucial for safety. While robotic devices embed sensors for this sole purpose, their design often is the results of a trade-off between technical capabilities and rarely considers human factors. We propose a novel approach to design and fabricate compliant Human-like artificial skin sensors for robots, with similar mechanical properties as human skin and capable of precisely detecting touch. Our artificial skin relies on the use of different silicone elastomers to replicate the human skin layers and comprises an embedded electrode matrix to perform mutual capacitance sensing. We present the sensor and describe its fabrication process which is scalable, low-cost and ensures flexibility, compliance and robustness. We introduce Muca, an open-source sensing development board and then evaluate the performance of the sensor.

Effective haptic feedback in virtual reality (VR) is an essential element for creating convincing immersive experiences. To design such feedback, state-of-the-art VR setups provide APIs for programmatically generating controller vibration patterns. While tools for designing vibrotactile feedback keep evolving, they often require expert knowledge and rarely support direct manipulation methods for mapping feedback to user interactions within the VR environment. To address these challenges, we contribute a novel concept called Weirding Haptics, that supports fast-prototyping by leveraging the user’s voice to design such feedback while manipulating virtual objects in-situ. Through a pilot study (N = 9) focusing on how tactile experiences are vocalized during object manipulation, we identify spatio-temporal mappings and supporting features needed to produce intended vocalizations. To study our concept, we built a VR design tool informed by the results of the pilot study. This tool enables users to design tactile experiences using their voice while manipulating objects, provides a set of modifiers for fine-tuning the created experiences in VR, and allows to rapidly compare various experiences by feeling them. Results from a validation study (N = 8) show that novice hapticians can vocalize experiences and refine their designs with the fine-tuning modifiers to match their intentions. We conclude our work by discussing uncovered design implications for direct manipulation and vocalization of vibrotactile feedback in immersive virtual environments.

We present a method for enabling arbitrary textiles to sense pressure and deformation: In-situ polymerization supports integration of piezoresistive properties at the material level, preserving a textile's haptic and mechanical characteristics. We demonstrate how to enhance a wide set of fabrics and yarns using only readily available tools. To further support customisation by the designer, we present methods for patterning, as needed to create circuits and sensors, and demonstrate how to combine areas of different conductance in one material. Technical evaluation results demonstrate the performance of sensors created using our method is comparable to off-the-shelf piezoresistive textiles. As application examples, we demonstrate rapid manufacturing of on-body interfaces, tie-dyed motion-capture clothing, and zippers that act as potentiometers.

Rapid prototyping of interactive textiles is still challenging, since manual skills, several processing steps, and expert knowledge are involved. We present Rapid Iron-On User Interfaces, a novel fabrication approach for empowering designers and makers to enhance fabrics with interactive functionalities. It builds on heat-activated adhesive materials consisting of smart textiles and printed electronics, which can be flexibly ironed onto the fabric to create custom interface functionality. To support rapid fabrication in a sketching-like fashion, we developed a handheld dispenser tool for directly applying continuous functional tapes of desired length as well as discrete patches. We introduce versatile compositions techniques that allow for creating complex circuits, utilizing commodity textile accessories and sketching custom-shaped I/O modules. We further contribute a comprehensive library of components for input, output, wiring and computing. Three example applications, results from technical experiments and expert reviews demonstrate the functionality, versatility and potential of this approach.

Advances in rapid prototyping platforms have made physiological sensing accessible to a wide audience. However, off-the-shelf electrodes commonly used for capturing biosignals are typically thick, non-conformal and do not support customization. We present PhysioSkin, a rapid, do-it-yourself prototyping method for fabricating custom multi-modal physiological sensors, using commercial materials and a commodity desktop inkjet printer. It realizes ultrathin skin-conformal patches (~1μm) and interactive textiles that capture sEMG, EDA and ECG signals. It further supports fabricating devices with custom levels of thickness and stretchability. We present detailed fabrication explorations on multiple substrate materials, functional inks and skin adhesive materials. Informed from the literature, we also provide design recommendations for each of the modalities. Evaluation results show that the sensor patches achieve a high signal-to-noise ratio. Example applications demonstrate the functionality and versatility of our approach for prototyping a next generation of physiological devices that intimately couple with the human body.

Many features of materials can be experienced through tactile cues, even using one's feet. For example, one can easily distinguish between moss and stone without looking at the ground. However, this type of material experience is largely not supported in AR and VR applications. We present bARefoot, a prototype shoe providing tactile impulses tightly coupled to motor actions. This enables generating virtual material experiences such as compliance, elasticity, or friction. To explore the parameter space of such sensorimotor coupled vibrations, we present a design tool enabling rapid design of virtual materials. We report initial explorations to increase understanding of how parameters can be optimized for generating compliance, and to examine the effect of dynamic parameters on material experiences. Finally, we present a series of use cases that demonstrate the potential of bARefoot for VR and AR.

This paper introduces LASEC, the first technique for instant do-it-yourself fabrication of circuits with custom stretchability on a conventional laser cutter and in a single pass. The approach is based on integrated cutting and ablation of a two-layer material using parametric design patterns. These patterns enable the designer to customize the desired stretchability of the circuit, to combine stretchable with non-stretchable areas, or to integrate areas of different stretchability. For adding circuits on such stretchable cut patterns, we contribute routing strategies and a real-time routing algorithm. An interactive design tool assists designers by automatically generating patterns and circuits from a high-level specification of the desired interface. The approach is compatible with off-the-shelf materials and can realize transparent interfaces. Several application examples demonstrate the versatility of the novel technique for applications in wearable computing, interactive textiles, and stretchable input devices.

We introduce Springlets, expressive, non-vibrating mechanotactile interfaces on the skin. Embedded with shape memory alloy springs, we implement Springlets as thin and flexible stickers to be worn on various body locations, thanks to their silent operation even on the neck and head. We present a technically simple and rapid technique for fabricating a wide range of Springlet interfaces and computer-generated tactile patterns. We developed Springlets for six tactile primitives: pinching, directional stretching, pressing, pulling, dragging, and expanding. A study placing Springlets on the arm and near the head demonstrates Springlets' effectiveness and wearability in both stationary and mobile situations. We explore new interactive experiences in tactile social communication, physical guidance, health interfaces, navigation, and virtual reality gaming, enabled by Springlets' unique and scalable form factor.

The emerging class of epidermal devices opens up new opportunities for skin-based sensing, computing, and interaction. Future design of these devices requires an understanding of how skin-worn devices affect the natural tactile perception. In this study, we approach this research challenge by proposing a novel classification system for epidermal devices based on flexural rigidity and by testing advanced adhesive materials, including tattoo paper and thin films of poly (dimethylsiloxane) (PDMS). We report on the results of three psychophysical experiments that investigated the effect of epidermal devices of different rigidity on passive and active tactile perception. We analyzed human tactile sensitivity thresholds, two-point discrimination thresholds, and roughness discrimination abilities on three different body locations (fingertip, hand, forearm). Generally, a correlation was found between device rigidity and tactile sensitivity thresholds as well as roughness discrimination ability. Surprisingly, thin epidermal devices based on PDMS with a hundred times the rigidity of commonly used tattoo paper resulted in comparable levels of tactile acuity. The material offers the benefit of increased robustness against wear and the option to re-use the device. Based on our findings, we derive design recommendations for epidermal devices that combine tactile perception with device robustness.

Single-hand microgestures have been recognized for their potential to support direct and subtle interactions. While pioneering work has investigated sensing techniques and presented first sets of intuitive gestures, we still lack a systematic understanding of the complex relationship between microgestures and various types of grasps. This paper presents results from a user elicitation study of microgestures that are performed while the user is holding an object. We present an analysis of over 2,400 microgestures performed by 20 participants, using six different types of grasp and a total of 12 representative handheld objects of varied geometries and size. We expand the existing elicitation method by proposing statistical clustering on the elicited gestures. We contribute detailed results on how grasps and object geometries affect single-hand microgestures, preferred locations, and fingers used. We also present consolidated gesture sets for different grasps and object size. From our findings, we derive recommendations for the design of microgestures compatible with a large variety of handheld objects.

Rapid prototyping of haptic output on 3D objects promises to enable a more widespread use of the tactile channel for ubiquitous, tangible, and wearable computing. Existing prototyping approaches, however, have limited tactile output capabilities, require advanced skills for design and fabrication, or are incompatible with curved object geometries. In this paper, we present a novel digital fabrication approach for printing custom, high-resolution controls for electro-tactile output with integrated touch sensing on interactive objects. It supports curved geometries of everyday objects. We contribute a design tool for modeling, testing, and refining tactile input and output at a high level of abstraction, based on parameterized electro-tactile controls. We further contribute an inventory of 10 parametric Tactlet controls that integrate sensing of user input with real-time electro-tactile feedback. We present two approaches for printing Tactlets on 3D objects, using conductive inkjet printing or FDM 3D printing. Empirical results from a psychophysical study and findings from two practical application cases confirm the functionality and practical feasibility of the Tactlets approach.

Despite the increasing popularity of soft interactive devices, their fabrication remains complex and time consuming. We contribute a process for rapid do-it-yourself fabrication of soft circuits using a conventional desktop inkjet printer. It supports inkjet printing of circuits that are stretchable, ultrathin, high resolution, and integrated with a wide variety of materials used for prototyping. We introduce multi-ink functional printing on a desktop printer for realizing multi-material devices, including conductive and isolating inks. We further present DIY techniques to enhance compatibility between inks and substrates and the circuits' elasticity. This enables circuits on a wide set of materials including temporary tattoo paper, textiles, and thermoplastic. Four application cases demonstrate versatile uses for realizing stretchable devices, e-textiles, body-based and re-shapeable interfaces.

Mutual capacitance-based multi-touch sensing is now a ubiquitous and high-fidelity input technology. However, due to the complexity of electrical and signal processing requirements, it remains very challenging to create interface prototypes with custom-designed multi-touch input surfaces. In this paper, we introduce Multi-Touch Kit, a technique enabling electronics novices to rapidly prototype customized capacitive multi-touch sensors. In contrast to existing techniques, it works with a commodity microcontroller and open-source software and does not require any specialized hardware. Evaluation results show that our approach enables multi-touch sensors with a high spatial and temporal resolution and can accurately detect multiple simultaneous touches. A set of application examples demonstrates the versatile uses of our approach for sensors of different scales, curvature, and materials.

In this paper, we propose and investigate a new text entry technique using micro thumb-tip gestures. Our technique features a miniature QWERTY keyboard residing invisibly on the first segment of the user's index finger. Text entry can be carried out using the thumb-tip to tap the tip of the index finger. The keyboard layout was optimized for eyes-free input by utilizing a spatial model reflecting the users' natural spatial awareness of key locations on the index finger. We present our approach of designing and optimizing the keyboard layout through a series of user studies and computer simulated text entry tests over 1,146,484 possibilities in the design space. The outcome is a 2×3 grid with the letters highly confining to the alphabetic and spatial arrangement of QWERTY. Our user evaluation showed that participants achieved an average text entry speed of 11.9 WPM and were able to type as fast as 13.3 WPM towards the end of the experiment.

Shape-changing interfaces have emerged as a new method for interacting with computers, using dynamic changes in a device's physical shape for input and output. With the advances of research into shape-changing interfaces, we see a need to synthesize the main, open research questions. The purpose of this synthesis is to formulate common challenges across the diverse fields engaged in shape-change research, to facilitate progression from single prototypes and individual design explorations to grander scientific goals, and to draw attention to challenges that come with maturity, including those concerning ethics, theory-building, and societal impact. In this article we therefore present 12 grand challenges for research on shape-changing interfaces, derived from a three-day workshop with 25 shape-changing interface experts with backgrounds in design, computer science, human-computer interaction, engineering, robotics, and material science.

Skin-based touch input opens up new opportunities for direct, subtle, and expressive interaction. However, existing skin-worn sensors are restricted to single-touch input and limited by a low resolution. We present the first skin overlay that can capture high-resolution multi-touch input. Our main contributions are: 1) Based on an exploration of functional materials, we present a fabrication approach for printing thin and flexible multi-touch sensors for on-skin interactions. 2) We present the first non-rectangular multi-touch sensor overlay for use on skin and introduce a design tool that generates such sensors in custom shapes and sizes. 3) To validate the feasibility and versatility of our approach, we present four application examples and empirical results from two technical evaluations. They confirm that the sensor achieves a high signal-to-noise ratio on the body under various grounding conditions and has a high spatial accuracy even when subjected to strong deformations.

Single-hand thumb-to-finger microgestures have shown great promise for expressive, fast and direct interactions. However, pioneering gesture recognition systems each focused on a particular subset of gestures. We are still in lack of systems that can detect the set of possible gestures to a fuller extent. In this paper, we present a consolidated design space for thumb-to-finger microgestures. Based on this design space, we present a thumb-to-finger gesture recognition system using depth sensing and convolutional neural networks. It is the first system that accurately detects the touch points between fingers as well as the finger flexion. As a result, it can detect a broader set of gestures than the existing alternatives, while also providing high-resolution information about the contact points. The system shows an average accuracy of 91% for the real-time detection of 8 demanding thumb-to-finger gesture classes. We demonstrate the potential of this technology via a set of example applications.

This paper introduces Tacttoo, a feel-through interface for electro-tactile output on the user's skin. Integrated in a temporary tattoo with a thin and conformal form factor, it can be applied on complex body geometries, including the fingertip, and is scalable to various body locations. At less than 35µm in thickness, it is the thinnest tactile interface for wearable computing to date. Our results show that Tacttoo retains the natural tactile acuity similar to bare skin while delivering high-density tactile output. We present the fabrication of customized Tacttoo tattoos using DIY tools and contribute a mechanism for consistent electro-tactile operation on the skin. Moreover, we explore new interactive scenarios that are enabled by Tacttoo. Applications in tactile augmented reality and on-skin interaction benefit from a seamless augmentation of real-world tactile cues with computer-generated stimuli. Applications in virtual reality and private notifications benefit from high-density output in an ergonomic form factor. Results from two psychophysical studies and a technical evaluation demonstrate Tacttoo's functionality, feel-through properties and durability.

We introduce Flexibles: 3D-printed flexible tangibles that are deformation-aware and operate on capacitive touchscreens. Flexibles add expressive deformation input to interaction with on-screen tangibles. Based on different types of deformation mapping, we contribute a set of 3D-printable mechanisms that capture pressing, squeezing, and bending input with multiple levels of intensities. They can be integrated into 3D printed objects with custom geometries and on different locations. A Flexible is printed in a single pass on a consumer-level 3D printer without requiring further assembly. Through a series of interactive prototypes, example applications and a technical evaluation, we show the technical feasibility and the wide applicability of Flexibles.

The body provides many recognizable landmarks due to the underlying skeletal structure and variations in skin texture, elasticity, and color. The visual and spatial cues of such body landmarks can help in localizing on-body interfaces, guide input on the body, and allow for easy recall of mappings. Our main contribution are SkinMarks, novel skin-worn I/O devices for precisely localized input and output on fine body landmarks. SkinMarks comprise skin electronics on temporary rub-on tattoos. They conform to fine wrinkles and are compatible with strongly curved and elastic body locations. We identify five types of body landmarks and demonstrate novel interaction techniques that leverage SkinMarks' unique touch, squeeze and bend sensing with integrated visual output. Finally, we detail on the conformality and evaluate sub-millimeter electrodes for touch sensing. Taken together, SkinMarks expands the on-body interaction space to more detailed, highly curved and challenging areas on the body.

Recent advances in printed electronics have enabled the design and fabrication of thin, flexible and customizable interactive surfaces. These interfaces create opportunities for a variety of novel interactions leveraging on the unique form factor, flexibility and customization options. Previous research has demonstrated the possibility of customizable multi-touch sensors, conformal on-skin interfaces and printable shape changing displays. The aim of this tutorial is to acquire basic conceptual and practical skills in developing interactive surfaces with printed electronics.

We contribute a new technique for fabricating highly customized 1D and 2D flex sensing surfaces on thin and flexible substrates. It enables designers and makers to easily, quickly and inexpensively realize thin physical objects in custom shapes with an embedded deformation sensor. The deformation sensor is digitally designed and then fabricated with a single layer of conductive material in a single pass, using an off-the-shelf inkjet printer. We establish a design space and investigate how to realize flex sensing surfaces of highly varied geometries. In a technical evaluation, we demonstrate the technical feasibility of such sensors and investigate their response. Lastly, we demonstrate the practical applicability for tangible interfaces by presenting five example applications.

Over the last decades, there have been numerous efforts in wearable computing research to enable interactive textiles. Most work focus, however, on integrating sensors for planar touch gestures, and thus do not fully take advantage of the flexible, deformable and tangible material properties of textile. In this work, we introduce SmartSleeve, a deformable textile sensor, which can sense both surface and deformation gestures in real-time. It expands the gesture vocabulary with a range of expressive interaction techniques, and we explore new opportunities using advanced deformation gestures, such as, Twirl, Twist, Fold, Push and Stretch. We describe our sensor design, hardware implementation and its novel non-rigid connector architecture. We provide a detailed description of our hybrid gesture detection pipeline that uses learning-based algorithms and heuristics to enable real-time gesture detection and tracking. Its modular architecture allows us to derive new gestures through the combination with continuous properties like pressure, location, and direction. Finally, we report on the promising results from our evaluations which demonstrate real-time classification.

While 3D printing offers great design flexibility before the object is printed, it is very hard for end-users to customize a 3D-printed object to their specific needs after it is printed. We propose HotFlex: a new approach allowing precisely located parts of a 3D object to transition on demand from a solid into a deformable state and back. This approach enables intuitive hands-on remodeling, personalization, and customization of a 3D object after it is printed. We introduce the approach and present an implementation based on computer-controlled printed heating elements that are embedded within the 3D object. We present a set of functional patterns that act as building blocks and enable various forms of hands-on customization. Furthermore, we demonstrate how to integrate sensing of user input and visual output. A series of technical experiments and various application examples demonstrate the practical feasibility of the approach.

Although different types of touch surfaces have gained extensive attention in HCI, this is the first work to directly compare them for two critical factors: performance and ergonomics. Our data come from a pointing task (N=40) carried out on five common touch surface types: public display (large, vertical, standing), tabletop (large, horizontal, seated), laptop (medium, adjustably tilted, seated), tablet (seated, in hand), and smartphone (single- and two-handed input). Ergonomics indices were calculated from biomechanical simulations of motion capture data combined with recordings of external forces. We provide an extensive dataset for researchers and report the first analyses of similarities and differences that are attributable to the different postures and movement ranges.

We propose iSkin, a novel class of skin-worn sensors for touch input on the body. iSkin is a very thin sensor overlay, made of biocompatible materials, and is flexible and stretchable. It can be produced in different shapes and sizes to suit various locations of the body such as the finger, forearm, or ear. Integrating capacitive and resistive touch sensing, the sensor is capable of detecting touch input with two levels of pressure, even when stretched by 30% or when bent with a radius of 0.5cm. Furthermore, iSkin supports single or multiple touch areas of custom shape and arrangement, as well as more complex widgets, such as sliders and click wheels. Recognizing the social importance of skin, we show visual design patterns to customize functional touch sensors and allow for a visually aesthetic appearance. Taken together, these contributions enable new types of on-body devices. This includes finger-worn devices, extensions to conventional wearable devices, and touch input stickers, all fostering direct, quick, and discreet input for mobile computing.

Foldios are foldable interactive objects with embedded input sensing and output capabilities. Foldios combine the advantages of folding for thin, lightweight and shape-changing objects with the strengths of thin-film printed electronics for embedded sensing and output. To enable designers and end-users to create highly custom interactive foldable objects, we contribute a new design and fabrication approach. It makes it possible to design the foldable object in a standard 3D environment and to easily add interactive high-level controls, eliminating the need to manually design a fold pattern and low-level circuits for printed electronics. Second, we contribute a set of printable user interface controls for touch input and display output on folded objects. Moreover, we contribute controls for sensing and actuation of shape-changeable objects. We demonstrate the versatility of the approach with a variety of interactive objects that have been fabricated with this framework.

3D printing is widely used to physically prototype the look and feel of 3D objects. Interaction possibilities of these prototypes, however, are often limited to mechanical parts or post-assembled electronics. In this paper, we present Capricate, a fabrication pipeline that enables users to easily design and 3D print highly customized objects that feature embedded capacitive multi-touch sensing. The object is printed in a single pass using a commodity multi-material 3D printer. To enable touch input on a wide variety of 3D printable surfaces, we contribute two techniques for designing and printing embedded sensors of custom shape. The fabrication pipeline is technically validated by a series of experiments and practically validated by a set of example applications. They demonstrate the wide applicability of Capricate for interactive objects.

We present a multimodal on-surface and near-surface sensing technique for planar, curved and flexible surfaces. Our technique leverages temporal multiplexing of signals coming from a universal interdigitated electrode design, which is printed as a single conductive layer on a flexible substrate. It supports sensing of touch and proximity input, and moreover is capable of capturing several levels of pressure and flexing. We leverage recent developments in conductive inkjet printing as a way to prototype electrode patterns, and combine this with our hardware module for supporting the full range of sensing methods. As the technique is low-cost and easy to implement, it is particularly well-suited for prototyping touch- and hover-based user interfaces, including curved and deformable ones.

We contribute Permulin, an integrated set of interaction and visualization techniques for multi-view tabletops to support co-located collaboration across a wide variety of collaborative coupling styles. These techniques (1) provide support both for group work and for individual work, as well as for the transitions in-between, (2) contribute sharing and peeking techniques to support mutual awareness and group coordination during phases of individual work, (3) reduce interference during group work on a group view, and (4) directly integrate with conventional multi-touch input. We illustrate our techniques in a proof-of-concept implementation with the two example applications of map navigation and photo collages. Results from two user studies demonstrate that Permulin supports fluent transitions between individual and group work and exhibits unique awareness properties that allow participants to be highly aware of each other during tightly coupled collaboration, while being able to unobtrusively perform individual work during loosely coupled collaboration.

This paper contributes results from an empirical study of on-skin input, an emerging technique for controlling mobile devices. Skin is fundamentally different from off-body touch surfaces, opening up a new and largely unexplored interaction space. We investigate characteristics of the various skin-specific input modalities, analyze what kinds of gestures are performed on skin, and study what are preferred input locations. Our main findings show that (1) users intuitively leverage the properties of skin for a wide range of more expressive commands than on conventional touch surfaces; (2) established multi-touch gestures can be transferred to on-skin input; (3) physically uncomfortable modalities are deliberately used for irreversible commands and expressing negative emotions; and (4) the forearm and the hand are the most preferred locations on the upper limb for on-skin input. We detail on users' mental models and contribute a first consolidated set of on-skin gestures. Our findings provide guidance for developers of future sensors as well as for designers of future applications of on-skin input.

This works introduces layer jamming as an enabling technology for designing deformable, stiffness-tunable, thin sheet interfaces. Interfaces that exhibit tunable stiffness properties can yield dynamic haptic feedback and shape deformation capabilities. In comparison to the particle jamming, layer jamming allows for constructing thin and lightweight form factors of an interface. We propose five layer structure designs and an approach which composites multiple materials to control the deformability of the interfaces. We also present methods to embed different types of sensing and pneumatic actuation layers on the layer-jamming unit. Through three application prototypes we demonstrate the benefits of using layer jamming in interface design. Finally, we provide a survey of materials that have proven successful for layer jamming.

PrintScreen is an enabling technology for digital fabrication of customized flexible displays using thin-film electroluminescence (TFEL). It enables inexpensive and rapid fabrication of highly customized displays in low volume, in a simple lab environment, print shop or even at home. We show how to print ultra-thin (120 µm) segmented and passive matrix displays in greyscale or multi-color on a variety of deformable and rigid substrate materials, including PET film, office paper, leather, metal, stone, and wood. The displays can have custom, unconventional 2D shapes and can be bent, rolled and folded to create 3D shapes. We contribute a systematic overview of graphical display primitives for customized displays and show how to integrate them with static print and printed electronics. Furthermore, we contribute a sensing framework, which leverages the display itself for touch sensing. To demonstrate the wide applicability of PrintScreen, we present application examples from ubiquitous, mobile and wearable computing.

Flexpad is an interactive system that combines a depth camera and a projector to transform sheets of plain paper or foam into flexible, highly deformable, and spatially aware handheld displays. We present a novel approach for tracking deformed surfaces from depth images in real time. It captures deformations in high detail, is very robust to occlusions created by the user's hands and fingers, and does not require any kind of markers or visible texture. As a result, the display is considerably more deformable than in previous work on flexible handheld displays, enabling novel applications that leverage the high expressiveness of detailed deformation. We illustrate these unique capabilities through three application examples: curved cross-cuts in volumetric images, deforming virtual paper characters, and slicing through time in videos. Results from two user studies show that our system is capable of detecting complex deformations and that users are able to perform them quickly and precisely.

Researchers have developed interaction concepts based on mobile projectors. Yet pursuing work in this area - particularly in building projector-based interactions techniques within an application - is cumbersome and time-consuming. To mitigate this problem, we contribute ProjectorKit, a flexible open-source toolkit that eases rapid prototyping mobile projector interaction techniques.

We propose cutting as a novel paradigm for ad-hoc customization of printed electronic components. As a first instantiation, we contribute a printed capacitive multi-touch sensor, which can be cut by the end-user to modify its size and shape. This very direct manipulation allows the end-user to easily make real-world objects and surfaces touch-interactive, to augment physical prototypes and to enhance paper craft. We contribute a set of technical principles for the design of printable circuitry that makes the sensor more robust against cuts, damages and removed areas. This includes novel physical topologies and printed forward error correction. A technical evaluation compares different topologies and shows that the sensor remains functional when cut to a different shape.

Sifting and sense-making of video collections are important tasks in many professions. In contrast to sense-making of paper documents, where physical structuring of many documents has proven to be key to effective work, interaction with video is still restricted to the traditional "one video at a time" paradigm. This paper investigates how interaction with video can benefit from paper-like displays that allow for working with multiple videos simultaneously in physical space. We present a corresponding approach and system called PaperVideo, including novel interaction concepts for both video and audio. These include spatial techniques for temporal navigation, arranging, grouping and linking of videos, as well as for managing video contents and simultaneous audio playback on multiple displays. An evaluation with users provides insights into how paper-based navigation with videos improves active video work.

Pico projectors have lately been investigated as mobile display and interaction devices. We propose to use them as 'light beams': Everyday objects sojourning in a beam are turned into dedicated projection surfaces and tangible interaction devices. This way, our daily surroundings get populated with interactive objects, each one temporarily chartered with a dedicated sub-issue of pervasive interaction. While interaction with objects has been studied in larger, immersive projection spaces, the affordances of pico projections are fundamentally different: they have a very small, strictly limited field of projection, and they are mobile. This paper contributes the results of an exploratory field study on how people interact with everyday objects in pico projections in nomadic settings. Based upon these results, we present novel interaction techniques that leverage the limited field of projection and trade-off between digitally augmented and traditional uses of everyday objects.

In this paper, we present a novel device concept that features double-sided displays which can be folded using predefined hinges. The device concept enables users to dynamically alter both size and shape of the display and also to access the backside using fold gestures. We explore the design of such devices by investigating different types and forms of folding. Furthermore, we propose a set of interaction principles and techniques. Following a user-centered design process, we evaluate our device concept in two sessions with low-fidelity and high-fidelity prototypes.

We present a device concept and a prototype of a future mobile device. By featuring a rollable display, its display size and its form factor can be dynamically changed. Moreover, we investigate how physical resizing of the display can be used as an input technique for interacting with digital contents and present a set of novel interaction techniques. Evaluation results show that physical resizing of the display can improve the way we interact with digital contents on mobile devices.

Increasingly powerful mobile devices like the Apple iPhone empower learners to watch e-lectures not only at home but also in mobile learning scenarios virtually anywhere and anytime. However, state of the art mobile video browsers do not support learners in getting an overview on and navigating between the large amounts of semantically related e-lectures, which are available in various digital libraries. We contribute a novel user interface for the mobile use of e-lectures. Leveraging a spatial navigation metaphor, it supports both linear and nonlinear interaction within a single lecture, as well as the efficient navigation within large e-lecture libraries. Evaluation results show that our e-lecture browser significantly improves the learning process and leads to significantly higher efficiency and user satisfaction.

Increasingly powerful mobile devices enable users to access and watch videos in mobile settings. While some concepts for mobile video browsing have been presented, the field still lacks a general understanding of the design space and of the characteristics of interaction concepts. In order to improve user interfaces for mobile video browsing, this paper includes three contributions. First, we setup a design space for mobile video browsing and contribute seven novel interface concepts. They rely on GUI-based, on touch-gesture-based, and on physical interaction. Second, we present the results of an in-depth evaluation and comparison of these concepts. They are based on an ascriptive analysis of 18 hours of video observations from a controlled experiment with 44 participants. The results provide insights into common usability errors and misconceptions. Third, we derive implications for the design of mobile video browsers to minimize errors and to increase usability.

Despite numerous predictions of the paperless office, knowledge work is still characterized by the combined use of paper and digital documents. Digital pen-and-paper user interfaces bridge the gap between both worlds by electronically capturing the interactions of a user with a pen on real paper. The contribution of this paper is two-fold: First, we introduce an interaction framework for pen-and-paper user interfaces consisting of six core interactions. This helps both in analyzing existing work practices and interfaces and in guiding the design of interfaces which offer complex functionality and nevertheless remain simple to use. Second, we apply this framework and contribute three novel pen-and-paper interaction strategies for creating hyperlinks between printed and digital documents and for tagging both types of documents.

In a study of notetaking in university courses, we found that the large majority of students prefer the paper medium to the computer for taking notes and making annotations. Based on this finding, we developed CoScribe, a system which supports students in making collaborative handwritten annotations on printed lecture slides. It includes mechanisms for the paper-based sharing and semantic tagging of annotations and slides. Moreover, we present a novel visualization of shared handwritten annotations providing an integrated access to annotations from all learners. A user study indicates that the system efficiently supports student annotation and can be easily integrated into current annotation practice.

Structuring constitutes an important means for enhancing individual and collaborative learning processes. This paper introduces two new concepts for augmented semantic structuring of paper-based learning processes. Digital Paper Bookmarks can be used for individually structuring, indexing and tagging learning documents. The Learner Cube provides a new structuring method for collaborative face-to-face learning meetings. Both concepts rely on the Anoto technology, permitting to track and digitalize the paper-based learning process. A first pilot study indicates that Digital Paper Bookmarks and tangible interaction are promising tools for structuring learning processes.

Typical tasks of learning groups and knowledge workers include working both with printed and digital documents. We present a pen-based and tangible interaction concept for the linking and tagging of documents in a mixed physical and digital environment. The interaction with printed and electronic documents is unified. We therefore use digital pens, which equally capture physical handwriting on real paper and act as an input device on a specific screen prototype. Links and tags can have variable scopes ranging from small document passages to a collection of several documents. Physical folders along with camera-based marker tracking provide for the tangible definition of document collections. Tags are intuitively defined on a paper tag menu card. We present two visualizations of links and tags (a document-centered viewer and a hyperstructure-centered graph view). These are closely coupled with the physical environment.