Human-scale inflatable robots that respond to users’ movements offer a unique opportunity to explore embodied and relational interaction. Yet, little is known about how their form and interactive behaviour shape engagement and perception. This study presents three inflatable robots designed to explore different levels of user control: Barrel affords high user freedom, allowing approach or retreat; Chair introduces partial constraint through a semi-enclosing structure; Jacket envelops the user, with movement largely determined by the robot. Across all systems, robot motion responds to user behaviour in semi-random ways, contributing to a sense of agency. Participants engaged through a think-aloud protocol, followed by semi-structured interviews and a drawing activity. Findings show that, in the absence of clear prior models, participants actively constructed interpretations of behaviour, attributing agency and intention, while variations in control shaped trust and relational dynamics. These results offer insights for designing human-scale soft robots that support rich, embodied interaction.

As robots are increasingly designed to take on relational qualities, new methods are needed to explore how their form, movement, and presence shape connection. We introduce puppetry as an embodied design practice in which designers and users inhabit and control physical prototypes to examine relational qualities before technical implementation. We ground this in equine-assisted interventions, where horses act as relational partners: their unpredictability, reciprocity, and embodied risk require humans to attune and regulate emotions. We utilised puppeteering to re-enact these dynamics safely through movement, materiality, and interaction, surfacing how such qualities may inform relational robots. Our findings show that puppeteering functions as a generative practice: movement becomes inquiry, playful performance reveals affective states, and low-fidelity prototypes enable safe exploration of relational complexity. Puppetry thus offers a pathway for developing robots as autonomous, equal partners; revealing insights into agency, co-embodiment, and materiality, often overlooked in traditional prototyping.

Recent advances in material-centric personal fabrication have enabled the use of custom inks and paints with functional properties to create free-form displays. However, working with these materials, such as through airbrushing, remains a skill-intensive process that limits non-specialist access, adoption, and further development. To address these challenges, we developed Dispray, an AR-augmented airbrush tool for fabricating electroluminescent displays. The tool was shaped by insights from a study with artists and engineers and refined through iterative prototyping. We evaluated Dispray by re-engaging with the original artists and engineers and conducting a follow-up study with novice users, demonstrating its effectiveness in supporting both skill acquisition and functional fabrication. This work contributes a novel approach to connecting emerging material fabrication with accessible, user-centered tools, advancing the democratisation of interactive device creation.

Shape-changing interfaces (SCIs) dynamically alter their form, an inherent characteristic that introduces fragility into their design. As a result, users’ perceptions of an interface’s fragility or its potential to move or break may influence their interaction, however the extent of this effect is unclear. To address this gap, we conducted a qualitative study (N = 18) using video stimuli showcasing 20 existing SCIs. Through thematic analysis, we identified key factors impacting perceived fragility and formalized these into a framework. We then conducted a second study (N = 36) for which we fabricated SCIs that varied across selected fragility-related dimensions. We recorded user interactions and compared how the selected dimensions shaped manipulation of the objects and how they were considered by users. Together, these studies provide a structured foundational understanding of perceived fragility in SCIs and offer insights to enhance perceived robustness and inform future SCI development.

Can J.J. Gibson’s concept of affordances be empirically examined using screen-based technology? We show how screen-based affordances can be examined through the use case of perceptual toughness, i.e. the break-ability of a virtual object. We present two user experiments (n=72, n=66) examining break-ability through a novel ’Perceptual Impact Testing’ methodology and online screen-based 3D virtual environment. We show that judgements of break-ability are systematically distorted when a perceiver’s virtual ‘Point of Observation’ or virtual environment’s ‘Horizonal Geometry’ are manipulated. These statistically significant results provide evidence that: 1) direct perception can account for perceptual distortions of break-ability; 2) Gibsonian affordances can be empirically examined through screen-based interactions.

Equine-Assisted Interventions (EAIs) aim to improve participant health and well-being through the development of a therapeutic relationship with a trained horse. These interventions leverage the horse’s ability to provide emotional feedback, as it responds to negative non-verbal cues with reciprocal negativity, thereby encouraging participants to regulate their emotions and achieve attunement with the horse. Despite their benefits, EAIs face significant challenges, including logistical, financial, and resource constraints, which hinder their widespread adoption and accessibility. To address these issues, we conducted an autoethnographic study of the lead researcher’s engagement in an EAI to investigate the underlying mechanisms and explore potential technological alternatives. Our findings suggest that the reciprocal and responsive non-verbal communication, combined with the horse’s considerable physical presence, supports the potential of an embodied robotic system as a viable alternative. Such a system could offer a scalable and sustainable solution to the current limitations of EAIs.

Multifractality is a property of biological signals that is informative about the effectiveness of behavioural coordination. Since fatigue is known to undermine the ability to coordinate behavioural resources, we hypothesise that multifractal signatures may function as effective markers of fatigue. To further this line of research we conduct a study to investigate the association between fatigue and multifractal signatures during typing. We observe that multifractality in keystroke timings is significantly lower after exposure to a mentally fatiguing task, and that multifractal properties correlate with the degree of self-reported fatigue. We discuss the implications of this research for the development of unobtrusive, low cost fatigue monitoring and interventions, noting that multifractal analysis requires only the capture of input behaviour for the task, and that multifractality has been observed in a range of task-directed behaviours, not only typing. This points to the potential to infer fatigue via existing commodity input hardware, in a range of tasks.

Bio-based and bio-degradable materials have shown promising results for sustainable Human-Computer Interaction (HCI) applications, including shape-changing interfaces. However, the diversity of shape-changing behaviors achievable with these materials remains unclear as the fabrication knowledge is scattered across multiple research fields. This paper introduces SoftBioMorph, a fabrication framework that aims to integrate the fabrication know-how of sustainable soft shape-changing interfaces with biopolymers. Based on the example of Sodium Alginate, the framework contributes (1) a set of material synthesis processes that modify the biopolymer’s properties to fulfill different functions; (2) a set of DIY crafting-based assembling techniques that functionalize the material and assembling properties to achieve three primitive types of change in shape; and (3) a series of application cases that demonstrate the versatility of the framework. We further discuss limitations, research questions, and fabrication challenges, presenting a comprehensive approach to sustainable prototyping in HCI.

Abstract: We investigate how deceptive patterns (sludge) within online grocery shopping can influence the purchase of alcohol through design intervention, and how to counter them. Previous research investigated online shoppers’ purchasing behaviors in sustainability and healthy eating. However, current research in alcohol is limited to modifying simulated platforms to aid in the increase of purchasing lower alcoholic beverages by altering product offerings. We conducted a heuristic evaluation on online shopping platforms highlighting the use of sludge, before developing five design intervention prototypes. We then iterated on these interventions through an online alcohol purchasing questionnaire (N=20) and two follow-up activities (N=11) (interview with design probes; product swap questionnaire), evaluating how the interventions could counter sludge. Our goal is to develop interventions that engage light to moderate drinkers in alcohol reduction. We found participants want a greater presence of alcohol units and product grading imagery in conjunction with neutral-toned health warnings.

Over recent years, there has been significant research within HCI towards free-form physical interactive devices. However, such devices are not straightforward to design, produce and deploy on demand. Traditional development revolves around iterative prototyping through component-based assembly, limiting device structure and implementation. Material-centric personal display fabrication (DisplayFab) opens the possibility of decentralised, configurable production by low-skill makers. Currently, DisplayFab is severely limited by its embryonic stage of development, the complexity of involved processes and materials, and the challenges around designing interactive structures. We present a development framework to provide a path for future research. DisplayFab has been developed by identifying 4 key breakpoints in the existing “Personal Fabrication” framework: Material and Deposition, Conception and Software, Feedback and Interactivity and Responsible Innovation. We use these breakpoints to form a targeted literature review of relevant work. Doing this we identify 30 challenges that act as roadmap for future research in DisplayFab.

We extend grasp models to encompass one-handed interaction with arbitrary shaped touchscreen devices. Current models focus on how objects are stably held by external forces. However, with touchscreen devices, we postulate that users do a trade-off between holding securely and exploring interactively. To verify this, we first conducted a qualitative study which asked participants to grasp 3D printed objects while considering its different interactivity. Results of the study confirm our hypothesis and reveal obvious change in postures. To further verify this trade-off and design interactions, we developed a simulation software capable of computing the stability of a grasp and its reachability. We conducted the second study based on the observed predominant grasps to validate our software with a glove. Results also confirm a consistent trade-off between stability and reachability. We conclude by discussing how this research can help designing computational tools focusing on hand-held interactions with arbitrary shaped touchscreen devices.

Large-scale shape-changing interfaces (SCIs) such as shape-changing walls offer opportunities for enhancing user experiences within buildings, e.g., for navigation. However, due to the embryonic nature of SCI technologies, designing and explaining the shape features that are beneficial to users is challenging. Previous work used virtual platforms (2D video or Projected Augmented Reality) to design SCI. This paper explores how Virtual Reality (VR) can provide an immersive experience that can help in designing large-scale SCI. We follow a co-design approach in which we use VR to obtain users’ impressions of shape-changing walls. Then, we conduct co-design sessions to understand how shape-changing walls can be designed to become ambient and blend with the environment. We report our results to guide the design of shape-changing walls as well as discuss how our approach can provide valuable insights into how a VR experience, prior to design, and can help in the design process.

Human agency and autonomy have always been fundamental concepts in HCI. New developments, including ubiquitous AI and the growing integration of technologies into our lives, make these issues ever pressing, as technologies increase their ability to influence our behaviours and values. However, in HCI understandings of autonomy and agency remain ambiguous. Both concepts are used to describe a wide range of phenomena pertaining to sense-of-control, material independence, and identity. It is unclear to what degree these understandings are compatible, and how they support the development of research programs and practical interventions. We address this by reviewing 30 years of HCI research on autonomy and agency to identify current understandings, open issues, and future directions. From this analysis, we identify ethical issues, and outline key themes to guide future work. We also articulate avenues for advancing clarity and specificity around these concepts, and for coordinating integrative work across different HCI communities.

The philosophical construct readiness-to-hand describes focused, intuitive, tool use, and has been linked to tool-embodiment and immersion. The construct has been influential in HCI and design for decades, but researchers currently lack appropriate measures and tools to investigate it empirically. To support such empirical work we investigate the possibility of operationalising readiness-to-hand in measurements of multfractality in movement, building on recent work in cognitive science. We conduct two experiments (N=44, N=30) investigating multifractality in mouse movements during a computer game, replicating prior results and contributing new findings. Our results show that multifractality correlates with dimensions associated with readiness-to-hand, including skill and task-engagement, during tool breakdown, task learning and normal play. We describe future possibilities for the application of these methods in HCI, supporting such work by sharing scripts and data (https://osf.io/2hm9u/), and introducing a new data-driven approach to parameter selection.

Abstract: FabricatINK explores the personal fabrication of irregularly-shaped low-power displays using electronic ink (E ink). E ink is a programmable bicolour material used in traditional form-factors such as E readers. It has potential for more versatile use within the scope of personal fabrication of custom-shaped displays, and it has the promise to be the pre-eminent material choice for this purpose. We appraise technical literature to identify properties of E ink, suited to fabrication. We identify a key roadblock, universal access to E ink as a material, and we deliver a method to circumvent this by upcycling broken electronics. We subsequently present a novel fabrication method for irregularly-shaped E ink displays. We demonstrate our fabrication process and E ink’s versatility through ten prototypes showing different applications and use cases. By addressing E ink as a material for display fabrication, we uncover the potential for users to create custom-shaped truly bistable displays.

Advanced technologies are increasingly enabling the creation of interactive devices with non-rectangular form-factors but it is currently unclear what alternative form-factors are desirable for end-users. We contribute an understanding of the interplay between the rationale for the form factors of such devices and their interactive content through think-aloud design sessions in which participants could mold devices as they wished using clay. We analysed their qualitative reflections on how the shapes affected interaction. Using thematic analysis, we identified shape features desirable on handheld freeform devices and discuss the particularity of three themes central to the choice of form factors: freeform dexterity, shape features discoverability and shape adaptability (to the task and context). In a second study following the same experimental set-up, we focused on the trade off between dexterity and discoverability and the relation to the concept of affordance. Our work reveals the shape features that impact the most the choice of grasps on freeform devices from which we derive design guidelines for the design of such devices.

One of the most fundamental interactions –pointing– is well understood on flat surfaces. However, pointing performance on tangible surfaces with physical targets is still limited for Tangible User Interfaces (TUIs). We investigate the effect of a target’s physical width, height, and distance on user pointing performance. We conducted a study using a reciprocal tapping task ( n =19) with physical rods arranged in a circle. We compared our data with five conventional interaction models designed for 2D/3D tasks rather than tangible targets. We show that variance in the movement times was only satisfactorily explained by a model established for volumetric displays ( r 2 =0.954). Analysis shows that movement direction and height should be included as parameters to this model to generalize for 3D tangible targets. Qualitative feedback from participants suggests that pointing at physical targets involves additional human factors (e.g., perception of sharpness or robustness) that need to be investigated further to understand how performance with tangible objects is affected.

Physical Human-Robot-Interaction (pHRI) is beneficial for communication in social interaction or to perform collaborative tasks but is also crucial for safety. While robotic devices embed sensors for this sole purpose, their design often is the results of a trade-off between technical capabilities and rarely considers human factors. We propose a novel approach to design and fabricate compliant Human-like artificial skin sensors for robots, with similar mechanical properties as human skin and capable of precisely detecting touch. Our artificial skin relies on the use of different silicone elastomers to replicate the human skin layers and comprises an embedded electrode matrix to perform mutual capacitance sensing. We present the sensor and describe its fabrication process which is scalable, low-cost and ensures flexibility, compliance and robustness. We introduce Muca, an open-source sensing development board and then evaluate the performance of the sensor.

We investigate how users touch arbitrary shapes. First, we performed semi-structured interviews with a fifteen-shape set as prop to identify touch strategies. Results reveal four main potential touch strategies, from which we devised nine mathematical candidate models. We investigate the ability of these models to predict human behaviour in a controlled experiment. We found that the center of a shape’s bounding box best approximates a user’s target location when touching arbitrary shapes. Our findings not only invite designers to use a larger variety of shapes, but can also be used to design touch interaction adapted to user behaviour using our model. As an example, they are likely to be valuable for the creation of applications exposing shapes of various complexities, like drawing applications.

It is well established that the presence of other people and the physical traces of their activities impact our navigation decisions, e.g. following a path in a park where trails are worn by frequent passage. Here we investigate such effect within a digital setup where social cues are displayed on the floor. We contribute the first lab study (n=24) investigating how social cues, presented as footprints on an interactive floor display, can impact navigation decisions. We particularly explore two task-based scenarios: exploration and search. A direct comparison of path choices did not reveal significant effects in both tasks but a further qualitative analysis showed that social cues affected path choices depending on how participants interpreted them. Additionally, results revealed a pattern of direct decision-making behaviours when social cues were present. We discuss how these insights can contribute to designing engaging floor interfaces that guide navigation decisions using social cues.

Environmental cues influence our spatial behaviour when we explore unfamiliar spaces. Research particularly shows that the presence/actions of other people affects our navigation decisions. Here we examine how such social information can be integrated digitally into the environment to support navigation in indoor public spaces. We carried out a study (n=12) to explore how to represent traces of navigation behaviour. We compared 6 floor visualisations and examined how they affect participants' navigational choices. Results suggest that direct representations such as footprints are most informative. To investigate further how such visualisation could work in practice, we implemented an interactive floor system and used it as probe during one-to-one design sessions (n=26). We particularly focused on four design challenges: the overall visual representation, representation of multiple people, designing more prominent visualisations and the incorporation of non-identifying information. Our results provide insights for designers looking to develop history-enriched floor interfaces.

The HCI community has a strong and growing interest in shape-changing interfaces (SCIs) that can offer dynamic affordance. In this context, there is an increasing need for HCI researchers and designers to form close relationships with disciplines such as robotics and material science in order to be able to truly harness the state-of-the-art in morphing technologies. To help these synergies arise, we presentMorphino: a card-based toolkit to inspire shape-changing interface designs. Our cards bring together a collection of morphing mechanisms already established in the multidisciplinary literature and illustrate them through familiar examples from nature. We begin by detailing the design of the cards, based on a review of shape-change in nature; then, report on a series of design sessions conducted to demonstrate their usefulness in generating new ideas and in helping end-users gain a better understanding of the possibilities for shape-changing materials.

ProtoSpray is a fabrication method that combines 3D printing and spray coating, to create interactive displays of arbitrary shapes. Our approach makes novel use of 3D printed conductive channels to create base electrodes on 3D shapes. This is then combined with spraying active materials to produce illumination. We demonstrate the feasibility and benefits of this combined approach in 6 evaluations exploring different shaped topologies. We analyze factors such as spray orientations, surface topologies and printer resolutions, to discuss how spray nozzles can be integrated into traditional 3D printers. We present a series of ProtoSprayed objects demonstrating how our technique goes beyond existing fabrication techniques by allowing creation of displays on objects with curvatures as complex as a Mobius strip. Our work provides a platform to empower makers to use displays as a fabrication material.

We present Sprayable User Interfaces: room-sized interactive surfaces that contain sensor and display elements created by airbrushing functional inks. Since airbrushing is inherently mobile, designers can create large-scale user interfaces on complex 3D geometries where existing stationary fabrication methods fail. To enable Sprayable User Interfaces, we developed a novel design and fabrication pipeline that takes a desired user interface layout as input and automatically generates stencils for airbrushing the layout onto a physical surface. After fabricating stencils from cardboard or projecting stencils digitally, designers spray each layer with an airbrush, attach a microcontroller to the user interface, and the interface is ready to be used. Our technical evaluation shows that Sprayable User Interfaces work on various geometries and surface materials, such as porous stone and rough wood. We demonstrate our system with several application examples including interactive smart home applications on a wall and a soft leather sofa, an interactive smart city application, and interactive architecture in public office spaces.

Autogrip is a new thimble that enables force feedback devices to autonomously attach themselves to a finger. Although self-attachment is a simple concept, it has never been explored in the space of force feedback devices where current thimble solutions require complex attachment procedures and often swapping between interchangeable parts. Self-attachment is advantageous in many applications such as: immersive spaces, multi-user, walk up and use contexts, and especially multi-point force feedback systems as it can allow a lone user to quickly attach multiple devices to fingers on both hands - a difficult task with current thimbles. We present the design of our open-source contraption, Autogrip, a one-size-fits-all thimble that retro-fits to existing force feedback devices, enabling them to automatically attach themselves to a fingertip. We demonstrate Autogrip by retrofitting it to a Phantom 1.5 and a 4-finger Mantis system. We report preliminary user-testing results that indicated Autogrip was three times faster to attach than a typical method. We also present further refinements based on user feedback.

We investigate the design of a shape-changing dial, i.e. a dial that can change its circumference and height to adapt to different contexts of interaction. We first explore how users grasp 3D printed dials of different heights and circumferences in order to inform the form factor of shape-changing dials. We then design a prototype, ExpanDial, inspired from morphing origami. We then use our prototype as a probe within design sessions and use the participants' feedback to devise a set of applications that can benefit from such reconfigurable devices. We also used the design sessions to better understand what kind of interaction and manipulation could be harnessed from such device.

Touchscreens are the predominant medium for interactions with digital services; however, their current fixed form factor narrows the scope for rich physical interactions by limiting interaction possibilities to a single, planar surface. In this paper we introduce the concept of PickCells, a fully re-configurable device concept composed of cells, that breaks the mould of rigid screens and explores a modular system that affords rich sets of tangible interactions and novel across-device relationships. Through a series of co-design activities -- involving HCI experts and potential end-users of such systems -- we synthesised a design space aimed at inspiring future research, giving researchers and designers a framework in which to explore modular screen interactions. The design space we propose unifies existing works on modular touch surfaces under a general framework and broadens horizons by opening up unexplored spaces providing new interaction possibilities. In this paper, we present the PickCells concept, a design space of modular touch surfaces, and propose a toolkit for quick scenario prototyping.

With upcoming breakthroughs in free-form display technologies, new user interface design challenges have emerged. Here, we investigate a question, which has been widely explored on traditional GUIs but unexplored on non-rectangular interfaces: what are the user strategies in terms of visual search when information is not presented in a traditional rectangular layout? To achieve this, we present two complementary studies investigating eye movements in different visual search tasks. Our results unveil which areas are seen first according to different visual structures. By doing so we address the question of where to place relevant content for the UI designers of non-rectangular displays.

Mantis is a highly scalable system architecture that democratizes haptic devices by enabling designers to create accurate, multiform and accessible force feedback systems. Mantis uses brushless DC motors, custom electronic controllers, and an admittance control scheme to achieve stable high-quality haptic rendering. It enables common desktop form factors but also: large workspaces (multiple arm lengths), multiple arm workspaces, and mobile workspaces. It also uses accessible components and costs significantly less than typical high-fidelity force feedback solutions which are often confined to research labs. We present our design and show that Mantis can reproduce the haptic fidelity of common robotic arms. We demonstrate its multiform ability by implementing five systems: a single desktop-sized device, a single large workspace device, a large workspace system with four points of feedback, a mobile system and a wearable one.

We propose a paradigm called Skin-On interfaces, in which interactive devices have their own (artificial) skin, thus enabling new forms of input gestures for end-users (e.g. twist, scratch). Our work explores the design space of Skin-On interfaces by following a bio-driven approach: (1) From a sensory point of view, we study how to reproduce the look and feel of the human skin through three user studies;(2) From a gestural point of view, we explore how gestures naturally performed on skin can be transposed to Skin-On interfaces; (3) From a technical point of view, we explore and discuss different ways of fabricating interfaces that mimic human skin sensitivity and can recognize the gestures observed in the previous study; (4) We assemble the insights of our three exploratory facets to implement a series of Skin-On interfaces and we also contribute by providing a toolkit that enables easy reproduction and fabrication.

In this paper we explore how screen-based smartphone interaction can be enriched when designers focus on the physical interaction issues surrounding the device. These consist of the hand grips used (Symmetric bimanual, Asymmetric bimanual with thumb, Single handed, Asymmetric bimanual with finger), body postures (Sitting at a table, Standing, Lying down) and the tilting of the smartphone itself. These physical interactions are well described in the literature and several research papers provide empirical metrics describing them. In this paper, we go one step further by using this data to generate new screen-based interactions. We achieved this by conducting two workshops to investigate how smartphone interaction design can be informed by the physicality of smartphone interaction. By analysing the outcomes, we provide 14 new screen interaction examples with additional insights comparing outcomes for various body postures and grips.

Shape-changing interfaces have emerged as a new method for interacting with computers, using dynamic changes in a device's physical shape for input and output. With the advances of research into shape-changing interfaces, we see a need to synthesize the main, open research questions. The purpose of this synthesis is to formulate common challenges across the diverse fields engaged in shape-change research, to facilitate progression from single prototypes and individual design explorations to grander scientific goals, and to draw attention to challenges that come with maturity, including those concerning ethics, theory-building, and societal impact. In this article we therefore present 12 grand challenges for research on shape-changing interfaces, derived from a three-day workshop with 25 shape-changing interface experts with backgrounds in design, computer science, human-computer interaction, engineering, robotics, and material science.

We present an investigation into how hand usage is affected by different body postures (Sitting at a table, Lying down and Standing) when interacting with smartphones. We theorize a list of factors (smartphone support, body support and muscle usage) and explore their influence the tilt and rotation of the smartphone. From this we draw a list of hypotheses that we investigate in a quantitative study. We varied the body postures and grips (Symmetric bimanual, Asymmetric bimanual finger, Asymmetric bimanual thumb and Single-handed) studying the effects through a dual pointing task. Our results showed that the body posture Lying down had the most movement, followed by Sitting at a table and finally Standing. We additionally generate reports of motions performed using different grips. Our work extends previous research conducted with multiple grips in a sitting position by including other body postures, it is anticipated that UI designers will use our results to inform the development of mobile user interfaces.

Physical controls are widely used by professionals such as sound engineers or aircraft pilots. In particular knobs and sliders are the most prevalent in such interfaces. They have advantages over touchscreen GUIs, especially when users require quick and eyes-free control. However, their interfaces (e.g., mixing consoles) are often bulky and crowded. To improve this, we present the results of a formative study with professionals who use physical controllers. Based on their feedback, we propose design requirements for future interfaces for parameters control. We then introduce the design of our KnobSlider that combines the advantages of a knob and a slider in one unique shape-changing device. A qualitative study with professionals shows how KnobSlider supports the design requirements, and inspired new interactions and applications.

Users interact with many reconfigurable objects in daily life. These objects embed reconfigurations and shape- changing features that users are familiar with. For this reason, everyday reconfigurable objects have informed the design and taxonomy of shape changing UI. However, they have never been explored systematically. In this paper, we present a data set of 82 everyday reconfigurable objects that we collected in a workshop. We discuss how they can inspire the design of reconfigurable interfaces. We particularly focus on taxonomies of reconfigurable interfaces. Taxonomies have been suggested to help design and communication among researchers, however despite their extensive use, taxonomies are rarely evaluated. This paper analyses two established taxonomies - Rasmussen's and Roudaut's - using daily reconfigurable objects. We show relationships between the taxonomies and area for improvements. We propose Morphees+, a refined taxonomy based on Roudaut's Shape Resolution Taxonomy.

With the proliferation of flexible displays and the advances in smart materials, it is now possible to create interactive devices that are not only flexible but can reconfigure into any shape on demand. Several Human Computer Interaction (HCI) and robotics researchers have started designing, prototyping and evaluating shape-changing devices, realising, however, that this vision still requires many engineering challenges to be addressed. On the material science front, we need breakthroughs in stable and accessible materials to create novel, proof-of-concept devices. On the interactive devices side, we require a deeper appreciation for the material properties and an understanding of how exploiting material properties can provide affordances that unleash the human interactive potential. While these challenges are interesting for the respective research fields, we believe that the true power of shape-changing devices can be magnified by bringing together these communities. In this paper we therefore present a review of advances made in shape-changing materials and discuss their applications within an HCI context.

In this paper we present an investigation into how hand usage is affected by different mobile phone form factors. Our initial (qualitative) study explored how users interact with various mobile phone types (touchscreen, physical keyboard and stylus). The analysis of the videos revealed that each type of mobile phone affords specific handgrips and that the user shifts these grips and consequently the tilt and rotation of the phone depending on the context of interaction. In order to further investigate the tilt and rotation effects we conducted a controlled quantitative study in which we varied the size of the phone and the type of grips (Symmetric bimanual, Asymmetric bimanual with finger, Asymmetric bimanual with thumb and Single handed) to better understand how they affect the tilt and rotation during a dual pointing task. The results showed that the size of the phone does have a consequence and that the distance needed to reach action items affects the phones' tilt and rotation. Additionally, we found that the amount of tilt, rotation and reach required corresponded with the participant's grip preference. We finish the paper by discussing the design lessons for mobile UI and proposing design guidelines and applications for these insights.

We present the concept of Frozen Suit, a type of clothing that restricts users' movements at joint positions (e.g. elbow, knee) via a changeable stiffness jamming material. The suit can "freeze" users' body parts, for example during a game in order to provide the physical sensation of being frozen by an enemy. In this paper we first present the Frozen Suit concept and its potential applications. We then systematically investigate how to design jamming patches in order to sufficiently restrict an arm or a leg. In particular we used low-fidelity prototypes to explore the restricting power of different material and particles. In order to push this analysis further we conducted a controlled experiment in order to compare the perceived stiffness of different patches sizes attached to the elbow. We performed a paired comparison experience and used a Bradley-Terry-Luce model to analyze the subjective feedback from participants. We found that 20cm long x 7cm large is the most restrictive patch and that an increase in patch area correlates with an increase in perceived stiffness (quadratic). We finish by presenting a use case application with a game that we implemented where enemies can freeze the player.

Graphical user interfaces are composed of varying elements (text, images, etc.) whose visual arrangement has been relatively well established in the context of rectangular interfaces. The advent of non-rectangular displays questions this knowledge. In this paper we study how traditional content layouts can be adapted to fit different non-rectangular displays. We performed a first qualitative study where graphic designers fitted text and images into different non-rectangular displays. From the analysis of their output we generalize and adapt ten composition principles that have been proposed in the literature for rectangular displays. We evaluate the revised principles through a paired comparison questionnaire where 57 participants compared pairs of layouts. Using the Bradley-Terry-Luce model to analyze our data we show that some results contradict current conventions on visual design for rectangular displays. We then extracted the most interesting cases and conducted a follow up study with additional shapes to investigate how the principles generalize. From these results we propose a set of guidelines for designing visual content for non-rectangular displays.

Emerging technologies allow for the creation of non-rectangular displays with unlimited constraints in shape. However, the introduction of such displays radically deviates from the prevailing tradition of placing content on rectangular screens and raises fundamental design questions. Among these is the foremost question of how to legibly present text. We address this fundamental concern through a multi-part exploration that includes: (1) a focus-group study from which we collected free-form display scenarios and extracted display shape properties; (2) a framework that identifies different mappings of text onto a non-rectangular shape and formulates hypotheses concerning legibility for different display shape properties; and (3) a series of quantitative text legibility studies to assess our hypotheses. Or results agree with and extend upon other findings in the existing literature on text legibility, but they also uncover unique instances in which different rules need to be applied for non-rectangular displays. These results also provide guidelines for the design of visual interfaces.

We introduce Cubimorph, a modular interactive device that accommodates touchscreens on each of the six module faces, and that uses a hinge-mounted turntable mechanism to self-reconfigure in the user's hand. Cubimorph contributes toward the vision of programmable matter where interactive devices reconfigure in any shape that can be made out of a chain of cubes in order to fit a myriad of functionalities, e.g. a mobile phone shifting into a console when a user launches a game. We present a design rationale that exposes user requirements to consider when designing homogeneous modular interactive devices. We present our Cubimorph mechanical design, three prototypes demonstrating key aspects (turntable hinges, embedded touchscreens and miniaturization), and an adaptation of the probabilistic roadmap algorithm for the reconfiguration.

Advances in shape changing assemblies have been made in reconfiguration algorithms, hardware designs and interaction techniques. However no tools exist for guiding designers in building those modular devices and especially for choosing the shape of the units. The task becomes even more complex when the units themselves can change their shapes to animate the entire assembly. In this paper, we contribute with the first analysis tool which helps the designer to both choose the right subset of forms for the units and to create an assembly with maximum accuracy from the set of given objects. We introduce the concept of Changibles that are interactive wireless units that can reshape themselves and be attached together to create an animated assembly. We present a use case to demonstrate the use of our tool, with an instantiation of six Changibles that are used to construct a pulsing heart assembly.

We introduce the term shape resolution, which adds to the existing definitions of screen and touch resolution. We propose a framework, based on a geometric model (Non-Uniform Rational B-splines), which defines a metric for shape resolution in ten features. We illustrate it by comparing the current related work of shape changing devices. We then propose the concept of Morphees that are self-actuated flexible mobile devices adapting their shapes on their own to the context of use in order to offer better affordances. For instance, when a game is launched, the mobile device morphs into a console-like shape by curling two opposite edges to be better grasped with two hands. We then create preliminary prototypes of Morphees in order to explore six different building strategies using advanced shape changing materials (dielectric electro active polymers and shape memory alloys). By comparing the shape resolution of our prototypes, we generate insights to help designers toward creating high shape resolution Morphees.

We propose using spatial gestures not only for input but also for output. Analogous to gesture input, the proposed gesture output moves the user's finger in a gesture, which the user then recognizes. We use our concept in a mobile scenario where a motion path forming a "5" informs users about new emails, or a heart-shaped path serves as a mes- sage from a friend. We built two prototypes: (1) The long- RangeOuija is a stationary prototype that offers a motion range of up to 4cm; (2) The pocketOuija is self-contained mobile device based on an iPhone with up to 1cm motion range. Both devices actuate the user's fingers by means of an actuated transparent foil overlaid onto a touchscreen. We conducted three studies with the longRangeOuija in which participants recognized 2cm marks with 97% accu- racy, Graffiti digits with 98.8%, pairs of Graffiti digits with 90.5%, and Graffiti letters with 93.4%. Participants previ- ously unfamiliar with Graffiti identified 96.2% of digits and 76.4% of letters, suggesting that properly designed gesture output is guessable. After the experiment, the same participants were able to enter 100% of Graffiti digits by heart and 92.2% of letters. This suggests that participants learned gesture input as a side effect of using gesture output on our prototypes.

Recent research proposes augmenting capacitive touch pads with tangible objects, enabling a new generation of mobile applications enhanced with tangible objects, such as game pieces and tangible controllers. In this paper, we extend the concept to capacitive tangibles consisting of multiple parts, such as stackable gaming pieces and tangible widgets with moving parts. We achieve this using a system of wires and connectors inside each block that causes the capacitance of the bottom-most block to reflect the entire assembly. We demonstrate three types of tangibles, called CapStones, Zebra Dials and Zebra Sliders that work with current consumer hardware and investigate what designs may become possible as touchscreen hardware evolves.

Advances in sensing technology are currently bringing touch input to non-planar surfaces, ranging from spherical touch screens to prototypes the size and shape of a ping-pong ball. To help interface designers create usable interfaces on such devices, we determine how touch surface curvature affects targeting. We present a user study in which participants acquired targets on surfaces of different curvature and at locations of different slope. We find that surface convexity increases pointing accuracy, and in particular reduces the offset between the input point perceived by users and the input point sensed by the device. Concave surfaces, in contrast, are subject to larger error offsets. This is likely caused by how concave surfaces hug the user's finger, thus resulting in a larger contact area. The effect of slope on targeting, in contrast, is unexpected at first sight. Some targets located downhill from the user's perspective are subject to error offsets in the opposite direction from all others. This appears to be caused by participants acquiring these targets using a different finger posture that lets them monitor the position of their fingers more effectively.

The input vocabulary for touch-screen interaction on handhelds is dramatically limited, especially when the thumb must be used. To enrich that vocabulary we propose to discriminate, among thumb gestures, those we call MicroRolls, characterized by zero tangential velocity of the skin relative to the screen surface. Combining four categories of thumb gestures, Drags, Swipes, Rubbings and MicroRolls, with other classification dimensions, we show that at least 16 elemental gestures can be automatically recognized. We also report the results of two experiments showing that the roll vs. slide distinction facilitates thumb input in a realistic copy and paste task, relative to existing interaction techniques.

International audience