In the physical world, the force required to interact with an object provides rich information about its material properties and intended use: some objects deform easily when force is applied (e.g. a pillow) while others resist our attempts (e.g. a spring). Recent advances in haptic technology support integrating force modalities into interactive surfaces, presenting novel ways to convey material properties and create engaging experiences. However, little is known about how people perceive and interpret surface-based force interactions. To address this, we designed the Transforcer, a device generating normal, shear, and rotational forces, and studied user perceptions in different scenarios. Our findings show that force is experienced as a multi-layered interaction: bodily perception grounds interpretation, material qualities convey intent and control, and meaningful interactions emerge when body, task, and system align. We contribute (i) a design space of surface-based force interactions; (ii) the design and implementation of the Transforcer; (iii) findings from a semi-structured interview study; and (iv) design recommendations.

Out-of-session or “homework” engagement is a primary limiting factor in clinical mental health outcomes. Despite weekly practitioner contact, adherence to prescribed Digital Mental Health Interventions (DMHIs) typically drops by 96.1% within two weeks. We evaluate Ankor, a handheld shape-changing biofeedback physicalization, as an adjunct to standard audio-guided mindfulness. In a longitudinal randomized controlled study (N=69), participants were assigned to Ankor+audio or audio-only control across six weekly 15-minute laboratory sessions, with optional DMHI use between sessions. Relative to control, Ankor yielded a 351% increase in total DMHI practice initiations and, by week 2, maintained 29.4% active users versus 2.9% in control, indicating substantially higher out-of-session engagement and reduced early disengagement. These findings demonstrate the capacity of shape-changing biofeedback physicalizations to extend adherence to DMHIs, highlighting kinaesthetic interactions as a promising design pathway for sustaining engagement in mental health interventions.

Computing interfaces are becoming increasingly sophisticated, with systems that engage multiple sensory channels simultaneously. Deformable and shape-changing interfaces offer rich tactile experiences, but there is limited understanding of how they can be combined with other modes of sensory feedback. We systematically explored the audio, visual and tactile cross-modal correspondences of deformable shapes with a particular focus on auditory feedback. 50 participants were asked to associate deformable tactile stimuli, varying in stiffness and shape, with the sound qualities pitch, brightness, fade-in time and fade-out time, under visuo-tactile and tactile-only conditions. Our findings provide the first insights on how (1) shape, both its form and visibility, play a significant role in associations for pitch and brightness; (2) stiffness plays a dominant role in associations over a sound's fade-in and fade-out times. These findings are distilled into the first design guidelines for integrating auditory feedback into physical interfaces.

The force modality fundamentally transforms the interaction space of traditional touch input. When paired with compliant devices, which deform under force and provide immediate haptic feedback, there is potential to enhance user interactions significantly. However, the effects of compliance on force-input remain under-explored, with limited understanding of their full potential. This article presents the first systematic investigation of the impact of deformable, movable, and rigid surfaces on user performance and experience through three rigorous studies (each N = 28). The results reveal previously unreported effects, including (1) higher maximum comfortable forces on deformable surfaces, (2) user preference for soft and deformable surfaces over rigid surfaces, and (3) improved ability to maintain force input on softer surfaces. These results highlight the benefits of compliant surfaces, contrasting with the dominant use of force-input on rigid devices. These findings guide researchers and designers in optimizing user experience and performance of force-input interactions.

Surfaces with deformable and shape-changing properties seek to enhance and diversify tangible interactions with computing systems. However, we currently lack fundamental knowledge and user interface design principles that connect the inherent properties of deformable shapes with our human senses and cognitive associations. To address this knowledge gap, we systematically explored deformable shapes’ cross-modal correspondences (CC) with colours and emotions. In our CC study, 52 participants were presented with deformable shape stimuli that varied in stiffness and angularity. They were asked to associate these stimuli with colours and emotions under (i) visuo-tactile and; (ii) tactile-only conditions. For the first time, our findings reveal (1) how stiffness level primarily influences the CC associations and; (2) that stiffness and angularity play a significant role in CC associations over the visibility of the shapes. The results were distilled into design guidelines for future deformable, shape-changing interfaces that engage specific human senses and responses.

The proliferation of misleading visualizations online, particularly during critical events like public health crises and elections, poses a significant risk. This study investigates the capability of GPT-4 models (4V, 4o, and 4o mini) to detect misleading visualizations. Utilizing a dataset of tweet-visualization pairs containing various visual misleaders, we test these models under four experimental conditions with different levels of guidance. We show that GPT-4 models can detect misleading visualizations with moderate accuracy without prior training (naive zero-shot) and that performance notably improves when provided with definitions of misleaders (guided zero-shot). However, a single prompt engineering technique does not yield the best results for all misleader types. Specifically, providing the models with misleader definitions and examples (guided few-shot) proves more effective for reasoning misleaders, while guided zero-shot performs better for design misleaders. This study underscores the feasibility of using large vision-language models to detect visual misinformation and the importance of prompt engineering for optimized detection accuracy.

Data physicalizations embody a fascinating intersection of design, technology, and data, where physical artifacts encode information through their geometry or material properties (Jansen et al. 201...

Digital interventions are often used to support people with mental health conditions, but low engagement frequently reduces their effectiveness. We investigate the use of a Physical Artefact for Well-being Support (PAWS) to improve engagement and effectiveness of an audio-only guided well-being intervention. Through our handheld shape-changing biofeedback-based PAWS, users can synchronously feel their breath via kinaesthetic haptic feedback. By evaluating our device in a randomised-controlled experimental paradigm (N=58), we demonstrate significant reductions in physiological and subjective (self-reported) anxiety compared to an audio-only control. Our findings conclude that synchronous interactions with one‘s own physiological data via the PAWS, improves engagement and effectiveness of an intervention.

Deformable interfaces provide unique interaction potential for force input, for example, when users physically push into a soft display surface. However, there remains limited understanding of which visual-tactile design elements signify the presence and stiffness of such deformable force-input components. In this paper, we explore how people correspond surface stiffness to colours, graphical shapes, and physical shapes. We conducted a cross-modal correspondence (CC) study, where 30 participants associated different surface stiffnesses with colours and shapes. Our findings evidence the CCs between stiffness levels for a subset of the 2D/3D shapes and colours used in the study. We distil our findings in three design recommendations: (1) lighter colours should be used to indicate soft surfaces, and darker colours should indicate stiff surfaces; (2) rounded shapes should be used to indicate soft surfaces, while less-curved shapes should be used to indicate stiffer surfaces, and; (3) longer 2D drop-shadows should be used to indicate softer surfaces, while shorter drop-shadows should be used to indicate stiffer surfaces.

Data-physicalizations encode data and meaning through geometry or material properties, providing a non-planar view of data, offering novel opportunities for interrogation, discovery and presentation. This field has explored how single users interact with complex 3D data, but the challenges in the application of this technology to collaborative situations have not been addressed. We describe a study exploring interactions and preferences among co-located individuals using a dynamic data-physicalization in the form of a shape-changing bar chart, and compare this to previous work with single participants. Results suggest that co-located interactions with physical data prompt non-interactive hand gestures, a mirroring of physicalizations, and novel hand gestures in comparison to single participant studies. We also note that behavioural similarities in participants between interactive tabletop studies and data-physicalizations may be capitalised upon for further development of these dynamic representations. Finally, we consider the implications and challenges for the adoption of these types of platforms.

Welcome to this issue of the Proceedings of the ACM on Human-Computer Interaction, the first to bring together contributions from the community on Mobile Human-Computer Interaction (MHCI). The pervasive impact of mobile technologies over the last few years of the global pandemic is clear. They have become the de facto means for connecting children to their virtual classrooms, for remote doctor visits, to keep us 'in touch' with loved ones, for allowing us to keep working, and everything else in between. Yet, as researchers and practitioners, we are humbled by the many gaps that have needed our renewed attention. The call for articles for this first issue focused on contributions on the Mobile HCI theme received 113 valid submissions from across the world. After the first round of reviews, 56 contributions were invited to submit revised versions of their articles. After second round reviews 41 articles were accepted, giving an acceptance rate of 36%. We would like to thank our dedicated team of 35 associate editors and the external reviewers they recruited who have ensured the articles in this issue were rigorously reviewed to form a collection of high-quality articles on the theme of Mobile HCI.

One of the most fundamental interactions –pointing– is well understood on flat surfaces. However, pointing performance on tangible surfaces with physical targets is still limited for Tangible User Interfaces (TUIs). We investigate the effect of a target’s physical width, height, and distance on user pointing performance. We conducted a study using a reciprocal tapping task ( n =19) with physical rods arranged in a circle. We compared our data with five conventional interaction models designed for 2D/3D tasks rather than tangible targets. We show that variance in the movement times was only satisfactorily explained by a model established for volumetric displays ( r 2 =0.954). Analysis shows that movement direction and height should be included as parameters to this model to generalize for 3D tangible targets. Qualitative feedback from participants suggests that pointing at physical targets involves additional human factors (e.g., perception of sharpness or robustness) that need to be investigated further to understand how performance with tangible objects is affected.

The VR community has introduced many object selection and manipulation techniques during the past two decades. Typically, they are empirically studied to establish their benefits over the state-of-the-art. However, the literature contains few guidelines on how to conduct such studies; standards developed for evaluating 2D interaction often do not apply. This lack of guidelines makes it hard to compare techniques across studies, to report evaluations consistently, and therefore to accumulate or replicate findings. To build such guidelines, we review 20 years of studies on VR object selection and manipulation. Based on the review, we propose recommendations for designing studies and a checklist for reporting them. We also identify research directions for improving evaluation methods and offer ideas for how to make studies more ecologically valid and rigorous.

Passwords have become a ubiquitous part of our everyday lives, needed for every web-service and system. However, it is challenging to create safe and diverse alphanumeric passwords, and to recall them, imposing a cognitive burden on the user. Through consecutive experiments, we explored the movement space, affordances and interaction, and memorability of a tangible, handheld, embodied password. In this context, we found that: (1) a movement space of 200 mm × 200 mm is preferred; (2) each context has a perceived level of safety, which—together with the affordances and link to familiarity—influences how the password is performed. Furthermore, the artefact’s dimensions should be balanced within the design itself, with the user, and the context, but there is a trade-off between the perceived safety and ergonomics; and (3) the designed embodied passwords can be recalled for at least a week, with participants creating unique passwords which were reproduced consistently.

Composite data physicalizations allow for the physical reconfiguration of data points, creating new opportunities for interaction and engagement. However, there is a lack of understanding of people’s strategies and behaviors when directly manipulating physical data objects. In this paper, we systematically characterize different reconfiguration strategies using six exemplar physicalizations. We asked 20 participants to reorganize these exemplars with two levels of restriction: changing a single data object versus changing multiple data objects. Our findings show that there were two main reconfiguration strategies used: changes in proximity and changes in atomic orientation. We further characterize these using concrete examples of participant actions in relation to the structure of the physicalizations. We contribute an overview of reconfiguration strategies, which informs the design of future manually reconfigurable and dynamic composite physicalizations.

The use of Virtual Reality (VR) in applications such as data analysis, artistic creation, and clinical settings requires high precision input. However, the current design of handheld controllers, where wrist rotation is the primary input approach, does not exploit the human fingers' capability for dexterous movements for high precision pointing and selection. To address this issue, we investigated the characteristics and potential of using a pen as a VR input device. We conducted two studies. The first examined which pen grip allowed the largest range of motion---we found a tripod grip at the rear end of the shaft met this criterion. The second study investigated target selection via 'poking' and ray-casting, where we found the pen grip outperformed the traditional wrist-based input in both cases. Finally, we demonstrate potential applications enabled by VR pen input and grip postures.

As physicalizations encode data in their physical 3D form, the orientation in which the user is viewing the physicalization may impact the way the information is perceived. However, this relation between user orientation and perception of physical properties is not well understood or studied. To investigate this relation, we conducted an experimental study with 20 participants who viewed 6 exemplars of physicalizations from 4 different perspectives. Our findings show that perception is directly influenced by user orientation as it affects (i) the number and type of clusters, (ii) anomalies and (iii) extreme values identified within a physicalization. Our results highlight the complexity and variability of the relation between user orientation and perception of physicalizations.

Digital-augmentation of print-media can provide contextually relevant audio, visual, or haptic content to supplement the static text and images. The design of such augmentation--its medium, quantity, frequency, content, and access technique--can have a significant impact on the reading experience. In the worst case, such as where children are learning to read, the print medium can become a proxy for accessing digital content only, and the textual content is avoided. In this work, we examine how augmented content can change the reader's behaviour with a comic book. We first report on the usage of a commercially available augmented comic for children, providing evidence that a third of all readers converted to simply viewing the digital media when printed content is duplicated. Second, we explore the design space for digital content augmentation in print media. Third, we report a user study with 136 children that examined the impact of both content length and presentation in a digitally-augmented comic book. From this, we report a series of design guidelines to assist designers and editors in the development of digitally-augmented print media.

Actuating, dynamic materials offer substantial potential to enhance interior designs but there are currently few examples of how they might be utilized or impact user experiences. As part of a design-led exploration, we have prototyped (Wizard-of-Oz) an actuating, dining table runner (ActuEater1), and then developed a fully-interactive fabric version that both changes shape and colour (ActuEater2). Four in-situ deployments of 'ActuEaters' in different dinner settings and subsequent 'design crits' showed insights into how people perceive, interpret and interact with such slow-technology in interesting (and often unexpected) ways. The results of our 'ActuEating' studies provide evidence for how an actuating artefact can be simultaneously a resource for social engagement and an interactive decorative. In response, we explore design opportunities for situating novel interactive materials in everyday settings, taking the leap into a new generation of interactive spaces, and critically considering new aesthetic possibilities.

Many shape-changing interfaces use an array of actuated rods to create a display surface; each rod working as a pixel. However, this approach only supports pixel height manipulation and cannot produce more radical shape changes of each pixel (and thus of the display). Examples of such changes include non-horizontal pixels, pixels that overhang other pixels, or variable gaps between pixels. We present a concept for composing shape-changing interfaces by vertically stacking tilt-enabled modules. Together, stacking and tilting allow us to create a more diverse range of display surfaces than using arrays. We demonstrate this concept through TiltStacks, a shape-changing prototype built using stacked linear actuators and displays. Each tiltable module provides three degrees of freedom (z-movement, roll, and pitch); two more degrees of freedom are added through stacking modules (i.e., planar x- and y-movement).

Shape-changing interfaces have emerged as a new method for interacting with computers, using dynamic changes in a device's physical shape for input and output. With the advances of research into shape-changing interfaces, we see a need to synthesize the main, open research questions. The purpose of this synthesis is to formulate common challenges across the diverse fields engaged in shape-change research, to facilitate progression from single prototypes and individual design explorations to grander scientific goals, and to draw attention to challenges that come with maturity, including those concerning ethics, theory-building, and societal impact. In this article we therefore present 12 grand challenges for research on shape-changing interfaces, derived from a three-day workshop with 25 shape-changing interface experts with backgrounds in design, computer science, human-computer interaction, engineering, robotics, and material science.

We present a design approach for rapid fabrication of high fidelity interactive shape-changing displays using bespoke semi-solid surfaces. This is achieved by segmenting virtual representations of the given data and mapping it to a dynamic physical polygonal surface. First, we establish the design and fabrication approach for generating semi-solid reconfigurable surfaces. Secondly, we demonstrate the generalizability of this approach by presenting design sessions using datasets provided by experts from a diverse range of domains. Thirdly, we evaluate user engagement with the prototype hardware systems that are built. We learned that all participants, all of whom had no previous interaction with shape-changing displays, were able to successfully design interactive hardware systems that physically represent data specific to their work. Finally, we reflect on the content generated to understand if our approach is effective at representing intended output based on a set of user defined functionality requirements.

This article characterizes actuation techniques for generating movement in shape-changing displays with physically reconfigurable geometry. To date, few works in human–computer interaction literature provide detailed and reflective descriptions of the implementation techniques used in shape-changing displays. This hinders the rapid development of novel interactions as researchers must initially spend time understanding technologies before prototyping new interactions and applications. To bridge this knowledge gap, the authors propose a taxonomy that classifies actuator characteristics and simplifies the process for designers to select appropriate technologies that match their requirements for developing shape displays. They scope the investigation to linear actuators that are used in grid configurations. The taxonomy is validated by (a) examining current implementation techniques of motorized, pneumatic, hydraulic, magnetic, and shape-memory actuators in the literature, (b) constructing prototypes to address limited technical details and explore actuator capabilities in depth, (c) describing a use-case scenario through a case study that details the construction of a 10 × 10 actuator shape-display, and (d) a set of guidelines to aid researchers in selecting actuation techniques for shape-changing applications. The significance of their taxonomy is twofold. First, the authors provide an original contribution that enables HCI researchers to appropriately select actuation techniques and build shape-changing applications. This is situated amongst other past works that have investigated broader application scenarios such as a shape-changing vocabulary, a framework for shape transformations, material properties, and technical characteristics of various actuators. Second, they carry out in-depth investigations to validate their taxonomy and expand the knowledge of vertical actuation in shape-changing applications to enable rapid development.

Gaze information provides indication of users focus which complements remote collaboration tasks, as distant users can see their partner’s focus. In this paper, we apply gaze for co-located collaboration, where users’ gaze locations are presented on the same display, to help collaboration between partners. We integrated various types of gaze indicators on the user interface of a collaborative search system, and we conducted two user studies to understand how gaze enhances coordination and communication between co-located users. Our results show that gaze indeed enhances co-located collaboration, but with a trade-off between visibility of gaze indicators and user distraction. Users acknowledged that seeing gaze indicators eases communication, because it let them be aware of their partner’s interests and attention. However, users can be reluctant to share their gaze information due to trust and privacy, as gaze potentially divulges their interests.

Physical data representations, or data physicalizations, are a promising new medium to represent and communicate data. Previous work mostly studied passive physicalizations which require humans to perform all interactions manually. Dynamic shape-changing displays address this limitation and facilitate data exploration tasks such as sorting, navigating in data sets which exceed the fixed size of a given physical display, or preparing "views" to communicate insights about data. However, it is currently unclear how people approach and interact with such data representations. We ran an exploratory study to investigate how non-experts made use of a dynamic physical bar chart for an open-ended data exploration and presentation task. We asked 16 participants to explore a data set on European values and to prepare a short presentation of their insights using a physical display. We analyze: (1) users' body movements to understand how they approach and react to the physicalization, (2) their hand-gestures to understand how they interact with physical data, (3) system interactions to understand which subsets of the data they explored and which features they used in the process, and (4) strategies used to explore the data and present observations. We discuss the implications of our findings for the use of dynamic data physicalizations and avenues for future work.

The benefits of two-point interaction for tasks that require users to simultaneously manipulate multiple entities or dimensions are widely known. Two-point interaction has become common, e.g., when zooming or pinching using two fingers on a smartphone. We propose a novel interaction technique that implements three-point interaction by augmenting two-finger direct touch with gaze as a third input channel. We evaluate two key characteristics of our technique in two multi-participant user studies. In the first, we used the technique for object selection. In the second, we evaluate it in a 3D matching task that requires simultaneous continuous input from fingers and the eyes. Our results show that in both cases participants learned to interact with three input channels without cognitive or mental overload. Participants' performance tended towards fast selection times in the first study and exhibited parallel interaction in the second. These results are promising and show that there is scope for additional input channels beyond two-point interaction.

Shape-changing displays - visual output surfaces with physically-reconfigurable geometry - provide new challenges for content generation. Content design must incorporate visual elements, physical surface shape, react to user input, and adapt these parameters over time. The addition of the 'shape channel' significantly increases the complexity of content design, but provides a powerful platform for novel physical design, animations, and physicalizations. In this work we use ShapeCanvas, a 4×4 grid of large actuated pixels, combined with simple interactions, to explore novice user behavior and interactions for shape-change content design. We deployed ShapeCanvas in a café for two and a half days and observed users generate 21 physical animations. These were categorized into seven categories and eight directly derived from people's personal interest. This paper describes these experiences, the generated animations and provides initial insights into shape-changing content design.

Bimanual pen and touch UIs are mainly based on the direct manipulation paradigm. Alternatively we propose partially-indirect bimanual input, where direct pen input is used with the dominant hand, and indirect-touch input with the non-dominant hand. As direct and indirect inputs do not overlap, users can interact in the same space without interference. We investigate two indirect-touch techniques combined with direct pen input: the first redirects touches to the user's gaze position, and the second redirects touches to the pen position. In this paper, we present an empirical user study where we compare both partially-indirect techniques to direct pen and touch input in bimanual pan, zoom, and ink tasks. Our experimental results show that users are comparatively fast with the indirect techniques, but more accurate as users can dynamically change the zoom-target during indirect zoom gestures. Further our studies reveal that direct and indirect zoom gestures have distinct characteristics regarding spatial use, gestural use, and bimanual parallelism.

Gaze can complement touch on surfaces for fast target selection and occlusion-free input. In this work, we look beyond single-user application of gaze and touch and explore how gaze can be leveraged for collaborative use. We present the design of a two-player shooter game in which targets are gaze-aware and able to react differently to attention by one of the players versus shared attention of both players. The game-play, evaluated in a study with 14 users, encourages users to adopt different strategies switching between individual and shared attention to achieve their collaborative goal.

Humans have represented data in many forms for thousands of years, yet the main sensory channel we use to perceive these representations today still remains largely exclusive to sight. Recent developments, such as advances in digital fabrication, microcontrollers, actuated tangibles, and shape-changing interfaces offer new opportunities to encode data in physical forms and have stimulated the emergence of 'Data Physicalization' as a research area.

This paper presents ShapeClip: a modular tool capable of transforming any computer screen into a z-actuating shape-changing display. This enables designers to produce dynamic physical forms by "clipping" actuators onto screens. ShapeClip displays are portable, scalable, fault-tolerant, and support runtime re-arrangement. Users are not required to have knowledge of electronics or programming, and can develop motion designs with presentation software, image editors, or web-technologies. To evaluate ShapeClip we carried out a full-day workshop with expert designers. Participants were asked to generate shape-changing designs and then construct them using ShapeClip. ShapeClip enabled participants to rapidly and successfully transform their ideas into functional systems.

Physical representations of data have existed for thousands of years. Yet it is now that advances in digital fabrication, actuated tangible interfaces, and shape-changing displays are spurring an emerging area of research that we call Data Physicalization. It aims to help people explore, understand, and communicate data using computer-supported physical data representations. We call these representations physicalizations, analogously to visualizations -- their purely visual counterpart. In this article, we go beyond the focused research questions addressed so far by delineating the research area, synthesizing its open challenges and laying out a research agenda.

Visualizations such as bar charts help users reason about data, but are mostly screen-based, rarely physical, and almost never physical and dynamic. This paper investigates the role of physically dynamic bar charts and evaluates new interactions for exploring and working with datasets rendered in dynamic physical form. To facilitate our exploration we constructed a 10x10 interactive bar chart and designed interactions that supported fundamental visualisation tasks, specifically; annotation, filtering, organization, and navigation. The interactions were evaluated in a user study with 17 participants. Our findings identify the preferred methods of working with the data for each task i.e. directly tapping rows to hide bars, highlight the strengths and limitations of working with physical data, and discuss the challenges of integrating the proposed interactions together into a larger data exploration system. In general, physical interactions were intuitive, informative, and enjoyable, paving the way for new explorations in physical data visualizations.

Our work investigates the use of gaze and multitouch to fluidly perform rotate-scale-translate (RST) tasks on large displays. The work specifically aims to understand if gaze can provide benefit in such a task, how task complexity affects performance, and how gaze and multitouch can be combined to create an integral input structure suited to the task of RST. We present four techniques that individually strike a different balance between gaze-based and touch-based translation while maintaining concurrent rotation and scaling operations. A 16 participant empirical evaluation revealed that three of our four techniques present viable options for this scenario, and that larger distances and rotation/scaling operations can significantly affect a gaze-based translation configuration. Furthermore we uncover new insights regarding multimodal integrality, finding that gaze and touch can be combined into configurations that pertain to integral or separable input structures.

The physical tools used when designing new objects for digital fabrication are mature, yet disconnected from their virtual accompaniments. SPATA is the digital adaptation of two spatial measurement tools, that explores their closer integration into virtual design environments. We adapt two of the traditional measurement tools: calipers and protractors. Both tools can measure, transfer, and present size and angle. Their close integration into different design environments makes tasks more fluid and convenient. We describe the tools' design, a prototype implementation, integration into different environments, and application scenarios validating the concept.

Modalities such as pen and touch are associated with direct input but can also be used for indirect input. We propose to combine the two modes for direct-indirect input modulated by gaze. We introduce gaze-shifting as a novel mechanism for switching the input mode based on the alignment of manual input and the user's visual attention. Input in the user's area of attention results in direct manipulation whereas input offset from the user's gaze is redirected to the visual target. The technique is generic and can be used in the same manner with different input modalities. We show how gaze-shifting enables novel direct-indirect techniques with pen, touch, and combinations of pen and touch input.

Digital fabrication machines such as 3D printers and laser-cutters allow users to produce physical objects based on virtual models. The creation process is currently unidirectional: once an object is fabricated it is separated from its originating virtual model. Consequently, users are tied into digital modeling tools, the virtual design must be completed before fabrication, and once fabricated, re-shaping the physical object no longer influences the digital model. To provide a more flexible design process that allows objects to iteratively evolve through both digital and physical input, we introduce bidirectional fabrication. To demonstrate the concept, we built ReForm, a system that integrates digital modeling with shape input, shape output, annotation for machine commands, and visual output. By continually synchronizing the physical object and digital model it supports object versioning to allow physical changes to be undone. Through application examples, we demonstrate the benefits of ReForm to the digital fabrication process.

During online search, the user's expectations often differ from those of the author. This is known as the "intention gap" and is particularly problematic when searching for and discriminating between online video content. An author uses description and meta-data tags to label their content, but often cannot predict alternate interpretations or appropriations of their work. To address this intention gap, we present ThumbReels, a concept for query-sensitive video previews generated from crowdsourced, temporally defined semantic tagging. Further, we supply an open-source tool that supports on-the-fly temporal tagging of videos, whose output can be used for later search queries. A first user study validates the tool and concept. We then present a second study that shows participants found ThumbReels to better represent search terms than contemporary preview techniques.

Audio and vibrotactile output are the standard mechanisms mobile devices use to attract their owner's attention. Yet in busy and noisy environments, or when the user is physically active, these channels sometimes fail. Recent work has explored the use of physical shape-change as an additional method for conveying notifications when the device is in-hand or viewable. However, we do not yet understand the effectiveness of physical shape-change as a method for communicating in-pocket notifications. This paper presents three robustly implemented, mobile-device sized shape-changing devices, and two user studies to evaluate their effectiveness at conveying notifications. The studies reveal that (1) different types and configurations of shape-change convey different levels of urgency and; (2) fast pulsing shape-changing notifications are missed less often and recognised more quickly than the standard slower vibration pulse rates of a mobile device.

The magic lens paradigm, a commonly used descriptor for handheld Augmented Reality (AR), presents the user with dual views: the augmented view (magic lens) that appears on the device, and the real view of the surroundings (what the user can see around the perimeter of the device). The augmented view is typically implemented by rendering the video captured by the rear-facing camera directly onto the device's screen. This results in dual perspectives - the real world being captured from the device's perspective rather than the user's perspective (what an observer would see looking through a transparent glass pane). These differences manifest themselves in misaligned and/or incorrectly scaled transparency resulting in the dual-view problem.

Within a pervasive computing environment, we see content on shared displays that we wish to acquire and use in a specific way i.e., with an application on a personal device, transferring from point-to-point. The eyes as input can indicate intention to interact with a service, providing implicit pointing as a result. In this paper we investigate the use of gaze and manual input for the positioning of gaze-acquired content on personal devices. We evaluate two main techniques, (1) Gaze Positioning, transfer of content using gaze with manual input to confirm actions, (2) Manual Positioning, content is selected with gaze but final positioning is performed by manual input, involving a switch of modalities from gaze to manual input. A first user study compares these techniques applied to direct and indirect manual input configurations, a tablet with touch input and a laptop with mouse input. A second study evaluated our techniques in an application scenario involving distractor targets. Our overall results showed general acceptance and understanding of all conditions, although there were clear individual user preferences dependent on familiarity and preference toward gaze, touch, or mouse input.

Gaze has the potential to complement multi-touch for interaction on the same surface. We present gaze-touch, a technique that combines the two modalities based on the principle of 'gaze selects, touch manipulates'. Gaze is used to select a target, and coupled with multi-touch gestures that the user can perform anywhere on the surface. Gaze-touch enables users to manipulate any target from the same touch position, for whole-surface reachability and rapid context switching. Conversely, gaze-touch enables manipulation of the same target from any touch position on the surface, for example to avoid occlusion. Gaze-touch is designed to complement direct-touch as the default interaction on multi-touch surfaces. We provide a design space analysis of the properties of gaze-touch versus direct-touch, and present four applications that explore how gaze-touch can be used alongside direct-touch. The applications demonstrate use cases for interchangeable, complementary and alternative use of the two modes of interaction, and introduce novel techniques arising from the combination of gaze-touch and conventional multi-touch.

In handheld Augmented Reality (AR) the magic-lens paradigm is typically implemented by rendering the video stream captured by the back-facing camera onto the device's screen. Unfortunately, such implementations show the real world from the device's perspective rather than the user's perspective. This dual-perspective results in misaligned and incorrectly scaled imagery, a predominate cause for the dual-view problem with potential to distort user's spatial perception. This paper presents a user study that analyzes users' expectations, spatial-perception, and their ability to deal with the dual-view problem, by comparing device-perspective and fixed Point-of-View (POV) user-perspective rendering. The results confirm the existence of the dual-view perceptual issue and that the majority of participants expect user-perspective rendering irrespective of their previous AR experience. Participants also demonstrated significantly better spatial perception and preference of the user-perspective view.

Handheld Augmented Reality (AR) is often presented using the magic-lens paradigm where the handheld device is portrayed as if it was transparent. Such a virtual transparency is usually implemented using video captured by a single camera rendered on the device's screen. This removes binocular-disparity, which may undermine user's ability to correctly estimate depth when seeing the world through the magic-lens. To confirm such an assumption this paper presents a qualitative user study that compares a magic-lens implemented on a mobile phone and a transparent glass replica. Observational results and questionnaire analysis indicate that binocular-disparity may play a significant role in participants' depth perception. These promising results led to the subsequent implementation of a stereoscopic magic-lens prototype on a commercially available mobile device.

The shared displays in our environment contain content that we desire. Furthermore, we often acquire content for a specific purpose, i.e., the acquisition of a phone number to place a call. We have developed a content transfer concept, Eye Drop. Eye Drop provides techniques that allow fluid content acquisition, transfer from shared displays, and local positioning on personal devices using gaze combined with manual input. The eyes naturally focus on content we desire. Our techniques use gaze to point remotely, removing the need for explicit pointing on the user's part. A manual trigger from a personal device confirms selection. Transfer is performed using gaze or manual input to smoothly transition content to a specific location on a personal device. This work demonstrates how techniques can be applied to acquire and apply actions to content through a natural sequence of interaction. We demonstrate a proof of concept prototype through five implemented application scenarios.

Foot-based gestures have recently received attention as an alternative interaction mechanism in situations where the hands are pre-occupied or unavailable. This paper investigates suitable real-world mappings of foot gestures to invoke commands and interact with virtual workspaces. Our first study identified user preferences for mapping common mobile-device commands to gestures. We distinguish these gestures in terms of discrete and continuous command input. While discrete foot-based input has relatively few parameters to control, continuous input requires careful design considerations on how the user's input can be mapped to a control parameter (e.g. the volume knob of the media player). We investigate this issue further through three user-studies. Our results show that rate-based techniques are significantly faster, more accurate and result if far fewer target crossings compared to displacement-based interaction. We discuss these findings and identify design recommendations.

Tangible objects placed on interactive surfaces allow users to employ a physical object to manipulate digital content. However, creating the reverse effect - having digital content manipulate a tangible object placed on the surface - is a more challenging task. We present a new approach to this problem, using ultrasound-based air pressure waves to move multiple tangible objects, independently, around an interactive surface. We describe the technical background, design, implementation, and test cases for such a system. We conclude by discussing practical uses of our system, Ultra-Tangibles, in the creation of new tangible user interfaces.

We present a new type of actuatable display, called Tilt Displays, that provide visual feedback combined with multi-axis tilting and vertical actuation. Their ability to physically mutate provides users with an additional information channel that facilitates a range of new applications including collaboration and tangible entertainment while enhancing familiar applications such as terrain modelling by allowing 3D scenes to be rendered in a physical-3D manner. Through a mobile 3x3 custom built prototype, we examine the design space around Tilt Displays, categorise output modalities and conduct two user studies. The first, an exploratory study examines users' initial impressions of Tilt Displays and probes potential interactions and uses. The second takes a quantitative approach to understand interaction possibilities with such displays, resulting in the production of two user-defined gesture sets: one for manipulating the surface of the Tilt Display, the second for conducting everyday interactions.

This paper presents a software toolkit designed to enable the rapid development of multimedia-rich, multi-touch enabled, and interactive projection-based displays. For instance: door displays, floor displays, wall displays, and interactive tables. Despite recent technological advances and the commercialization of hardware required to achieve this at a relatively low cost, creating and deploying such displays remains a difficult task, even for those with the essential technical skills and experience. We assert that greater accessibility of toolkits like the one presented in this paper will reduce these barriers and allow people (not necessarily from the ubiquitous computing domain) to apply the technology to their own fields. To assess this toolkit's suitability for this role, we present a system and usability evaluation. We observed that participants were able to quickly create their own novel display deployments. Our findings offer insights for potential toolkit users and those considering how to write programs for future ubiquitous projected display environments.

In this paper we describe the use of kick gestures for interaction with mobile devices. Kicking is a well-st udied leg action that can be harnessed in mobile contexts where the hands are busy or too dirty to interact with the phone. In this paper we examine the design space of kicki ng as an interaction technique through two user studies. The first study investigated how well users were able to control the direction of their kicks. Users were able to aim their kicks best when the movement range is divided into segments of at least 24°. In the second study we looked at the velocity of a kick. We found that the users are able to kick with at least two varying velocities. However, they also often undershoot the target velocity. Finally, we propose some specific applications in which kicks can prove beneficial.

Accelerometers are common on many devices, including those required for text-entry. We investigate how to enter text with devices that are solely enabled with accelerometers. The challenge of text-entry with such devices can be overcome by the careful investigation of the human limitations in gestural movements with accelerometers. Preliminary studies provide insight into two potential text-entry designs that purely use accelerometers for gesture recognition. In two experiments, we evaluate the effectiveness of each of the text-entry designs. The first experiment involves novice users over a 45 minute period while the second investigates the possible performance increases over a four day period. Our results reveal that a matrix-based text-entry system with a small set of simple gestures is the most efficient (5.4wpm) and subjectively preferred by participants.

With gesture-based interactions in mobile settings becoming more popular, there is a growing concern regarding the social acceptance of these interaction techniques. In this paper we begin by examining the various definitions of social acceptance that have been proposed in the literature to synthesize a definition that is based on how the user feels about performing a particular interaction as well as how the bystanders perceive the user during this interaction. We then present the main factors that influence gestures' social acceptance including culture, time, interaction type and the user's position on the innovation adoption curve. Through a user study we show that an important factor in determining social acceptance of gesture-based interaction techniques is the user's perception of others ability to interpret the potential effect of a manipulation.

In this paper, we show that people frequently return to previously-visited regions within their documents, and that scrollbars can be enhanced to ease this task. We analysed 120 days of activity logs from Microsoft Word and Adobe Reader. Our analysis shows that region revisitation is a common activity that can be supported with relatively short recency lists. This establishes an empirical foundation for the design of an enhanced scrollbar containing scrollbar marks that helps people return to previously visited document regions. Two controlled experiments show that scrollbar marks decrease revisitation time, and that a large number of marks can be used effectively. We then design an enhanced Footprints scrollbar that supports revisitation with several features, including scrollbar marks and mark thumbnails. Two further experiments show that the Footprints scrollbar was frequently used and strongly preferred over traditional scrollbars.

Interface designers normally strive for a design that minimises the user's effort. However, when the design's objective is to train users to interact with interfaces that are highly dependent on spatial properties (e.g. keypad layout or gesture shapes) we contend that designers should consider explicitly increasing the mental effort of interaction. To test the hypothesis that effort aids spatial memory, we designed a "frost-brushing" interface that forces the user to mentally retrieve spatial information, or to physically brush away the frost to obtain visual guidance. We report results from two experiments using virtual keypad interfaces -- the first concerns spatial location learning of buttons on the keypad, and the second concerns both location and trajectory learning of gesture shape. The results support our hypothesis, showing that the frost-brushing design improved spatial learning. The participants' subjective responses emphasised the connections between effort, engagement, boredom, frustration, and enjoyment, suggesting that effort requires careful parameterisation to maximise its effectiveness.

Scrolling is the standard way to navigate through many types of digital documents. However, moving more than a few pages can be slow because all scrolling techniques constrain visual search to only a small document region. To improve document navigation, we developed Space-Filling Thumbnails (SFT), an overview display that eliminates most scrolling. SFT provides two views: a standard page view for reading, and a thumbnail view that shows all pages. We tested SFT in three experiments that involved finding pages in documents. The first study (n=13) compared seven current scrolling techniques, and showed that SFT is significantly faster than the other methods. The second and third studies (n=32 and n=14) were detailed comparisons of SFT with thumbnail-enhanced scrollbars (TES), which performed well in the first experiment. SFT was faster than TES across all document types and lengths, particularly when tasks involved revisitation. In addition, SFT was strongly preferred by participants.

In order to use co-evolutionary techniques successfully one needs to investigate the dynamics of co-evolution. Assuming an open-ended evolutionary process, it would be desirable to establish the necessary and sufficient conditions which lead to a kind of arms race where species continually adapt in response to one another. In this paper we present a model of competitive co-evolution which is intended to investigate these conditions. In our model, co-evolving species are modeled as points which are placed randomly on a uniform landscape which is deformed by the species. The impact a species induces on its surrounding is not immediate. Instead, the deformation follows the species after some latency period. Evolution is modeled as a simple hill climbing process of the species. We investigate different conditions and their impact on the evolutionary dynamics. Some lead to stasis, some lead to cyclic behavior and others lead to an arms race.