In virtual reality, users are typically represented by and interact through a virtual body. Research frequently manipulates different features of these bodies. We analyze 208 studies to identify what aspects of virtual bodies are manipulated, how these manipulations affect interaction and other outcomes, and why they are assumed to do so. Based on the analysis, we propose a design space comprising seven types of visual manipulations: appearance, size, morphology, viewpoint, transfer, remapping, and control. We also synthesize findings on their effects—ranging from task performance to physiological responses and social outcomes—and examine the theories used to explain them, such as embodiment, Proteus effect, and presence. The design space helps researchers identify key variables and their interconnections in design and empirical research of virtual bodies. The synthesis further reveals unexplored causal connections and highlights theories that may account for observed effects.

In HCI, frameworks function as a type of theoretical contribution, often supporting ideation, design, and evaluation. Yet, little is known about how they are actually used, what functions they serve, and which scholarly practices that shape them. To address this gap, we conducted a systematic review of 615 papers from a decade of CHI proceedings (2015-2024) that prominently featured the term framework. We classified these papers into six engagement types. We then examined the role, form, and essential components of newly proposed frameworks through a functional typology, analyzing how they are constructed, validated, and articulated for reuse. Our results show that enthusiasm for proposing new frameworks exceeds the willingness to iterate on existing ones. They also highlight the ambiguity in the function of frameworks and the scarcity of systematic validation. Based on these insights, we call for more rigorous, reflective, and cumulative practices in the development and use of frameworks in HCI.

Visions for the future of computing, such as those on Ubiquitous Computing or Tangible Interfaces, are highly cited and frequently used in teaching. Yet, we know little about the practical value of these visions for research on Human-Computer Interaction (HCI) or how HCI researchers engage with them individually and collectively. To address this gap, we conducted a survey with 172 HCI researchers. We identified key benefits and pitfalls as well as specific uses of visions by researchers. Researchers appreciate how visions guide us, drive us, and initiate new fields. Simultaneously, researchers acknowledge how visions create hype, restrict our creativity, and make us disregard real-world problems. Based on these insights, we derive tensions related to the pursuit of visions and discuss critical reading practices. Our paper offers a metascientific account of visions in the HCI field along with tools for critical reflection when engaging with them.

Despite decades of research, behavior change technologies remain limited in effectiveness because they mainly focus on offering information. Immersive technologies, such as extended, virtual, and augmented reality, promise to address this limitation by shifting from information-centric to experience-centric interventions. However, we do not know which technical and psychological components of immersive technologies make interventions effective. This scoping review of 53 articles analyzes how immersive technology components are linked to theory-informed behavior change techniques, summarizes their effectiveness, and synthesizes their unique advantages for experience-based immersive behavior change interventions. We offer both a formalization of the impact of immersive technology components on behavior change, and summarize practical suggestions for designing and systematically evaluating immersive behavior change interventions in a framework toward theory-driven Extended Reality behavior change interventions.

An increasing number of HCI researchers have embraced open science ideas, such as sharing data and analysis code. However, such practices are only meaningful when the shared data and code enable other researchers to reproduce and reuse the reported findings. To investigate the reproducibility of HCI research, we identified all CHI papers that have shared study data and analysis code, and attempted to reproduce the results. We were able to fully reproduce 49% of the papers. We surveyed and interviewed authors, asking them to assess the reproducibility of their own work and to reflect on their motivations and obstacles in doing open science. We discuss what improves and hinders reproducibility and provide recommendations on how to increase reproducibility rates in HCI. While the value of replicability remains contested in HCI, we argue that the more modest goal of reproducibility is desirable.

A VR body can move faster than its user, making actions like reaching more efficient. We propose a VR body that not only moves faster during reaching, but also starts moving before the user has decided which target to reach for. However, it is unclear whether such a VR body would speed up choices, since moving towards a wrong target might cause confusion, or influence users’ choices. To explore these questions, we built a faster VR body prototype, focusing on an accelerated hand and arm, in a choice task. Thirty-four participants viewed random-dot displays to judge the overall motion direction and indicated their choice by pressing the corresponding button. Task difficulty was varied to influence choice uncertainty. Results showed that choice time decreased when the virtual body was 0.1 seconds ahead of the physical body, but increased when it was 0.3 seconds ahead. Users’ choice distributions showed no significant differences.

In virtual reality (VR), users typically control one virtual body — their avatar. Previous works enable users to act through multiple avatars simultaneously. These works, though, lack a systematic account of why users want to interact with multiple avatars and do not explain how users can manipulate and generate avatars. To address this, we run six workshops with 12 VR experts and develop a design space that captures four fundamental dimensions for acting through multiple avatars in VR: Appearance , Context , User-Avatar Mapping , and Motion Control . Researchers can use the design space to generate novel interaction opportunities involving multiple avatars or analyze existing work. We then run a usability study with 17 participants to understand the practicalities of an interface that integrates parts of the design space, which reveals conceptual and technical challenges that we address through design recommendations. • Our design space explains how users can interact through multiple avatars in VR. • Users want multiple avatars to stand in for themselves, other persons, or things. • Users use multiple avatars to perceive and act in multiple contexts simultaneously. • Challenges with multiple avatars include clutter, syncing, and switching between them.

Low-frequency sound Heartbeat sensationsFigure 1: Heartbeat signals are modulated using low-frequency sound waves to create the perception of heartbeat sensations within the user's chest cavity.

Effects of embodying virtual avatars are routinely validated experimentally by comparing synchronous and asynchronous movements between virtual and real bodies. This experimental paradigm, however, lacks justification, validation, and standardization. Asynchrony is implemented in numerous ways, such as through delayed, dislocated, or prerecorded movements, and these may impact embodiment and user experience distinctively. An online study ($N = 202$) revealed that variations of asynchrony cause disparate responses to embodiment and user experience, with prerecorded movements distorting embodiment the most. A think-aloud study ($N = 16$) revealed that asynchronous conditions lead to peculiar and oftentimes negative experiences. Furthermore, asynchronous conditions in some cases maintain, rather than break the body ownership illusion, as participants imitate the virtual body. Our results show that asynchrony in experiments on embodiment entails profound validity issues and should therefore be used with caution.

Although the literature on Human-Computer Interaction (HCI) catalogues many theories, it offers surprisingly few tools for theorising.This paper critiques dominant approaches to engaging with theory and proposes a working model for theorising in HCI.We then present graphical causal modelling as an effective theorising tool.This includes a step-by-step guide to building causal models and examples of their use in different stages of the research process.We explain how causal models help develop method-agnostic representations of research problems using directed acyclic graphs, identify potential confounders, and construct alternative interpretations of data.Finally, we discuss their limitations and challenges for adoption by the HCI community.

Presence appears an important concept for virtual reality (VR): It is frequently measured with questionnaires, and theory and methods about it have been discussed in numerous works. Yet, it is unclear how to actually work with this concept: Why is presence important to measure, how to choose an appropriate questionnaire, and what to conclude about it based on findings? To answer these questions, we review how the concept is put to work in 288 VR papers from 2023 measuring presence with questionnaires. Our findings include that measuring presence is often motivated by another construct, such as user experience; the reasons for choosing a specific questionnaire are often weak or not reported at all; and high presence values are frequently used simply to validate an interaction technique. We propose recommendations for working with presence and formulate questions to direct future research.

Designing efficient selection techniques for graphical user interfaces (GUIs) is fundamental in HCI research.We derive selection techniques based on the multiple process model, a theory that details the motor control processes during goal-directed movements.Specifically, we deduce three theoretical assumptions on how control processes of pre-planning, impulse control, and limb-target control could influence selection movements when adjusting GUI elements, including visual feedback, cursor position, and target position.Corresponding to our assumptions, we develop three techniques that hide the cursor when a target is highlighted, snap the cursor when selection begins, and expand clustered objects during selection movements.After that, we pre-register the assumptions and research methodology and evaluate the techniques in three crowdsourcing-based pointing studies.Our results show that all techniques improved the selection efficiency compared to established baselines.We further discuss the design implications and reflect on how we derived techniques from theory.

The user interface is a central concept in human–computer interaction, but peculiarly fuzzy. This affects discussions of the fundamentals of our discipline and the positioning of our work. We propose a model of the user interface that consists of four elements: Devices, Interaction Techniques, Representations, and Assemblies (DIRA). We explain their roles in the user interface and discuss some associated concerns about evaluation and design. We then show how to use the model to describe the elements of user interfaces (with examples that include a menu, a fisheye interface, and notifications) and to analyze the central characteristics of user interface paradigms (including tangible user interfaces and mixed reality). Finally, we discuss how describing user interfaces with the model can drive their design and evaluation. • The concept of user interface (UI) is central to HCI. • The elements of UI are Devices, Interaction Techniques, Representations, and Assemblies (DIRA). • The DIRA model can help describing and analyzing UIs and drive their evaluation and design.

Haptic experiences – technology-mediated experiences of touch – are complex and enigmatic; they are intertwined with the user’s motivations, context, and past experiences and are deeply interwoven with other sensory modalities. To better understand such experiences, we develop the Inference-Design Model for Haptic Experience. The model separates stimulus, sensation, and experience and identifies two diametrical processes: inference and design. Inference is the cognitive process through which humans interpret a sensory environment, while design involves the creation of such a sensory environment. Grounded in philosophical, neuroscientific, and psychological literature, we discuss the formation of haptic experiences facilitated through the structure and components of the Inference-Design Model. We use the model to present six key observations about the inference of haptic experiences and the design processes that enable them. The observations and their implications for researchers contribute to a fundamental understanding of haptic experiences in a model-centric research paradigm.

Demand characteristics refer to cues that can inform participants in experiments about the hypothesis and influence their behavior. They lead researchers to erroneously infer non-existing effects, undermining the experimental integrity of empirical studies. Despite a widespread acknowledgment of their confounding influence in experimental psychology, experiments involving humans and computers to a lesser extent consider effects of demand characteristics, as computerized protocols are thought to be immune to some experimenter biases. Furthermore, demand characteristics are considered to mainly effect subjective measures. As a result, demand characteristics often remain uncontrolled in studies involving computers, and in particular for objective measures such as performance. In this paper, we present two experiments that underline the importance of demand characteristics in human–computer interaction experiments. In a text-entry study, we made participants believe they were evaluating a research-based keyboard. This belief led to increased performance and self-reported user experience. In a second study, we conducted a thought experiment on the illusion of body ownership in virtual reality, where the experimental design indicated the study hypothesis. We found hypothesis-compliant responses from participants, even when they did not experience the illusion. We conclude that demand characteristics pose a significant challenge to the interpretation and validity of human–computer experiments, even when they are fully automated. We discuss the implications and offer guidelines to mitigate effects of demand characteristics.

While body ownership is central to research in virtual reality (VR), it remains unclear how experiencing an avatar over time shapes a person’s summary judgment of it. Such a judgment could be a simple average of the bodily experience, or it could follow the peak-end rule, which suggests that people’s retrospective judgment correlates with the most intense and recent moments in their experiencing. We systematically manipulate body ownership over a three-minute avatar embodiment using visuomotor asynchrony. Asynchrony here serves to negatively influence body ownership. We conducted one lab study ( N = 28 ) and two online studies (pilot: N = 97 and formal: N = 128 ) to investigate the influence of visuomotor asynchrony given (1) order, meaning early or late, (2) duration and magnitude while controlling the order, and (3) the interaction between order and magnitude. Our results indicate a significant order effect—later visuomotor asynchrony decreased the rating of body-ownership judgments more—but no convergent evidence on magnitude. We discuss how body-ownership judgments may be formed sensorily or affectively. • Late asynchrony reduces body-ownership judgments more than early asynchrony. • The end effect shows potential designs for enhancing virtual body experiences. • The peak effect of body-ownership judgments is challenging to influence.

Computing devices commonly act as tools, extending our abilities and shaping how we interact with the world. We investigate one such tool, the calculator, which helps with arithmetic, but also commonly offers specialized functions for conversions, formulas, or graphing. Through an analysis of calculator apps and use cases, we describe limitations of current calculators. Crucially, calculator apps remain detached from tasks, motivating us to explore how to more closely integrate calculation with the world through augmented reality (AR). AR calculators can directly use measurements and numbers from the world in calculations as well as display results of calculations in the world. We provide a conceptual account of calculation in AR, as well as video prototypes that concretize the concept across different scenarios. These examples demonstrate how moving tools like the calculator to AR offers tighter task integration and reduces the work required in translating between the world and computational tools.

Decision making often requires a series of low-level tasks involving descriptive statistics, each of them supported by a single visual representation, such as a bar chart or a violin plot. However, a single representation conveys limited information and little work has investigated using a combination of visual representations to facilitate the decision-making process. We propose multi-state visualizations, which allow users to switch easily between different visual representations (or states). Such visualizations provide users with more opportunities to explore and interpret the underlying data. We present three candidate multi-state visualizations, pairing error bars with violin plots, quantile dot plots, or hypothetical outcome plots. In a crowd-sourced study, we compare multi-state and single-state visualizations to investigate if they enhance users’ accuracy and confidence in making probability estimates. The results show that participants using multi-state visualization feel more confident and make more accurate estimations. Furthermore, we discuss the benefits of using multiple states in visualization for uncertainty visualizations.

Augmented Reality (AR) excels at altering what we see but non-visual sensations are difficult to augment. To augment non-visual sensations in AR, we draw on the visual language of comic books. Synthesizing comic studies, we create a design space describing how to use comic elements (e.g., onomatopoeia) to depict non-visual sensations (e.g., hearing). To demonstrate this design space, we built eight demos, such as speed lines to make a user think they are faster and smell lines to make a scent seem stronger. We evaluate these elements in a qualitative user study (N=20) where participants performed everyday tasks with comic elements added as augmentations. All participants stated feeling a change in perception for at least one sensation, with perceived changes detected by between four participants (touch) and 15 participants (hearing). The elements also had positive effects on emotion and user experience, even when participants did not feel changes in perception.

This paper proposes a method for generating non-contact sensations using low-frequency sound waves without requiring user instrumentation. This method leverages the fundamental acoustic response of a confined space to produce predictable pressure spatial distributions at low frequencies, called modes. These modes can be used to produce sensations either throughout the body, in localized areas of the body, or within the body. We first validate the location and strength of the modes simulated by acoustic modeling. Next, a perceptual study is conducted to show how different frequencies produce qualitatively different sensations across and within the participants’ bodies. The low-frequency sound offers a new way of delivering non-contact sensations throughout the body. The results indicate a high accuracy for predicting sensations at specific body locations.

In the 1979 book “The Meaning of Things” Csikszentmihalyi and Rochberg-Halton studied people’s perception of the significance of things in the home. They emphasized how things influence the self, and vice versa. We propose that their method and analytical framework can help to understand the analogous question for smartphones: Why are some apps special to users? Using the framework, we conduct and analyze 60 interviews with people aged 21 to 41; with participants’ consent, we made the anonymized transcripts publicly available. The analysis of the interviews shows that participants find apps special because they are convenient, support personal goals and social communication, help them remember, and serve emotional functions. Participants report that their identity is intertwined with certain apps, even if they are annoying or cause dependency. Importantly, we also find that participants actively regulate their use of apps through their organization and particular use strategies.

Digital self-control tools (DSCTs) help people control their time and attention on digital devices, using interventions like distraction blocking or usage tracking. Most studies of DSCTs’ effectiveness have focused on whether a single intervention reduces time spent on a single device. In reality, people may require combinations of DSCTs to achieve more subjective goals across multiple devices. We studied how DSCTs can address individual needs of university students (n = 280), using a workshop where students reflect on their goals before exploring relevant tools. At 1-3 month follow-ups, 95% of respondents still used at least one type of DSCT, typically applied across multiple devices, and there was substantial variation in the tool combinations chosen. We observed a large increase in self-reported digital self-control, suggesting that providing a space to articulate goals and self-select appropriate DSCTs is a powerful way to support people who struggle to self-regulate digital device use.

Providing attention guidance, such as assisting in search tasks, is a prominent use for Augmented Reality. Typically, this is achieved by graphically overlaying geometrical shapes such as arrows. However, providing visual guidance can cause side effects such as attention tunnelling or scene occlusions, and introduce additional visual clutter. Alternatively, visual guidance can adjust saliency but this comes with different challenges such as hardware requirements and environment dependent parameters. In this work we advocate for using flicker as an alternative for real-world guidance using Augmented Reality. We provide evidence for the effectiveness of flicker from two user studies. The first compared flicker against alternative approaches in a highly controlled setting, demonstrating efficacy (N = 28). The second investigated flicker in a practical task, demonstrating feasibility with higher ecological validity (N = 20). Finally, our discussion highlights the opportunities and challenges when using flicker to provide real-world visual guidance using Augmented Reality.

Advances in haptic technologies can alter how real objects feel to our touch and create the experience of haptic augmented reality (AR). However, the definition, use cases, and value to end users of such haptic AR remain unclear. Existing work is concerned with technological implementation and lacks a user-centered perspective. To address these limitations, we analyze haptic AR systems in the literature to understand what constitutes haptic AR, why we would want to alter our sense of touch, and how haptic AR interactions take place. To demonstrate the value of studying haptic AR in the context of real-world tasks and user impressions of the concept itself, we also conducted a small exploratory study with five prototypical applications of different haptic AR systems. Our analysis highlights unexplored areas for haptics and HCI researchers and the need to conduct user evaluations of the overall concept rather than just point examples.

Robots increasingly interact with humans through touch, where people are touching or being touched by robots. Yet, little is known about how such interactions shape a user’s experience. To inform future work in this area, we conduct a systematic review of 44 studies on physical human–robot interaction (pHRI). Our review examines the parameters of the touch (e.g., the role of touch, location), the experimental variations used by researchers, and the methods used to assess user experience. We identify five facets of user experience metrics from the questionnaire items and data recordings for pHRI studies. We highlight gaps and methodological issues in studying pHRI and compare user evaluation trends with the Human–Computer Interaction (HCI) literature. Based on the review, we propose a conceptual model of the pHRI experience. The model highlights the components of such touch experiences to guide the design and evaluation of physical interactions with robots and inform future user experience questionnaire development.

Most contemporary Virtual Reality (VR) experiences are made for standing users. However, when a user is lying down—either by choice or necessity—it is unclear how they can walk around, dodge obstacles, or grab distant objects. We rotate the virtual coordinate space to study the movement requirements and user experience of using VR while lying down. Fourteen experienced VR users engaged with various popular VR applications for 40 minutes in a study using a think-aloud protocol and semi-structured interviews. Thematic analysis of captured videos and interviews reveals that using VR while lying down is comfortable and usable and that the virtual perspective produces a potent illusion of standing up. However, commonplace movements in VR are surprisingly difficult when lying down, and using alternative interactions is fatiguing and hampers performance. To conclude, we discuss design opportunities to tackle the most significant challenges and to create new experiences.

Research on Extended Reality (XR) and Artificial Intelligence (AI) is booming, which has led to an emerging body of literature in their intersection. However, the main topics in this intersection are unclear, as are the benefits of combining XR and AI. This paper presents a scoping review that highlights how XR is applied in AI research and vice versa. We screened 2619 publications from 203 international venues published between 2017 and 2021, followed by an in-depth review of 311 papers. Based on our review, we identify five main topics at the intersection of XR and AI, showing how research at the intersection can benefit each other. Furthermore, we present a list of commonly used datasets, software, libraries, and models to help researchers interested in this intersection. Finally, we present 13 research opportunities and recommendations for future work in XR and AI research.

Ultrasound mid-air haptic technology provides a large space of design possibilities, as one can modulate the ultrasound intensity in a continuous 3D space at a high speed over time. Yet, the need for programming the patterns limits rapid ideation and testing of alternatives. We present Feellustrator, a graphical design tool for quickly creating and editing ultrasound mid-air haptics. With Feellustrator, one can create custom ultrasound patterns, layer or sequence them into complex effects, project them on the user’s hand, and export them for use in external programs (e.g., Unity). To create the tool, we interviewed 13 designers who had from a few months to several years of experience with ultrasound, then derived a set of requirements for supporting ultrasound design. We demonstrate the design power of Feellustrator through example applications and an evaluation with 15 participants. Then, we outline future directions for ultrasound haptic design.

When computers unexpectedly delay or thwart goal attainment, frustration ensues. The central studies of the extent, content, and impact of such frustration were done more than 15 years ago. We revisit this issue after computers have become more mature and computer use is more extensive. To this end, we had 234 crowdsourced participants log the frustrating episodes they experienced with their computers during one hour of computer use. The average time lost due to frustrating episodes was between 11% and 20% of the one-hour period. Though this is less time lost than in the earlier studies, frustration remains a common user experience. While shorter, the median level of frustration during the episodes was high (7 on a 9-point scale). The frustration level correlated with task importance and time lost but was unaffected by computer experience and largely unaffected by computer self-efficacy. In addition, participants indicated that 84% of the episodes had happened before, that 87% could happen again, and that they were unable to resolve 26% of the episodes. This high rate of recurrence and lack of control likely added to the frustration level. The episodes spanned various issues pertaining to performance (49%), usability (36%), and utility (16%).

Body ownership illusions (BOIs) occur when participants experience that their actual body is replaced by a body shown in virtual reality (VR). Based on a systematic review of the cumulative evidence on BOIs from 111 research articles published in 2010 to 2021, this article summarizes the findings of empirical studies of BOIs. Following the PRISMA guidelines, the review points to diverse experimental practices for inducing and measuring body ownership. The two major components of embodiment measurement, body ownership and agency, are examined. The embodiment of virtual avatars generally leads to modest body ownership and slightly higher agency. We also find that BOI research lacks statistical power and standardization across tasks, measurement instruments, and analysis approaches. Furthermore, the reviewed studies showed a lack of clarity in fundamental terminology, constructs, and theoretical underpinnings. These issues restrict scientific advances on the major components of BOIs, and together impede scientific rigor and theory-building.

In virtual reality (VR), interactions may fail when users encounter new, unknown, or unexpected objects. We propose using feedforward in VR to help users interact with objects by revealing how such objects work. Feedforward lets users know what to do and how to do it by showing the available actions and outcomes before an interaction. In this article, we first chart the design space of feedforward in VR and illustrate how to design feedforward for specific VR interactions. We discuss starting the feedforward, previewing actions and outcomes, and returning the virtual world to its state before the feedforward. Second, we implement three real-world VR applications to show how feedforward can be applied to multistep interactions, perceived interactivity, and discoverability. Third, we conduct an evaluation of the design space with 14 VR experts to understand its usefulness. Finally, we summarize the findings of our work on VR feedforward in 15 guidelines.

When giving input with a button, users follow one of two strategies: (1) react to the output from the computer or (2) proactively act in anticipation of the output from the computer. We propose a technique to quantify reactiveness and proactiveness to determine the degree and characteristics of each input strategy. The technique proposed in this study uses only screen recordings and does not require instrumentation beyond the input logs. The likelihood distribution of the time interval between the button inputs and system outputs, which is uniquely determined for each input strategy, is modeled. Then the probability that each observed input/output pair originates from a specific strategy is estimated along with the parameters of the corresponding likelihood distribution. In two empirical studies, we show how to use the technique to answer questions such as how to design animated transitions and how to predict a player’s score in real-time games.

We introduce OVRlap, a VR interaction technique that lets the user perceive multiple places at the same time from a first-person perspective. OVRlap achieves this by overlapping viewpoints. At any time, only one viewpoint is active, meaning that the user may interact with objects therein. Objects seen from the active viewpoint are opaque, whereas objects seen from passive viewpoints are transparent. This allows users to perceive multiple locations at once and easily switch to the one in which they want to interact. We compare OVRlap and a single-viewpoint technique in a study where 20 participants complete object-collection and monitoring tasks. We find that in both tasks, participants are significantly faster and move their head significantly less with OVRlap. We propose how the technique might be improved through automated switching of the active viewpoint and intelligent viewpoint rendering.

This essay addresses a foundational topic in applied sciences with interest in design: how do theories inform design? Previous work has attributed theory-use to abduction and deduction. However, design is about creating an intervention, a possible state that does not exist presently, and these accounts fail to explain how theories permit taking this leap. We argue that the practical value of a theory lies in counterfactual thinking. Theories are like “speculation pumps”: they produce (pump) counterfactual thought experiments of the type: If design was , then interaction would be . The more valid these thought experiments are and the better they direct the solution of design problems toward desirable and reliable outcomes, the more useful the theory. Counterfactual thinking sheds new light to design methods and, importantly, can reconcile an underlying tension between design sciences and applied sciences.

Mid-air haptic stimulation can enrich user experience during human-computer interaction. However, the design space of such stimuli is large due to the number and range of stimulation parameters. It therefore remains difficult for designers to select a stimulus to induce an intended experience. We derive a mapping for mid-air experiences based on two user studies. In the first study, participants rated 36 stimuli varied across three parameters (frequency, pattern, and repetitions). These ratings allowed us to determine a set of five experientially distinct stimuli. In the second study, participants vocalized their experiences with those five stimuli. This allows us to generate a mapping of 17 sensations and 23 experiences related to the stimuli. Finally, we discuss how the mapping can inform designers and researchers working with mid-air haptic technologies.

One of the most fundamental interactions –pointing– is well understood on flat surfaces. However, pointing performance on tangible surfaces with physical targets is still limited for Tangible User Interfaces (TUIs). We investigate the effect of a target’s physical width, height, and distance on user pointing performance. We conducted a study using a reciprocal tapping task ( n =19) with physical rods arranged in a circle. We compared our data with five conventional interaction models designed for 2D/3D tasks rather than tangible targets. We show that variance in the movement times was only satisfactorily explained by a model established for volumetric displays ( r 2 =0.954). Analysis shows that movement direction and height should be included as parameters to this model to generalize for 3D tangible targets. Qualitative feedback from participants suggests that pointing at physical targets involves additional human factors (e.g., perception of sharpness or robustness) that need to be investigated further to understand how performance with tangible objects is affected.

Sense of control is increasingly used as a measure of quality in human-computer interaction. Control has been investigated mainly at a high level, using subjective questionnaire data, but also at a low level, using objective data on participants’ sense of agency. However, it remains unclear how differences in higher level, experienced control reflect lower level sense of control. We study that link in two experiments. In the first one we measure the low-level sense of agency with button, touchpad, and on-skin input. The results show a higher sense of agency with on-skin input. In the second experiment, participants played a simple game controlled with the same three inputs. We find that on-skin input results in both increased sense and experience of control compared to touchpad input. However, the corresponding difference is not found between on-skin and button input, whereas the button performed better in the experiment task. These results suggest that other factors of user experience spill over to the experienced control at rates that overcome differences in the sense of control. We discuss the implications for using subjective measures about the sense of control in evaluating qualities of interaction.

The VR community has introduced many object selection and manipulation techniques during the past two decades. Typically, they are empirically studied to establish their benefits over the state-of-the-art. However, the literature contains few guidelines on how to conduct such studies; standards developed for evaluating 2D interaction often do not apply. This lack of guidelines makes it hard to compare techniques across studies, to report evaluations consistently, and therefore to accumulate or replicate findings. To build such guidelines, we review 20 years of studies on VR object selection and manipulation. Based on the review, we propose recommendations for designing studies and a checklist for reporting them. We also identify research directions for improving evaluation methods and offer ideas for how to make studies more ecologically valid and rigorous.

Virtual reality (VR) is increasingly used in complex social and physical settings outside of the lab. However, not much is known about how these settings influence use, nor how to design for them. We analyse 233 YouTube videos of VR Fails to: (1) understand when breakdowns occur, and (2) reveal how the seams between VR use and the social and physical setting emerge. The videos show a variety of fails, including users flailing, colliding with surroundings, and hitting spectators. They also suggest causes of the fails, including fear, sensorimotor mismatches, and spectator participation. We use the videos as inspiration to generate design ideas. For example, we discuss more flexible boundaries between the real and virtual world, ways of involving spectators, and interaction designs to help overcome fear. Based on the findings, we further discuss the ‘moment of breakdown’ as an opportunity for designing engaging and enhanced VR experiences.

A compelling property of virtual reality is that it allows users to interact with objects as they would in the real world. However, such interactions are limited to space within reach. We present Poros, a system that allows users to rearrange space. After marking a portion of space, the distant marked space is mirrored in a nearby proxy. Thereby, users can arrange what is within their reachable space, making it easy to interact with multiple distant spaces as well as nearby objects. Proxies themselves become part of the scene and can be moved, rotated, scaled, or anchored to other objects. Furthermore, they can be used in a set of higher-level interactions such as alignment and action duplication. We show how Poros enables a variety of tasks and applications and also validate its effectiveness through an expert evaluation.

Selection and manipulation in virtual reality often happen using an avatar’s hands. However, objects outside the immediate reach require effort to select. We develop a target selection technique called Ninja Hands. It maps the movement of a single real hand to many virtual hands, decreasing the distance to targets. We evaluate Ninja Hands in two studies. The first study shows that compared to a single hand, 4 and 8 hands are significantly faster for selecting targets. The second study complements this finding by using a larger target layout with many distractors. We find no decrease in selection time across 8, 27, and 64 hands, but an increase in the time spent deciding which hand to use. Thereby, net movement time still decreases significantly. In both studies, the physical motion exerted also decreases significantly with more hands. We discuss how these findings can inform future implementations of the Ninja Hands technique.

Learning to move the hands in particular ways is essential in many training andleisure virtual reality applications, yet challenging. Existing techniques that support learning of motor movement in virtual reality rely on external cues such as arrows showing where to move or transparent hands showing the target movement. We propose a technique where the avatar's hand movement is corrected to be closer to the target movement. This embeds guidance in the user's avatar, instead of in external cues and minimizes visual distraction. Through two experiments, we found that such movement guidance improves the short-term retention of the target movement when compared to a control condition without guidance.

Abstract The time we each spend using smartphones is increasing. So is the extent of discussions on whether that time is well spent and whether it results in positive experiences and ultimately improves well-being. However, research on this question rarely links the time spent on smartphones, the specific applications used, the motivation for using them and their effects on well-being. We had 70 participants compare experiences with a frequently used smartphone application and an occasionally used one. The participants used the Screen Time feature of the iPhone to select the applications and provided qualitative and quantitative data on their use of the applications. The findings show that the experience of pragmatic and hedonic value differs between the two application types, as does the experience of regret. The motivation for using the applications also in'uences whether the time is experienced as well spent. We use these findings to nuance the general discussion of smartphone usage and well-being.

A key challenge HCl researchers face when designing a controlled experiment is choosing the appropriate number of participants, or sample size. A priori power analysis examines the relationships among multiple parameters, including the complexity associated with human participants, e.g., order and fatigue effects, to calculate the statistical power of a given experiment design. We created Argus, a tool that supports interactive exploration of statistical power: Researchers specify experiment design scenarios with varying confounds and effect sizes. Argus then simulates data and visualizes statistical power across these scenarios, which lets researchers interactively weigh various trade-offs and make informed decisions about sample size. We describe the design and implementation of Argus, a usage scenario designing a visualization experiment, and a think-aloud study.

Evaluating the forecasting ability of time-series involves observations of multiple charts representing different aspects of model accuracy. However, the sequence of the charts observed by users is not controlled and it is difficult for users to discover relations among charts. Therefore, we propose a method for constructing a navigation structure that shows these relations based on the syntax and semantics of the charts. An excerpt from the structure is used as a context menu that allows users to navigate through a series of charts and explore their relations in a structured way. A qualitative study is conducted to evaluate the system and the results show that our approach helps users explore the connections among charts and enhances the understanding of time-series forecasting performance.

The process of verifying linear model assumptions and remedying associated violations is complex, even when dealing with simple linear regression. This process is not well supported by current tools and remains time-consuming, tedious, and error-prone. We present RegLine, a visual analytics tool supporting the iterative process of assumption verification and violation remedy for simple linear regression models. To identify the best possible model, RegLine helps novices perform data transformations, deal with extreme data points, analyze residuals, validate models by their assumptions, and compare and relate models visually. A qualitative user study indicates that these features of RegLine support the exploratory and refinement process of model building, even for those with little statistical expertise. These findings may guide visualization designs on how interactive visualizations can facilitate refining and validating more complex models.

Spatial recordings allow viewers to move within them and freely choose their viewpoint. However, such recordings make it easy to miss events and difficult to follow moving objects when skipping through the recording. To alleviate these problems we present the Who Put That There system that allows users to navigate through time by directly manipulating objects in the scene. By selecting an object, the user can navigate to moments where the object changed. Users can also view trajectories of objects that changed location and directly manipulate them to navigate. We evaluated the system with a set of sensemaking questions in a think-aloud study. Participants understood the system and found it useful for finding events of interest, while being present and engaged in the recording.

Virtual Reality allows users to embody avatars that do not match their real bodies. Earlier work has selected changes to the avatar arbitrarily and it therefore remains unclear how to change avatars to improve users' performance. We propose a systematic approach for iteratively adapting the avatar to perform better for a given task based on users' performance. The approach is evaluated in a target selection task, where the forearms of the avatar are scaled to improve performance. A comparison between the optimised and real arm lengths shows a significant reduction in average tapping time by 18.7%, for forearms multiplied in length by 5.6. Additionally, with the adapted avatar, participants moved their real body and arms significantly less, and subjective measures show reduced physical demand and frustration. In a second study, we modify finger lengths for a linear tapping task to achieve a better performing avatar, which demonstrates the generalisability of the approach.

Technological advances are enabling a new class of augmented reality (AR) applications that use bodies as substrates for input and output. In contrast to sensing and augmenting objects, body-based AR applications track people around the user and layer information on them. However, prototyping such applications is complex, time-consuming, and cumbersome, due to a lack of easily accessible tooling and infrastructure. We present Body LayARs, a toolkit for fast development of body-based AR prototypes. Instead of directly programming for a device, Body LayARs provides an extensible graphical programming environment with a device-independent runtime abstraction. We focus on face-based experiences for headset AR, and show how Body LayARs makes a range of body-based AR applications fast and easy to prototype.

The human sense of smell is powerful. However, the way we use smell as an interaction modality in human–computer interaction (HCI) is limited. We lack a common reference point to guide designers’ choices when using smell. Here, we map out an olfactory design space to provide designers with such guidance. We identified four key design features: (i) chemical, (ii) emotional, (iii) spatial, and (iv) temporal. Each feature defines a building block for smell-based interaction design and is grounded in a review of the relevant scientific literature. We then demonstrate the design opportunities in three application cases. Each application (i.e., one desktop, two virtual reality implementations) highlights the design choices alongside the implementation and evaluation possibilities in using smell. We conclude by discussing how identifying those design features facilitates a healthy growth of this research domain and contributes to an intermediate-level knowledge space. Finally, we discuss further challenges the HCI community needs to tackle.

Tool use extends people's representations of the immediately actionable space around them. Physical tools thereby become integrated in people's body schemas. We introduce a measure for tool extension in HCI by using a visual-tactile interference paradigm. In this paradigm, an index of tool extension is given by response time differences between crossmodally congruent and incongruent stimuli; tactile on the hand and visual on the tool. We use this measure to examine if and how findings on tool extension apply to interaction with computer-based tools. Our first experiment shows that touchpad and mouse both provide tool extension over a baseline condition without a tool. A second experiment shows a higher degree of tool extension for a realistic avatar hand compared to an abstract pointer for interaction in virtual reality. In sum, our measure can detect tool extension with computer-based tools and differentiate interfaces by their degree of extension.

Wall displays support people in interacting with large information spaces in two ways: On the one hand, the physical space in front of such displays enables them to navigate information spaces physically. On the other hand, the visual overview of the information space on the display may promote the formation of spatial memory; from studies of desktop computers we know this can boost performance. However, it remains unclear how the benefits of locomotion and overviews relate and whether one is more important than the other. We study this question through a wall display adaptation of the classic Data Mountain system to separate the effects of locomotion and visual overview. Our findings suggest that overview improves recall and that the combination of overview and locomotion outperforms all other combinations of factors.

We rely on our sight when manipulating objects. When objects are occluded, manipulation becomes difficult. Such occluded objects can be shown via augmented reality to re-enable visual guidance. However, it is unclear how to do so to best support object manipulation. We compare four views of occluded objects and their effect on performance and satisfaction across a set of everyday manipulation tasks of varying complexity. The best performing views were a see-through view and a displaced 3D view. The former enabled participants to observe the manipulated object through the occluder, while the latter showed the 3D view of the manipulated object offset from the object's real location. The worst performing view showed remote imagery from a simulated hand-mounted camera. Our results suggest that alignment of virtual objects with their real-world location is less important than an appropriate point-of-view and view stability.

Around-device interaction methods expand the available interaction space for mobile devices; however, there is currently no way to simultaneously track a user's input and provide haptic feedback at the tracked point away from the device. We present Magnetips, a simple, mobile solution for around-device tracking and mid-air haptic feedback. Magnetips combines magnetic tracking and electromagnetic feedback that works regardless of visual occlusion, through most common materials, and at a size that allows for integration with mobile devices. We demonstrate: (1) high-frequency around-device tracking and haptic feedback; (2) the accuracy and range of our tracking solution which corrects for the effects of geomagnetism, necessary for enabling mobile use; and (3) guidelines for maximising strength of haptic feedback, given a desired tracking frequency. We present technical and usability evaluations of our prototype, and demonstrate four example applications of its use.

The view of quality in human-computer interaction continuously develops, having in past decades included consistency, transparency, usability, and positive emotions. Recently, meaning is receiving increased interest in the user experience literature and in industry, referring to the end, purpose or significance of interaction with computers. However, the notion of meaning remains elusive and a bewildering number of senses are in use. We present a framework of meaning in interaction, based on a synthesis of psychological meaning research. The framework outlines five distinct senses of the experience of meaning: connectedness, purpose, coherence, resonance, and significance. We illustrate the usefulness of the framework by analyzing a selection of recent papers at the CHI conference and by raising a series of open research questions about the interplay of meaning, user experience, reflection, and well-being.

Human-computer interaction is replete with ways of talking about qualities of interaction or interfaces, including if they are expressive, rich, fluid, or playful. An example of such a quality is subtle. While this word is frequently used in the literature, we lack a coherent account of what it means to be subtle, how to achieve subtleness in an interface, and what theoretical backing subtleness has. To create such an account, we analyze a sample of 55 publications that use the word subtle. We describe the variants of subtle interaction in the literature, including claimed benefits, empirical approaches, and ethical considerations. Not only does this create a basis for thinking about subtleness as a quality of interaction, it also works to show how to solidify varieties of quality in HCI. We conclude by outlining some open empirical and conceptual questions about subtleness.

This paper provides resources and design recommendations for optimizing position input for pressure sensor matrices, a sensor design often used in eTextiles. Currently applications using pressure matrices for precise continuous position control are rare. One reason designers opt against using these sensors for continuous position control is that when the finger transitions from one sensing electrode to the next, jerky motion, jumps or other non-linear artifacts appear. We demonstrate that interdigitation can improve transition behavior and discuss interpolation algorithms to best leverage such designs. We provide software for reproducing our sensors and experiment, as well as a dataset consisting of 1122 swipe gestures performed on 17 sensors.

The notion of interaction is essential to human-computer interaction, yet rarely studied. We use quantitative and qualitative methods to investigate how this notion has been used across 35 years of proceedings from the ACM Conference on Human Factors in Computing (CHI). Using natural language processing, we extract 53,568 occurrences of the word “interaction” across 4,604 papers. In these occurrences, we categorize 2,668 unique words that modify how “interaction” is used in a sentence. We show that the use of “interaction” is both increasing and diversifying, suggesting the importance of the notion, but also the difficulty in developing theory about interaction. Our findings show that styles of interaction are closely associated with changes in technology and that modalities and characteristics of interaction are becoming more of a topic than specific devices or widgets. Interaction qualities, relating to structure, feel, effectiveness, and efficiency, are consistently prominent, and the quality of novelty is increasingly frequent. From this analysis, we identify open questions about interaction, including how to build knowledge across changing technologies, how to work toward a model of quality for interaction, and what the core of a science of interaction could be.

Many shape-changing interfaces use an array of actuated rods to create a display surface; each rod working as a pixel. However, this approach only supports pixel height manipulation and cannot produce more radical shape changes of each pixel (and thus of the display). Examples of such changes include non-horizontal pixels, pixels that overhang other pixels, or variable gaps between pixels. We present a concept for composing shape-changing interfaces by vertically stacking tilt-enabled modules. Together, stacking and tilting allow us to create a more diverse range of display surfaces than using arrays. We demonstrate this concept through TiltStacks, a shape-changing prototype built using stacked linear actuators and displays. Each tiltable module provides three degrees of freedom (z-movement, roll, and pitch); two more degrees of freedom are added through stacking modules (i.e., planar x- and y-movement).

Shape-changing interfaces have emerged as a new method for interacting with computers, using dynamic changes in a device's physical shape for input and output. With the advances of research into shape-changing interfaces, we see a need to synthesize the main, open research questions. The purpose of this synthesis is to formulate common challenges across the diverse fields engaged in shape-change research, to facilitate progression from single prototypes and individual design explorations to grander scientific goals, and to draw attention to challenges that come with maturity, including those concerning ethics, theory-building, and societal impact. In this article we therefore present 12 grand challenges for research on shape-changing interfaces, derived from a three-day workshop with 25 shape-changing interface experts with backgrounds in design, computer science, human-computer interaction, engineering, robotics, and material science.

Input on the skin is emerging as an interaction style. At CHI 2012, Coyle and colleagues identified an increase in the sense of agency (SoA) as one benefit of skin input. However, their study only compared skin input to button presses and has not, to our knowledge, been replicated. Therefore, we had 24 participants compare skin input to both button presses and touchpad input, measuring SoA using the Libet Clock paradigm. We replicate previous findings regarding increased SoA in skin versus button input and also find that SoA for skin is significantly increased compared to touchpad input. Interview data addressing subjective experience further support these findings. We discuss agency and the experiences associated with skin input, as well as differences to input with non-skin devices.

Advances in sensing technologies allow for using the forearm as a touch surface to give input to off-skin displays. However, it is unclear how users perceive the mapping between an on-skin input area and an off-skin display area. We empirically describe such mappings to improve on-skin interaction. We collected discrete and continuous touch data in a study where participants mapped display content from an AR headset, a smartwatch, and a desktop display to their forearm. We model those mappings and estimate input accuracy from the spreads of touch data. Subsequently, we show how to use the models for designing touch surfaces to the forearm for a given display area, input type, and touch resolution.

The concept of power pose originates from a Psychology study from 2010 which suggested that holding an expansive pose can change hormone levels and increase risk-taking behavior. Follow-up experiments suggested that expansive poses incidentally imposed by the design of an environment lead to more dishonest behaviors. While multiple replication attempts of the 2010 study failed, the follow-up experiments on incidental postures have so far not been replicated. As UI design in HCI can incidentally lead to expansive body postures, we attempted two conceptual replications: we first asked 44 participants to tap areas on a wall-sized display and measured their self-reported sense of power; we then asked 80 participants to play a game on a large touch-screen and measured risk-taking. Based on Bayesian analyses we find that incidental power poses had little to no effect on our measures but could cause physical discomfort. We conclude by discussing our findings in the context of theory-driven research in HCI.

Research on virtual reality (VR) has studied users' experience of immersion, presence, simulator sickness, and learning effects. However, the momentary experience of exiting VR and transitioning back to the real-world is not well understood. Do users become self-conscious of their actions upon exit? Are users nervous of their surroundings? Using explicitation interviews, we explore the moment of exit from VR across four applications. Analysis of the interviews reveals five components of experience: space, control, sociality, time, and sensory adaptation. Participants described spatial disorientation, for example, regardless of the complexity of the VR scene. Participants also described a window across which they exit VR, for example mentally first and then physically. We present six designs for easing or heightening the exit experience, as described by the participants. Based on these findings, we further discuss the ?moment of exit' as an opportunity for designing engaging and enhanced VR experiences.

The Sense of Agency (SoA) refers to our capability to control our own actions and influence the world around us. Recent research in HCI has been investigating SoA to provide users an instinctive sense of "I did that" as opposed to "the system did that". However, current agency measurements are limited. The Intentional Binding (IB) paradigm provides an implicit measure of the SoA, however, it is constrained by requiring high visual attention to a "Libet clock" on-screen. In this paper, we extended the timing stimuli through auditory and tactile cues. Our results demonstrate that audio timing through voice commands and haptic timing through tactile cues on the hand, are an effective alternative measure of the SoA using the IB paradigm. They both address current limitations of the traditional method such as visual attention overload and lack of engagement. We discuss how our results can be applied to measure SoA in tasks involving different interactive scenarios such as in Mixed/Virtual Reality.

We introduce the concept of Veritaps: a communication layer to help users identify truths and lies in mobile input. Existing lie detection research typically uses features not suitable for the breadth of mobile interaction. We explore the feasibility of detecting lies across all mobile touch interaction using sensor data from commodity smartphones. We report on three studies in which we collect discrete, truth-labelled mobile input using swipes and taps. The studies demonstrate the potential of using mobile interaction as a truth estimator by employing features such as touch pressure and the inter-tap details of number entry, for example. In our final study, we report an F1-score of .98 for classifying truths and .57 for lies. Finally we sketch three potential future scenarios of using lie detection in mobile applications; as a security measure during online log-in, a trust layer during online sale negotiations, and a tool for exploring self-deception.

Can we experience haptic textures in mid-air? Typically, the experience of texture is caused by vibration of the fingertip as it moves over the surface of an object. This object's surface also guides the finger's movement, creating an implicit motion-to-vibration mapping. If we wish to simulate a texture in mid-air, such guidance does not exist, making the choice of motion-to-vibration mapping non-obvious. We evaluate the experience of moving a pointer with four different motion-to vibration mappings in an interview study. We found that some mappings lead to a perception shift, transforming the experience. When this occurs, the pointer is no longer perceived as vibrating, interactions become more pleasurable, and users have an increased experience of agency and control. We discuss how to leverage this in the design of haptic interfaces.

We present zPatch: an eTextile patch for hover, touch, and pressure input, using both resistive and capacitive sensing. zPatches are made by layering a piezo-resistive material between silver-plated ripstop, and embedding it in non-conductive fabric to form a patch. zPatches can be easily ironed onto most fabrics, in any location, enabling easy prototyping or ad hoc modifications of existing garments. We provide open-source resources for building and programming zPatches and present measures of the achieva-ble sensing resolution of a zPatch. A pressure based targeting task demonstrated users could reliably hit pressure tar-gets at up to 13 levels, given appropriate feedback. We demonstrate that the hybrid sensing approach reduces false activations and helps distinguish between gestures. Finally, we present example applications in which we use zPatches for controlling a music player, text entry and gaming input.

Grabbing users' attention is a fundamental aspect of interactive systems. However, there is a disconnect between the ways our devices notify us and how our bodies do so naturally. In this paper, we explore the body's modality of itching as a way to provide such natural feedback. We create itching sensations via low-current electric stimulation, which allows us to quickly generate this sensation on demand. In a first study we explore the design space around itching and how changes in stimulation parameters influence the resulting sensation. In a second study we compare vibration feedback and itching integrated in a smartwatch form factor. We find that we can consistently induce itching sensations and that these are perceived as more activating and interrupting than vibrotactile stimuli.

In the article “The Usability Construct: A Dead End?”, Noam Tractinsky (2017) argues that the construct of usability is in dire straits. One reason is that usability is an umbrella construct, subsu...

An instructional approach is presented to improve human performance in solving Bayesian inference problems. Starting from the original text of the classic Mammography Problem, the textual expression is modified and visualizations are added according to Mayer’s principles of instruction. These principles concern coherence, personalization, signaling, segmenting, multimedia, spatial contiguity, and pretraining. Principles of self-explanation and interactivity are also applied. Four experiments on the Mammography Problem showed that these principles help participants answer the questions at significantly improved rates. Nonetheless, in novel interactivity conditions, performance was lowered suggesting that more interaction can add more difficulty for participants. Overall, a leap forward in accuracy was found, with more than twice the participant accuracy of previous work. This indicates that an instructional approach to improving human performance in Bayesian inference is a promising direction.

The human skin provides an ample, always-on surface for input to smart watches, mobile phones, and remote displays. Using touch on bare skin to issue commands, however, requires users to recall the location of items without direct visual feedback. We present an in-depth study in which participants placed 30 items on the hand and forearm and attempted to recall their locations. We found that participants used a variety of landmarks, personal associations, and semantic groupings in placing the items on the skin. Although participants most frequently used anatomical landmarks (e.g., fingers, joints, and nails), recall rates were higher for items placed on personal landmarks, including scars and tattoos. We further found that personal associations between items improved recall, and that participants often grouped important items in similar areas, such as family members on the nails. We conclude by discussing the implications of our findings for design of skin-based interfaces.

The term interaction is field-defining, yet surprisingly confused. This essay discusses what interaction is. We first argue that only few attempts to directly define interaction exist. Nevertheless, we extract from the literature distinct and highly developed concepts, for instance viewing interaction as dialogue, transmission, optimal behavior, embodiment, and tool use. Importantly, these concepts are associated with different scopes and ways of construing the causal relationships between the human and the computer. This affects their ability to inform empirical studies and design. Based on this discussion, we list desiderata for future work on interaction, emphasizing the need to improve scope and specificity, to better account for the effects and agency that computers have in interaction, and to generate strong propositions about interaction.

Vibrotactile actuation is mainly used to deliver buzzing sensations. But if vibrotactile actuation is tightly coupled to users' actions, it can be used to create much richer haptic experiences. It is not well understood, however, how this coupling should be done or which vibrotactile parameters create which experiences. To investigate how actuation parameters relate to haptic experiences, we built a physical slider with minimal native friction, a vibrotactile actuator and an integrated position sensor. By vibrating the slider as it is moved, we create an experience of texture between the sliding element and its track. We conducted a magnitude estimation experiment to map how granularity, amplitude and timbre relate to the experiences of roughness, adhesiveness, sharpness and bumpiness. We found that amplitude influences the strength of the perceived texture, while variations in granularity and timbre create distinct experiences. Our study underlines the importance of action in haptic perception and suggests strategies for deploying such tightly coupled feedback in everyday devices.

Usability has a distinct subjective component, yet surprisingly little is known about its neural basis and relation to the neuroanatomy of aesthetics. To begin closing this gap, we conducted two functional magnetic resonance imaging studies in which participants were shown static webpages (in the first study) and videos of interaction with webpages (in the second study). The webpages were controlled so as to exhibit high and low levels of perceived usability and perceived aesthetics. Our results show unique links between perceived usability and brain areas involved in functions such as emotional processing (left fusiform gyrus, superior frontal gyrus), anticipation of physical interaction (precentral gyrus), task intention (anterior cingulate cortex), and linguistic processing (medial and bilateral superior frontal gyri). We use these findings to discuss the brain correlates of perceived usability and the use of fMRI for usability evaluation and for generating new user experiences.

Many user studies are now conducted outside laboratories to increase the number and heterogeneity of participants. These studies are conducted in diverse settings, with the potential to give research greater external validity and statistical power at a lower cost. The feasibility of conducting virtual reality (VR) studies outside laboratories remains unclear because these studies often use expensive equipment, depend critically on the physical context, and sometimes study delicate phenomena concerning body awareness and immersion. To investigate, we explore pointing, 3D tracing, and body-illusions both in-lab and out-of-lab. The in-lab study was carried out as a traditional experiment with state-of-the-art VR equipment; 31 completed the study in our laboratory. The out-of-lab study was conducted by distributing commodity cardboard VR glasses to participants; 57 completed the study anywhere they saw fit. The effects found in-lab were comparable to those found out-of-lab, with much larger variations in the settings in the out-of-lab condition. A follow-up study showed that performance metrics are mostly governed by the technology used, where more complex VR phenomena depend more critically on the internal control of the study. We argue that conducting VR studies outside the laboratory is feasible, and that certain types of VR studies may advantageously be run this way. From the results, we discuss the implications and limitations of running VR studies outside the laboratory.

Understanding the mechanisms that shape the adoption and use of information technology is central to human--computer interaction. Two accounts are particularly vocal about these mechanisms, namely the technology acceptance model (TAM) and work on user experience (UX) models. In this study, we review 37 papers in the overlap between TAM and UX models to explore the experiential component of human--computer interactions. The models provide rich insights about what constructs influence the experiential component of human--computer interactions and about how these constructs are related. For example, the effect of perceived enjoyment on attitude is stronger than those of perceived usefulness and perceived ease of use. It is less clear why the relations exist and under which conditions the models apply. We discuss four of the main theories used in reasoning about the experiential component and, for example, point to the near absence of psychological needs and negative emotions in the models. In addition, most of the reviewed studies are not tied to specific use episodes, thereby bypassing tasks as an explanatory variable and undermining the accurate measurement of experiences, which are susceptible to moment-to-moment changes. We end by summarizing the implications of our review for future research.

Physical data representations, or data physicalizations, are a promising new medium to represent and communicate data. Previous work mostly studied passive physicalizations which require humans to perform all interactions manually. Dynamic shape-changing displays address this limitation and facilitate data exploration tasks such as sorting, navigating in data sets which exceed the fixed size of a given physical display, or preparing "views" to communicate insights about data. However, it is currently unclear how people approach and interact with such data representations. We ran an exploratory study to investigate how non-experts made use of a dynamic physical bar chart for an open-ended data exploration and presentation task. We asked 16 participants to explore a data set on European values and to prepare a short presentation of their insights using a physical display. We analyze: (1) users' body movements to understand how they approach and react to the physicalization, (2) their hand-gestures to understand how they interact with physical data, (3) system interactions to understand which subsets of the data they explored and which features they used in the process, and (4) strategies used to explore the data and present observations. We discuss the implications of our findings for the use of dynamic data physicalizations and avenues for future work.

The design of shape-changing interfaces to show emotions relies on craft skill with few clear guidelines. Through two experiments, we explore how to design such interfaces using theories of the relation between biological motion and affect. In the first experiment, 19 participants viewed six shape-changing behaviors that varied the velocity, fluidity, direction, and orientation of the movement of an extrusion from a small box in accordance with existing theories of affective motion. Participants were able to recognize four of the six intended basic Ekman emotions (sadness, fear, happiness, surprise) with above-chance probability. The second experiment used 36 shape-changing behaviors that systematically varied speed, regularity of motion, and direction. For each behavior, 23 participants rated valence, arousal, and dominance. Speed, direction, and orientation impacted emotion ratings significantly and in the predicted directions. These results offer an initial basis for the systematic design of emotions in shape-changing interfaces.

Wall-sized displays support group work by allowing several people to work both separately and together. However, whether people interact directly through touch input or indirectly through mouse input can have profound effects on collaboration. We present a study that compares how groups collaborate using either multitouch or multiple mice on a wall-display. Participants used both input methods to work on two tasks: a shared-goal task and a mixed-motive task. Results show differences in participants' awareness in collaborative tasks between the two input methods. The results also help understand the physical constraints touch input set on participants' control of actions in collaborative tasks. We discuss how this influences collaboration. Results also show that touch input did not promote more equal participation than mouse input. We contrast the findings to earlier research on wall-display and tabletop collaboration.

The size of information spaces often exceeds the limits of even the largest displays. This makes navigating such spaces through on-screen interactions demanding. However, if users imagine the information space extending in a plane beyond the display's boundaries, they might be able to use the space beyond the display for input. This paper investigates Off-Limits, an interaction concept extending the input space of a large display into the space beyond the screen through the use of mid-air pointing. We develop and evaluate the concept through three empirical studies in one-dimensional space: First, we explore benefits and limitations of off-screen pointing compared to touch interaction and mid-air on-screen pointing; next, we assess users' accuracy in off-screen pointing to model the distance-to-screen vs. accuracy trade-off; and finally, we show how Off-Limits is further improved by applying that model to the naïve approach. Overall, we found that the final Off-Limits concept provides significant performance benefits over on-screen and touch pointing conditions.

User experience (UX) research has expanded our notion of what makes interactive technology good, often putting hedonic aspects of use such as fun, affect, and stimulation at the center. Outside of UX, the hedonic is often contrasted to the eudaimonic, the notion of striving towards one's personal best. It remains unclear, however, what this distinction offers to UX research conceptually and empirically. We investigate a possible role for eudaimonia in UX research by empirically examining 266 reports of positive experiences with technology and analyzing its relation to established UX concepts. Compared to hedonic experiences, eudaimonic experiences were about striving towards and accomplishing personal goals through technology use. They were also characterized by increased need fulfillment, positive affect, meaning, and long-term importance. Taken together, our findings suggest that while hedonic UX is about momentary pleasures directly derived from technology use, eudaimonic UX is about meaning from need fulfilment.

This essay contributes a meta-scientific account of human-computer interaction (HCI) research as problem-solving. We build on the philosophy of Larry Laudan, who develops problem and solution as the foundational concepts of science. We argue that most HCI research is about three main types of problem: empirical, conceptual, and constructive. We elaborate upon Laudan's concept of problem-solving capacity as a universal criterion for determining the progress of solutions (outcomes): Instead of asking whether research is 'valid' or follows the 'right' approach, it urges us to ask how its solutions advance our capacity to solve important problems in human use of computers. This offers a rich, generative, and 'discipline-free' view of HCI and resolves some existing debates about what HCI is or should be. It may also help unify efforts across nominally disparate traditions in empirical research, theory, design, and engineering.

Shape-changing interfaces allow designers to create user interfaces that physically change shape. However, presently, we lack studies of how such interfaces are designed, as well as what high-level strategies, such as metaphors and affordances, designers use. This paper presents an analysis of sketches made by 21 participants designing either a shape-changing radio or a shape-changing mobile phone. The results exhibit a range of interesting design elements, and the analysis points to a need to further develop or revise existing vocabularies for sketching and analyzing movement. The sketches show a prevalent use of metaphors, say, for communicating volume though big-is-on and small-is-off, as well as a lack of conventions. Furthermore, the affordances used were curiously asymmetrical compared to those offered by non-shape-changing interfaces. We conclude by offering implications on how our results can influence future research on shape-changing interfaces.

We present TableHop, a tabletop display that provides controlled self-actuated deformation and vibro-tactile feedback to an elastic fabric surface while retaining the ability for high-resolution visual projection. The surface is made of a highly stretchable pure spandex fabric that is electrostatically actuated using electrodes mounted on its top or underside. It uses transparent indium tin oxide electrodes and high-voltage modulation to create controlled surface deformations. Our setup actuates pixels and creates deformations in the fabric up to +/- 5 mm. Since the electrodes are transparent, the fabric surface functions as a diffuser for rear-projected visual images, and avoid occlusion by users or actuators. Users can touch and interact with the fabric to experience expressive interactions as with any fabric based shape-changing interface. By using frequency modulation in the high-voltage circuit, it can also create localized tactile sensations on the user's fingertip when touching the surface. We provide simulation and experimental results for the shape of the deformation and frequency of the vibration of the surface. These results can be used to build prototypes of different sizes and form-factors. We present a working prototype of TableHop that has 30x40 cm2 surface area and uses a grid of 3x3 transparent electrodes. It uses a maximum of 9.46 mW and can create tactile vibrations of up to 20 Hz. TableHop can be scaled to make large interactive surfaces and integrated with other objects and devices. TableHop will improve user interaction experience on 2.5D deformable displays.

Research on shape-changing interfaces has explored various technologies, parameters for shape changes, and transformations between shapes. While much is known about how to implement these variations, it is unclear what affordance they provide, how users understand their relation to the underlying system state, and how feedback via shape change is perceived. We investigated this by studying how 15 participants perceived and used 13 shape-changing buttons. The buttons covered several aspects of affordance, system state, and feedback, including invite-to-touch movements, two styles of transition animation, and two actuation technologies. Participants explored and interacted with the buttons while thinking aloud. The results show that affordances are hard to communicate clearly with shape change; while some movements invited actions, others were seen as a malfunction. The best clue as to button state was provided by the position of the button in combination with vibration. Linear transition animation for changes in button state was the best received form of shape-change feedback. We discuss also how these findings can inform the design of shape-changing interfaces more generally.

Current techniques to computationally detect human affect often depend on specialized hardware, work only in laboratory settings, or require substantial individual training. We use sensors in commodity smartphones to estimate affect in the wild with no training time based on a link between affect and movement. The first experiment had 55 participants do touch interactions after exposure to positive or neutral emotion-eliciting films; negative affect resulted in faster but less precise interactions, in addition to differences in rotation and acceleration. Using off-the-shelf machine learning algorithms we report 89.1% accuracy in binary affective classification, grouping participants by their self-assessments. A follow up experiment validated findings from the first experiment; the experiment collected naturally occurring affect of 127 participants, who again did touch interactions. Results demonstrate that affect has direct behavioral effect on mobile interaction and that affect detection using common smartphone sensors is feasible.

Recent research suggests that psychological needs such as competence and relatedness are involved in users’ experience with technology and are related to the perception of a product's hedonic and pragmatic quality. This line of research, however, predominately focuses on positive leisure experiences, and it is unclear whether need fulfillment plays a similar role in negative experiences or in other activity domains such as work. Therefore, this study investigates need fulfillment in positive and negative experiences, and in work and leisure experiences in two separate studies by analyzing almost 600 users’ experiences with technology along with ratings on need fulfillment, affect, and perceived product quality. Results suggest that work and leisure experiences as well as positive and negative experiences differ in terms of need fulfillment. Hence, both activity domain and valence of experiences are important factors that should be taken in account when modeling user experience.

Physical visualizations, or data physicalizations, encode data in attributes of physical shapes. Despite a considerable body of work on visual variables, "physical variables" remain poorly understood. One of them is physical size. A difficulty for solid elements is that "size" is ambiguous - it can refer to either length/diameter, surface, or volume. Thus, it is unclear for designers of physicalizations how to effectively encode quantities in physical size. To investigate, we ran an experiment where participants estimated ratios between quantities represented by solid bars and spheres. Our results suggest that solid bars are compared based on their length, consistent with previous findings for 2D and 3D bars on flat media. But for spheres, participants' estimates are rather proportional to their surface. Depending on the estimation method used, judgments are rather consistent across participants, thus the use of perceptually-optimized size scales seems possible. We conclude by discussing implications for the design of data physicalizations and the need for more empirical studies on physical variables.

Physical movement plays an important role in interaction with wall-displays. Earlier work on its effect on performance has been inconclusive, however, because movement has not been experimentally controlled. In a first experiment, we controlled participants' ability to physically move in front of a 3-meter wide 24-megapixel wall-display. Participants performed a classification task involving navigation using a zoom-and-pan interface. Results suggest that the ability to move does not increase performance, and that a majority of participants used virtual navigation (i.e., zooming and panning) and little or no physical navigation (i.e., moving their bodies). To isolate the effects of physical and virtual navigation, a second experiment compared conditions where participants could navigate using either only physical movement or only virtual navigation. The second experiment showed that physical movement does benefit performance. The results from the experiments suggest that moving may not be improving performance, depending on the use of virtual navigation.

Physical representations of data have existed for thousands of years. Yet it is now that advances in digital fabrication, actuated tangible interfaces, and shape-changing displays are spurring an emerging area of research that we call Data Physicalization. It aims to help people explore, understand, and communicate data using computer-supported physical data representations. We call these representations physicalizations, analogously to visualizations -- their purely visual counterpart. In this article, we go beyond the focused research questions addressed so far by delineating the research area, synthesizing its open challenges and laying out a research agenda.

Visualizations such as bar charts help users reason about data, but are mostly screen-based, rarely physical, and almost never physical and dynamic. This paper investigates the role of physically dynamic bar charts and evaluates new interactions for exploring and working with datasets rendered in dynamic physical form. To facilitate our exploration we constructed a 10x10 interactive bar chart and designed interactions that supported fundamental visualisation tasks, specifically; annotation, filtering, organization, and navigation. The interactions were evaluated in a user study with 17 participants. Our findings identify the preferred methods of working with the data for each task i.e. directly tapping rows to hide bars, highlight the strengths and limitations of working with physical data, and discuss the challenges of integrating the proposed interactions together into a larger data exploration system. In general, physical interactions were intuitive, informative, and enjoyable, paving the way for new explorations in physical data visualizations.

Deformable interfaces offer new possibilities for gestures, some of which have been shown effective in controlled laboratory studies. Little work, however, has attempted to match deformable interfaces to a demanding domain and evaluate them out of the lab. We investigate how musicians use deformable interfaces to perform electronic music. We invited musicians to three workshops, where they explored 10 deformable objects and generated ideas on how to use these objects to perform music. Based on the results from the workshops, we implemented sensors in the five preferred objects and programmed them for controlling sounds. Next, we ran a performance study where six musicians performed music with these objects at their studios. Our results show that (1) musicians systematically map deformations to certain musical parameters, (2) musicians use deformable interfaces especially to filter and modulate sounds, and (3) musicians think that deformable interfaces embody the parameters that they control. We discuss what these results mean to research in deformable interfaces.

While current mobile devices detect the presence of surrounding devices, they lack a truly spatial awareness to bring them into the user's natural 3D space. We present Tracko, a 3D tracking system between two or more commodity devices without added components or device synchronization. Tracko achieves this by fusing three signal types. 1) Tracko infers the presence of and rough distance to other devices from the strength of Bluetooth low energy signals. 2) Tracko exchanges a series of inaudible stereo sounds and derives a set of accurate distances between devices from the difference in their arrival times. A Kalman filter integrates both signal cues to place collocated devices in a shared 3D space, combining the robustness of Bluetooth with the accuracy of audio signals for relative 3D tracking. 3) Tracko incorporates inertial sensors to refine 3D estimates and support quick interactions. Tracko robustly tracks devices in 3D with a mean error of 6.5 cm within 0.5 m and a 15.3 cm error within 1 m, which validates Trackoffs suitability for cross-device interactions.

Before using an interactive product, people form expectations about what the experience of use will be like. These expectations may affect both the use of the product and users’ attitudes toward it. This article briefly reviews existing theories of expectations to design and perform two crowdsourced experiments that investigate how expectations affect user experience measures. In the experiments, participants saw a primed or neutral review of a simple online game, played it, and rated it on various user experience measures. Results suggest that when expectations are confirmed, users tend to assimilate their ratings with their expectations; conversely, if the product quality is inconsistent with expectations, users tend to contrast their ratings with expectations and give ratings correlated with the level of disconfirmation. Results also suggest that expectation disconfirmation can be used more widely in analyses of user experience, even when the analyses are not specifically concerned with expectation disconfirmation.

This article investigates Herzberg's [1959] notion of hygienes, factors contributing to dissatisfaction but not to satisfaction, and motivators, factors contributing to satisfaction but not to dissatisfaction, in the context of user experience (UX). Earlier work has theorized that the notion of hygienes and motivators applies to UX but has neither shown empirical evidence for this theory nor exemplified what such factors would look like in UX. We adapt Herzberg's methodology to analyze 303 events where users felt good or bad about their smartphone and derive factors that may work as hygienes or motivators. We identified technical quality and price as hygienes, and utility and convenience as motivators. These factors do not correspond to those mentioned as typical examples of hygienes and motivators in the UX literature (i.e., instrumental qualities such as usability for hygienes and non-instrumental qualities such as beauty for motivators). We discuss this discrepancy in the context of pragmatic and hedonic quality and psychological need fulfillment.

Technologies for shape-changing user interfaces are rapidly evolving, but our understanding of the design space of such interfaces is still limited. We report a repertory grid study that aims to describe the design space from the users' point of view by eliciting personal constructs about shape-change. The study is based on six similar-sized, shape-changing artifacts that combine simple sensing of users with actuation that change volume, texture, and orientation. Our results show that the 18 respondents distinguish artifacts on dimensions that differ from those of most models of shape change. For instance, they characterize shape-change in terms of personality, territoriality, and state of mind, in addition to more common categories such as appearance and product properties. We discuss how the dimensions derived from users might be used to design shape-changing interfaces.

We present Mimic, an input capture and visual analytics system that records online user behavior to facilitate the discovery of micro-interactions that may affect problem understanding and decision making. As aggregate statistics and visualizations can mask important behaviors, Mimic can help interaction designers to improve the usability of their designs by going beyond aggregates to examine many individual user sessions in detail. To test Mimic, we replicate a recent crowd-sourcing experiment to better understand why participants consistently perform poorly in answering a canonical conditional probability question called the Mammography Problem. To analyze the micro-interactions, the Mimic web application is used to play back user sessions collected through remote logging of client-side events. We use Mimic to demonstrate the value of using advanced visual interfaces to interactively study interaction data. In the Mammography Problem, issues like user confusion, low confidence, and divided-attention were found based on participants changing their answers, doing repeated scrolling, and overestimating a base rate. Mimic shows how helpful detailed observational data can be and how important the careful design of micro-interactions is in helping users to successfully understand a problem, find a solution, and achieve their goals.

Many tasks performed in multiscale visual spaces require the user to have several foci. Using bimanual interaction, multitouch devices can facilitate the simultaneous definition and exploration of several foci. However, multitouch is rarely used for multifocus navigation, and may limit the interaction to a sequential definition of areas of interest. We introduce two novel navigation techniques that combine multiple foci and bimanual touch, and therefore enable the isochronic definition of areas of interest, leading to simultaneous multifocus navigation. SpaceFold folds the visual space in the third dimension, allowing users to bring objects closer to each other. Our technique enables a direct, bimanual manipulation of a folded space and therefore provides high flexibility. PhysicLenses uses multiple magnification lenses to compare objects. Using a physics model, PhysicLenses introduces a general solution for the arrangement of multiple lenses within the viewport. We conducted a controlled experiment with 24 participants to compare the techniques with split screen. The results show that SpaceFold significantly outperformed all other techniques, whereas PhysicLenses was just as fast as split screen.

Elastic, deformable displays allow users to give input by pinching, pushing, folding, and twisting the display. However, little is known about what gestures users prefer or how they will use elasticity and deformability as input. We report a guessability study where 17 participants performed gestures to solve 29 tasks, including selection, navigation, and 3D modeling. Based on the resulting 493 gestures, we describe a user-defined gesture set for elastic, deformable displays. We show how participants used depth and elasticity of the display to simulate deformation, rotation, and displacement of objects. In addition, we show how the use of desktop computers as well as multi-touch interaction affected users' choice of gestures. Finally, we discuss some unique uses of elasticity and deformability in gestures.

A replication is an attempt to confirm an earlier study's findings. It is often claimed that research in Human-Computer Interaction (HCI) contains too few replications. To investigate this claim we examined four publication outlets (891 papers) and found 3% attempting replication of an earlier result. The replications typically confirmed earlier findings, but treated replication as a confirm/not-confirm decision, rarely analyzing effect sizes or comparing in depth to the replicated paper. When asked, most authors agreed that their studies were replications, but rarely planned them as such. Many non-replication studies could have corroborated earlier work if they had analyzed data differently or used minimal effort to collect extra data. We discuss what these results mean to HCI, including how reporting of studies could be improved and how conferences/journals may change author instructions to get more replications.

Word-gesture keyboards enable fast text entry by letting users draw the shape of a word on the input surface. Such keyboards have been used extensively for touch devices, but not in mid-air, even though their fluent gestural input seems well suited for this modality. We present Vulture, a word-gesture keyboard for mid-air operation. Vulture adapts touch based word-gesture algorithms to work in mid-air, projects users' movement onto the display, and uses pinch as a word delimiter. A first 10-session study suggests text-entry rates of 20.6 Words Per Minute (WPM) and finds hand-movement speed to be the primary predictor of WPM. A second study shows that with training on a few phrases, participants do 28.1 WPM, 59% of the text-entry rate of direct touch input. Participants' recall of trained gestures in mid-air was low, suggesting that visual feedback is important but also limits performance. Based on data from the studies, we discuss improvements to Vulture and some alternative designs for mid-air text entry.

Technologies for capturing and generating smell are emerging, and our ability to engineer such technologies and use them in HCI is rapidly developing. Our understanding of how these technologies match the experiences with smell that people have or want to have is surprisingly limited. We therefore investigated the experience of smell and the emotions that accompany it. We collected stories from 439 participants who described personally memorable smell experiences in an online questionnaire. Based on the stories we developed 10 categories of smell experience. We explored the implications of the categories for smell-enhanced technology design by (a) probing participants to envision technologies that match their smell story and (b) having HCI researchers brainstorm technologies using the categories as design stimuli. We discuss how our findings can benefit research on personal memories, momentary and first time experiences, and wellbeing.

This paper investigates users' ability to perform force-sensitive tapping and explores its potential as an input modality in touch-based systems. We study force-sensitive tapping using Expressive Touch, a tabletop interface that infers tapping force from the sound waves created by the users' finger upon impact. The first part of the paper describes the implementation details of Expressive Touch and shows how existing tabletop interfaces can be augmented to reliably detect tapping force across the entire surface. The second part of the paper reports on the results of three studies of force-sensitive tapping. First, we use a classic psychophysic task to gain insights into participants' perception of tapping force (Study 1). Results show that although participants tap with different absolute tapping forces, they have a similar perception of relative tapping force. Second, we investigate participants' ability to control tapping force (Study 2) and find that users can produce two force levels with 99% accuracy. For six levels of force, accuracy drops to 58%. Third, we investigate the usability of force tapping by studying participants' reactions to seven force-sensitive touch applications (Study 3).

Shape-changing handheld devices are emerging as research prototypes, but it is unclear how users perceive them and which experiences they engender. The little data we have on user experience is from single prototypes, only covering a small part of the possibilities in shape change. We produce 51 videos of a shape-changing handheld device by systematically varying seven parameters of shape change. In a crowd-sourced study, 187 participants watched the videos and described their experiences using rating scales and free text. We find significant and large differences among parameters of shape change. Shapes that have previously been used for notifications were rated the least urgent; the degree of shape change was found to impact experience more than type of shape change. The experience of shape change was surprisingly complex: hedonic quality were inversely related to urgency, and some shapes were perceived as ugly, yet useful. We discuss how to advance models of shape change and improve research on the experience of shape change.

Whereas research in usability evaluation abounds, few evaluation approaches focus on utility. We present the utility inspection method (UIM), which prompts evaluators about the utility of the system they evaluate. The UIM asks whether a system uses global platforms, provides support infrastructure, is robust, gives access to rich content, allows customisation, offers symbolic value and supports companionship among users and between users and developers. We compare 47 participants’ use of UIM and heuristic evaluation (HE). The UIM helps identify more than three times as many problems as HE about the context of activities; HE helps identify 2.5 times as many problems as UIM about the interface. Usability experts consider the problems found with UIM more severe and more complex to solve compared to those found with HE. We argue that UIM complements existing usability evaluation methods and discuss future research on utility inspection.

Multitouch wall-sized displays afford new forms of collaboration: They can be used up close by several users simultaneously, offer high resolution, and provide sufficient space for intertwining individual and joint work. The difference to displays without these capabilities is not well understood. To better understand the collaboration of groups around high-resolution multitouch wall displays, we conducted an exploratory study. Pairs collaborated on a problem-solving task using a 2.8m × 1.2m multitouch display with 24.8 megapixels. The study examines how participants collaborate; navigate relative to the display and to each other; and interact with and share the display. Participants physically navigated among different parts of the display, switched fluidly between parallel and joint work, and shared the display evenly. The results contrast earlier research that suggests difficulties in sharing and collaborating around wall displays. The study suggests that multitouch wall displays can support different collaboration styles and fluid transitions in group work.

Recent research in user experience (UX) has studied narratives, users' account of their interaction with technology. It has emphasized specific constructs (e.g., affect, needs, hedonics) and their interrelation, but rarely analyzed the content of the narratives. We analyze the content and structure of 691 user-generated narratives on positive and negative experiences with technology. We use a multi-method approach consisting of manual (structural analysis of narratives) as well as of automated content analysis methods (psycholinguistic analysis and machine learning). These analyses show converging evidence that positive narratives predominantly concern social aspects such as family and friends. In addition, technology is positively experienced when it enables users to do things more efficiently or in a new way. In contrast, negative narratives often express anger and frustration due to technological failures. Our multi-method approach illustrates the potential of automated (as opposed to manual) content analysis methods for studying text-based experience reports.

The skyline operator has recently emerged as an alternative to ranking queries. It retrieves a number of potential best options for arbitrary monotone preference functions. The success of this operator in the database community is based on the belief that users benefit from the limited effort required to specify skyline queries compared to, for instance, ranking. While application examples of the skyline operator exist, there is no principled analysis of its benefits and limitations in data retrieval tasks. Our study investigates the degree to which users understand skyline queries, how they specify query parameters and how they interact with skyline results made available in listings or map-based interfaces.

In point-and-click interfaces the location of targets is sometimes known to the user before visually identifying it, and sometimes not. This experiment investigates how pointing is affected by whether the target location is precued so that users know it in advance or nonprecued so that users learn it only at the onset of pointing trials. The study investigates this for young, adult, and elderly participants pointing with mouse and touchpad. Target precuing affects the trial completion time, the reaction time, the sheer movement time, and multiple movement kinematics. In addition, target precuing interacts with the use of either mouse or touchpad, with target distance, and with target size, but surprisingly little with participant age. Because the target location was always made known to participants no later than at the onset of the pointing trial, the effects of target precuing must be due to the different possibilities for mental and motor preparations.

In controlled experiments on the relation of display size (i.e., the number of pixels) and the usability of visualizations, the size of the information space can either be kept constant or varied relative to display size. Both experimental approaches have limitations. If the information space is kept constant then the scale ratio between an overview of the entire information space and the lowest zoom level varies, which can impact performance; if the information space is varied then the scale ratio is kept constant, but performance cannot be directly compared. In other words, display size, information space, and scale ratio are interrelated variables. We investigate this relation in two experiments with interfaces that implement classic information visualization techniques-focus+context, overview+detail, and zooming-for multi-scale navigation in maps. Display size varied between 0.17, 1.5, and 13.8 megapixels. Information space varied relative to display size in one experiment and was constant in the other. Results suggest that for tasks where users navigate targets that are visible at all map scales the interfaces do not benefit from a large display: With a constant map size, a larger display does not improve performance with the interfaces; with map size varied relative to display size, participants found interfaces harder to use with a larger display and task completion times decrease only when they are normalized to compensate for the increase in map size. The two experimental approaches show different interaction effects between display size and interface. In particular, focus+context performs relatively worse at a large display size with variable map size, and relatively worse at a small display size with a fixed map size. Based on a theoretical analysis of the interaction with the visualization techniques, we examine individual task actions empirically so as to understand the relative impact of display size and scale ratio on the visualization techniques' performance and to discuss differences between the two experimental approaches.

People typically interact with information visualizations using a mouse. Their physical movement, orientation, and distance to visualizations are rarely used as input. We explore how to use such spatial relations among people and visualizations (i.e., proxemics) to drive interaction with visualizations, focusing here on the spatial relations between a single user and visualizations on a large display. We implement interaction techniques that zoom and pan, query and relate, and adapt visualizations based on tracking of users' position in relation to a large high-resolution display. Alternative prototypes are tested in three user studies and compared with baseline conditions that use a mouse. Our aim is to gain empirical data on the usefulness of a range of design possibilities and to generate more ideas. Among other things, the results show promise for changing zoom level or visual representation with the user's physical distance to a large display. We discuss possible benefits and potential issues to avoid when designing information visualizations that use proxemics.

Analysis is a key part of conducting usability evaluations, yet rarely systematically studied. Thus, we lack direction on how to do research on supporting practitioners' analysis and lose an opportunity for practitioners to learn from each other. We have surveyed 155 usability practitioners on the analysis in their latest usability evaluation. Analysis is typically flexible and light-weight. At the same time, practitioners see a need to strengthen reliability in evaluation. Redesign is closely integrated with analysis; more than half of the respondents provide visual redesign suggestions in their evaluation deliverables. Analysis support from academic research, including tools, forms and structured formats, does not seem to have direct impact on analysis practice. We provide six recommendations for future research to better support analysis.

Shape change is increasingly used in physical user interfaces, both as input and output. Yet, the progress made and the key research questions for shape-changing interfaces are rarely analyzed systematically. We review a sample of existing work on shape-changing interfaces to address these shortcomings. We identify eight types of shape that are transformed in various ways to serve both functional and hedonic design purposes. Interaction with shape-changing interfaces is simple and rarely merges input and output. Three questions are discussed based on the review: (a) which design purposes may shape-changing interfaces be used for, (b) which parts of the design space are not well understood, and (c) why studying user experience with shape-changing interfaces is important.

This paper reviews how empirical research on User Experience (UX) is conducted. It integrates products, dimensions of experience, and methodologies across a systematically selected sample of 51 publications from 2005-2009, reporting a total of 66 empirical studies. Results show a shift in the products and use contexts that are studied, from work towards leisure, from controlled tasks towards open use situations, and from desktop computing towards consumer products and art. Context of use and anticipated use, often named key factors of UX, are rarely researched. Emotions, enjoyment and aesthetics are the most frequently assessed dimensions. The methodologies used are mostly qualitative, and known from traditional usability studies, though constructive methods with unclear validity are being developed and used. Many studies use self-developed questionnaires without providing items or statistical validations. We discuss underexplored research questions and potential improvements of UX research.

Whereas the literature is clear on the benefits of large displays and visualizations, little is known about their combination, that is, how display size affect the usability of visualizations. We describe a controlled experiment where 19 participants used focus+context, overview+detail, and zooming techniques with varying display sizes (13.8, 1.5, and 0.17 megapixels). Participants navigated geographical maps to find specific locations, compare items, and follow routes. Results show that for multi-scale navigation, classic interactive visualization techniques did not benefit from being scaled to a large display: In contrast to the literature we find similar performance on medium and large displays. Across display sizes, overview+detail works the best, in particular for comparing items. Focus+context is relatively more difficult to use at a small display size. We explain these findings and discuss the design of interactive visualization techniques for large displays.

We present interaction techniques for tangible tabletop interfaces that use active, motorized tangibles, what we call Tangible Bots. Tangible Bots can reflect changes in the digital model and assist users by haptic feedback, by correcting errors, by multi-touch control, and by allowing efficient interaction with multiple tangibles. A first study shows that Tangible Bots are usable for fine-grained manipulation (e.g., rotating tangibles to a particular orientation); for coarse movements, Tangible Bots become useful only when several tangibles are controlled simultaneously. Participants prefer Tangible Bots and find them less taxing than passive, non-motorized tangibles. A second study focuses on usefulness by studying how electronic musicians use Tangible Bots to create music with a tangible tabletop application. We conclude by discussing the further potential of active tangibles, and their relative benefits over passive tangibles and multi-touch.

Whereas the concept of usability is predominantly defined analytically, people relate to systems through personal usability constructs. Based on 48 repertory-grid interviews, this study investigates how such personal constructs are affected by two factors crucial to the international development and uptake of systems: nationality (Chinese, Danish, or Indian) and stakeholder group (developer or user). We find no significant overall difference across nationalities, but further analyses suggest that conventional usability aspects such as ease of use and simplicity are prominent for Chinese and Danish but not Indian participants and that a distinction between work and leisure-related communication is central to Chinese and Indian but not Danish participants. For stakeholder groups, we find a significant overall difference between developers and users. Unlike developers, users associate ease of use with leisure and, conversely, difficulty in use with work-relatedness. Further, users perceive usefulness as related to frustration and separate from ease of use, whereas developers construe usefulness, fun, and ease of use as related. In construing usability, participants make use of several constructs that are not part of prevailing usability definitions, including usefulness, fun, and security.

To better support usability practice, most usability research focuses on evaluation methods. New ideas in usability research are mostly proposed as new evaluation methods. Many publications describe experiments that compare methods. Comparisons may indicate that some methods have important deficiencies, and thus often advise usability practitioners to prefer a specific method in a particular situation. An expectation persists in human–computer interaction (HCI) that results about evaluation methods should be the standard “unit of contribution” rather than favoring larger units (e.g., usability work as a whole) or smaller ones (e.g., the impact of specific aspects of a method). This article argues that these foci on comparisons and method innovations ignore the reality that usability evaluation methods are loose incomplete collections of resources, which successful practitioners configure, adapt, and complement to match specific project circumstances. Through a review of existing research on methods and resources, resources associated with specific evaluation methods, and ones that can complement existing methods, or be used separately, are identified. Next, a generic classification scheme for evaluation resources is developed, and the scheme is extended with project specific resources that impact the effective use of methods. With these reviews and analyses in place, implications for research, teaching, and practice are derived. Throughout, the article draws on culinary analogies. A recipe is nothing without its ingredients, and just as the quality of what is cooked reflects the quality of its ingredients, so too does the quality of usability work reflect the quality of resources as configured and combined. A method, like a recipe, is at best a guide to action for those adopting approaches to usability that are new to them. As with culinary dishes, HCI needs to focus more on what gets cooked, and how it gets cooked, and not just on how recipes suggest that it could be cooked.

Usability evaluation methods (UEMs) are widely recognised as an essential part of systems development. Assessments of the performance of UEMs, however, have been criticised for low validity and limited reliability. The present study extends this critique by describing seven dogmas in recent work on UEMs. The dogmas include using inadequate procedures and measures for assessment, focusing on win–lose outcomes, holding simplistic models of how usability evaluators work, concentrating on evaluation rather than on design and working from the assumption that usability problems are real. We discuss research approaches that may help move beyond the dogmas. In particular, we emphasise detailed studies of evaluation processes, assessments of the impact of UEMs on design carried out in real-world systems development and analyses of how UEMs may be combined.

Effects of age on pointing performance have become increasingly important as computers have become extensively used by still larger parts of the population. This study empirically investigates young (12–14 years), adult (25–33 years), and elderly (61–69 years) participants' performance when pointing with mouse and touchpad. The goal is to provide an integrated analysis of (a) how these three age groups differ in pointing performance, (b) how these differences are affected by the two pointing devices, and (c) how the submovement structure of cursor trajectories may explain performance differences. Results show that adult participants perform better than both young and elderly participants in that adult participants make fewer errors than young participants and complete trials quicker than elderly participants. Moreover, young participants are quicker than elderly participants, who make neither more nor less errors than young and adult participants. All three age groups were slower and made more errors with the touchpad than the mouse, but the touchpad slowed down elderly participants more than young participants, who in turn were slowed down more than adult participants. Adult participants made more efficient submovements than elderly participants; young participants had an intermediate position in that they were similar to adult participants for some submovement measures and similar to elderly participants for others.

Most research assumes that usability is understood similarly by users in different cultures, implying that the notion of usability, its aspects, and their interrelations are constant across cultures. The present study shows that this is not the case for a sample of 412 users from China and Denmark, who differ in how they understand and prioritize different aspects of usability. Chinese users appear to be more concerned with visual appearance, satisfaction, and fun than Danish users; Danish users prioritize effectiveness, efficiency, and lack of frustration higher than Chinese users. The results suggest that culture influences perceptions of usability. We discuss implications for usability research and for usability practice.

Information visualizations have been shown useful in numerous laboratory studies, but their adoption and use in real-life tasks are curiously under-researched. We present a field study of ten programmers who work with an editor extended with a fisheye view of source code. The study triangulates multiple methods (experience sampling, logging, thinking aloud, and interviews) to describe how the visualization is adopted and used. At the concrete level, our results suggest that the visualization was used as frequently as other tools in the programming environment. We also propose extensions to the interface and discuss features that were not used in practice. At the methodological level, the study identifies contributions distinct to individual methods and to their combination, and discusses the relative benefits of laboratory studies and field studies for the evaluation of information visualizations.

Tangible user interfaces for manipulating audio and music focus mostly on generating music on the spot, but rarely on how electronic musicians balance preparation and improvisation in staging live performances or on how the audience perceives the performances. We present mixiTUI, a tangible sequencer that allows electronic musicians to import and perform electronic music. mixiTUI is developed in collaboration with electronic musicians, with a focus on live arranging, on visualizations of music, on tokens that represent key elements in live performances, and on how the audience experiences the tangible interface. We present an evaluation of mixiTUI in a concert with 117 participants and argue that mixiTUI improves both the audience's and the musician's experience.

The format used to present feedback from usability evaluations to developers affects whether problems are understood, accepted, and fixed. Yet, little research has investigated which formats are the most effective. We describe an explorative study where three developers assess 40 usability findings presented using five feedback formats. Our usability findings comprise 35 problems and 5 positive comments. Data suggest that feedback serves multiple purposes. Initially, feedback must convince developers about the relevance of a problem and convey an understanding of this. Feedback must next be easy to use and finally serve as a reminder of the problem. Prior to working with the feedback, developers rated redesign proposals, multimedia reports, and annotated screen dumps as more valuable than lists of problems, all of which were rated as more valuable than scenarios. After having spent some time working with the feedback to address the usability problems, there were no significant differences among the developers' ratings of the value of the different formats. This suggests that all of the formats may serve equally well as reminders in later stages of working with usability problems, but that redesign proposals, multimedia reports, and annotated screen dumps best address the initial feedback goals convincing developers that a usability problem exists and of conveying an understanding of the problem.

We discuss the impact of cultural differences on usability evaluations that are based on the thinking-aloud method (TA). The term ‘cultural differences’ helps distinguish differences in the perception and thinking of Westerners (people from Western Europe and US citizens with European origins) and Easterners (people from China and the countries heavily influenced by its culture). We illustrate the impact of cultural cognition on four central elements of TA: (1) instructions and tasks, (2) the user’s verbalizations, (3) the evaluator’s reading of the user, and (4) the overall relationship between user and evaluator. In conclusion, we point to the importance of matching the task presentation to users’ cultural background, the different effects of thinking aloud on task performance between Easterners and Westerners, the differences in nonverbal behaviour that affect usability problem detection, and, finally, the complexity of the overall relationship between a user and an evaluator with different cultural backgrounds.

The utility and impact of a usability evaluation depend on how well its results align with the business goals of the system under evaluation. However, how to achieve such alignment is not well understood. We propose a simple technique that requires active consideration of a system's business goals in planning and reporting evaluations. The technique is tested in an experiment with 44 novice evaluators using think aloud testing. The evaluators considering business goals report fewer usability problems compared to evaluators that did not use the technique. The company commissioning the evaluation, however, assesses those problems 30-42% higher on four dimensions of utility. We discuss how the findings may generalize to usability professionals, and how the technique may be used in realistic usability evaluations. More generally, we discuss how our results illustrate one of a variety of ways in which business goals and other facets of a system's context may enter into usability evaluations.

The evaluator effect names the observation that usability evaluators in similar conditions identify substantially different sets of usability problems. Yet little is known about the factors involved in the evaluator effect. We present a study of 50 novice evaluators' usability tests and subsequent comparisons, in teams and individually, of the resulting usability problems. The same problems were analyzed independently by 10 human–computer interaction experts. The study shows an agreement between evaluators of about 40%, indicating a substantial evaluator effect. Team matching of problems following the individual matching appears to improve the agreement, and evaluators express greater satisfaction with the teams' matchings. The matchings of individuals, teams, and independent experts show evaluator effects of similar sizes; yet individuals, teams, and independent experts fundamentally disagree about which problems are similar. Previous claims in the literature about the evaluator effect are challenged by the large variability in the matching of usability problems; we identify matching as a key determinant of the evaluator effect. An alternative view of usability problems and evaluator agreement is proposed in which matching is seen as an activity that helps to make sense of usability problems and where the existence of a correct matching is not assumed.

Matching of usability problem descriptions consists of determining which problem descriptions are similar and which are not. In most comparisons of evaluation methods matching helps determine the overlap among methods and among evaluators. However, matching has received scant attention in usability research and may be fundamentally unreliable. We compare how 52 novice evaluators match the same set of problem descriptions from three think aloud studies. For matching the problem descriptions the evaluators use either (a) the similarity of solutions to the problems, (b) a prioritization effort for the owner of the application tested, (c) a model proposed by Lavery and colleagues [Lavery, D., Cockton, G., Atkinson, M.P., 1997. Comparison of evaluation methods using structured usability problem reports. Behaviour and Information Technology, 16 (4/5), 246–266], or (d) the User Action Framework [Andre, T.S., Hartson, H.R., Belz, S.M., McCreary, F.A., 2001. The user action framework: a reliable foundation for usability engineering support tools. International Journal of Human–Computer Studies, 54 (1), 107–136]. The resulting matches are different, both with respect to the number of problems grouped or identified as unique, and with respect to the content of the problem descriptions that were matched. Evaluators report different concerns and foci of attention when using the techniques. We illustrate how these differences among techniques might adversely influence the reliability of findings in usability research, and discuss some remedies.

Analyzing usability improvement processes as they take place in real-life organizations is necessary to understand the practice of usability work. This paper describes a case study where the usability of an information system is improved and a relationship between the improvements and the evaluation efforts is established. Results show that evaluation techniques complemented each other by suggesting different kinds of usability improvement. Among the techniques applied, a combination of questionnaires and Metaphors of Human Thinking (MOT) showed the largest mean impact and MOT produced the largest number of impacts. Logging of real-life use of the system over 6 months indicated six aspects of improved usability, where significant differences among evaluation techniques were found. Concerning five of the six aspects Think Aloud evaluations and the above-mentioned combination of questionnaire and MOT performed equally well, and better than MOT. Based on the evaluations 40 redesign proposals were developed and 30 of these were implemented. Four of the implemented redesigns where considered especially important. These evolved with inspiration from multiple evaluations and were informed by stakeholders with different kinds of expertise. Our results suggest that practitioners should not rely on isolated evaluations. Instead complementing techniques should be combined, and people with different expertise should be involved.

Usability inspection techniques are widely used, but few focus on users' thinking and many are appropriate only for particular devices and use contexts. We present a new technique (MOT) that guides inspection by metaphors of human thinking. The metaphors concern habit, the stream of thought, awareness and associations, the relation between utterances and thought, and knowing. The main novelty of MOT is its psychological basis combined with its use of metaphors to stimulate inspection. The first of three experiments shows that usability problems uncovered with MOT are more serious and more complex to repair than problems found with heuristic evaluation. Problems found with MOT are also judged more likely to persist for expert users. The second experiment shows that MOT finds more problems than cognitive walkthrough, and has a wider coverage of a reference collection of usability problems. Participants prefer using MOT over cognitive walkthrough; an important reason being the wider scope of MOT. The third experiment compares MOT, cognitive walkthrough, and think aloud testing, in the context of nontraditional user interfaces. Participants prefer using think aloud testing, but identify few problems with that technique that are not found also with MOT or cognitive walkthrough. MOT identifies more problems than the other techniques. Across experiments and measures of usability problems' utility in systems design, MOT performs better than existing inspection techniques and is comparable to think aloud testing.

Many applications require exploration of alternative scenarios; most support it poorly. Subjunctive interfaces provide mechanisms for the parallel setup, viewing and control of scenarios, aiming to support users' thinking about and interaction with their choices. We illustrate how applications for information access, real-time simulation, and document design may be extended with these mechanisms. To investigate the usability of this form of extension, we compare a simple census browser against a version with a subjunctive interface. In the first of three studies, subjects reported higher satisfaction with the subjunctive interface, and relied less on interim marks on paper. No reduction in task completion time was found, however, mainly because some subjects encountered problems in setting up and controlling scenarios. At the end of a second, five-session study, users of a redesigned interface completed tasks 27% more quickly than with the simple interface. In the third study we examined how subjects reasoned about multiple-scenario setups in pursuing complex, open-ended data explorations. Our main observation was that subjects treated scenarios as information holders, using them creatively in various ways to facilitate task completion.

Understanding the relation between usability measures seems crucial to deepen our conception of usability and to select the right measures for usability studies. We present a meta-analysis of correlations among usability measures calculated from the raw data of 73 studies. Correlations are generally low: effectiveness measures (e.g., errors) and efficiency measures (e.g., time) have a correlation of .247 ± .059 (Pearson's product-moment correlation with 95% confidence interval), efficiency and satisfaction (e.g., preference) one of .196 ± .064, and effectiveness and satisfaction one of .164 ± .062. Changes in task complexity do not influence these correlations, but use of more complex measures attenuates them. Standard questionnaires for measuring satisfaction appear more reliable than homegrown ones. Measures of users' perceptions of phenomena are generally not correlated with objective measures of the phenomena. Implications for how to measure usability are drawn and common models of usability are criticized.

Fisheye menus have become a prominent example of fisheye interfaces, yet contain several nonfisheye elements and have not been systematically evaluated. This study investigates whether fisheye menus are useful, and tries to untangle the impact on usability of the following properties of fisheye menus: use of distortion, index of letters for coarse navigation, and the focus-lock mode for accurate movement. Twelve participants took part in an experiment comparing fisheye menus with three alternative menu designs across known-item and browsing tasks, as well as across alphabetical and categorical menu structures. The results show that for finding known items, conventional hierarchical menus are the most accurate and by far the fastest. In addition, participants rate the hierarchical menu as more satisfying than fisheye and multifocus menus, but do not consistently prefer any one menu. For browsing tasks, the menus neither differ with respect to accuracy nor selection time. Eye-movement data show that participants make little use of nonfocus regions of the fisheye menu, though these are a defining feature of fisheye interfaces. Nonfocus regions are used more with the multifocus menu, which enlarges important menu items in these regions. With the hierarchical menu, participants make shorter fixations and have shorter scanpaths, suggesting lower requirements for mental activity and visual search. We conclude by discussing why fisheye menus are inferior to the hierarchical menu and how both may be improved.

Think-aloud testing is a widely employed usability evaluation method, yet its use in practice is rarely studied. We report an explorative study of 14 think-aloud sessions, the audio recordings of which were examined in detail. The study shows that immediate analysis of observations made in the think-aloud sessions is done only sporadically, if at all. When testing, evaluators seem to seek confirmation of problems that they are already aware of. During testing, evaluators often ask users about their expectations and about hypothetical situations, rather than about experienced problems. In addition, evaluators learn much about the usability of the tested system but little about its utility. The study shows how practical realities rarely discussed in the literature on usability evaluation influence sessions. We discuss implications for usability researchers and professionals, including techniques for fast-paced analysis and tools for capturing observations during sessions.

Navigating and understanding the source code of a program are highly challenging activities. This paper introduces a fisheye view of source code to a Java programming environment. The fisheye view aims to support a programmer's navigation and understanding by displaying those parts of the source code that have the highest degree of interest given the current focus. An experiment was conducted which compared the usability of the fisheye view with a common, linear presentation of source code. Sixteen participants performed tasks significantly faster with the fisheye view, although results varied dependent on the task type. The participants generally preferred the interface with the fisheye view. We analyse participants' interaction with the fisheye view and suggest how to improve its performance. In the calculation of the degree of interest, we suggest to emphasize those parts of the source code that are semantically related to the programmer's current focus.

Many tasks require users to extract information from diverse sources, to edit or process this information locally, and to explore how the end results are affected by changes in the information or in its processing. We present the RecipeSheet, a general-purpose tool for assisting users in such tasks. The RecipeSheet lets users create information processors, called recipes, which may take input in a variety of forms such as text, Web pages, or XML, and produce results in a similar variety of forms. The processing carried out by a recipe may be specified using a macro or query language, of which we currently support Rexx, Smalltalk and XQuery, or by capturing the behaviour of a Web application or Web service. In the RecipeSheet's spreadsheet-inspired user interface, information appears in cells, with inter-cell dependencies defined by recipes rather than formulas. Users can also intervene manually to control which information flows through the dependency connections. Through a series of examples we illustrate how tasks that would be challenging in existing environments are supported by the RecipeSheet.

Usability evaluations inform user interaction design in a relevant manner, and successful user interaction design can be attained through usability evaluation. These are obvious conjectures about a mature usability engineering discipline. Unfortunately, research and practice suggest that, in reality, the interplay between usability evaluation and user interaction design is significantly more complex and too often far from optimal. This article provides a simple model of the interplay between usability evaluation and user interaction design that captures their main relationships. From the model, what is seen as the key challenges in improving the interplay between evaluation and design is outlined. The intention is to create a background against which the remainder of this special issue, containing5researcharticlespresenting empirical data on the interplay between design and evaluation and a commentary, can be contrasted.

Usability problems predicted by evaluation techniques are useful input to systems development; it is uncertain whether redesign proposals aimed at alleviating those problems are likewise useful. We present a study of how developers of a large web application assess usability problems and redesign proposals as input to their systems development. Problems and redesign proposals were generated by 43 evaluators using an inspection technique and think aloud testing. Developers assessed redesign proposals to have higher utility in their work than usability problems. In interviews they explained how redesign proposals gave them new ideas for tackling well known problems. Redesign proposals were also seen as constructive and concrete input. Few usability problems were new to developers, but the problems supported prioritizing ongoing development of the application and taking design decisions. No developers, however, wanted to receive only problems or redesigns. We suggest developing and using redesign proposals as an integral part of usability evaluation.

Touchpad devices are widely used but lacking in pointing efficiency. The TouchGrid, an instance of what we term cell cursors, replaces moving the cursor through dragging the finger on a touchpad with tapping in different regions of the touchpad. The touchpad regions are recursively mapped to smaller display regions and thereby enable high-precision pointing without requiring high tapping precision. In an experiment, six subjects used the TouchGrid and a standard touchpad across different numbers of targets, distances to targets and target widths. Whereas standard touchpad operation follows Fitts' law, target selection time with the TouchGrid is a linear function of the required number of taps. The TouchGrid was significantly faster for small targets and for tasks requiring one tap, and marginally faster for two-tap tasks. Error rates tended to be higher with the TouchGrid than the standard touchpad. All subjects preferred the TouchGrid.

Many applications require comparison between alternative scenarios; most support it poorly. A subjunctive interface supports comparison through its facilities for parallel setup, viewing and control of scenarios. To evaluate the usability and benefits of these facilities, we ran experiments in which subjects used both a simple and a subjunctive interface to make comparisons in a census data set. In the first experiment, subjects reported higher satisfaction and lower workload with the subjunctive interface, and relied less on interim marks on paper. Subjects also used fewer interface actions. However, we found no reduction in task completion time, mainly because some subjects encountered problems in using the facilities for setting up and controlling scenarios. Based on a detailed analysis of subjects' actions we redesigned the subjunctive interface to alleviate frequent problems, such as accidentally adjusting only one scenario when the intention was to adjust them all. At the end of a second, five-session experiment, users of this redesigned interface completed tasks 27% more quickly than with the simple interface.

Many applications provide a form-like interface for requesting information: the user fills in some fields, submits the form, and the application presents corresponding results. Such a procedure becomes burdensome if (1) the user must submit many different requests, for example in pursuing a trial-and-error search, (2) results from one application are to be used as inputs for another, requiring the user to transfer them by hand, or (3) the user wants to compare results, but only the results from one request can be seen at a time. We describe how users can reduce this burden by creating custom interfaces using three mechanisms: clipping of input and result elements from existing applications to form cells on a spreadsheet; connecting these cells using formulas, thus enabling result transfer between applications; and cloning cells so that multiple requests can be handled side by side. We demonstrate a prototype of these mechanisms, initially specialised for handling Web applications, and show how it lets users build new interfaces to suit their individual needs.

A new usability inspection technique based on metaphors of human thinking has been experimentally compared to heuristic evaluation (HE). The aim of metaphors of thinking (MOT) is to focus inspection on users' mental activity and to make inspection easily applicable to different devices and use contexts. Building on classical introspective psychology, MOT bases inspection on metaphors of habit formation, stream of thought, awareness and associations, the relation between utterances and thought, and knowing. An experiment was conducted in which 87 novices evaluated a large Web application, and its key developer assessed the problems found. Compared to HE, MOT uncovered usability problems that were assessed as more severe for users and also appeared more complex to repair. The evaluators using HE found more cosmetic problems. The time spent learning and performing an evaluation with MOT was shorter. A discussion of strengths and weaknesses of MOT and HE is provided, which shows how MOT can be an effective alternative or supplement to HE.

Rotation of three-dimensional objects by a two-dimensional mouse is a typical task in computer-aided design, operation simulations, and desktop virtual reality. The most commonly used rotation technique is a virtual trackball surrounding the object and operated by the mouse pointer. This article reviews and provides a mathematical foundation for virtual trackballs. The first, but still popular, virtual trackball was described by Chen et al. We show that the virtual trackball by Chen et al. does not rotate the object along the intended great circular arc on the virtual trackball and we give a correction. Another popular virtual trackball is Shoemake's quaternion implementation, which we show to be a special case of the virtual trackball by Chen et al.. Shoemake extends the scope of the virtual trackball to the full screen. Unfortunately, Shoemake's virtual trackball is inhomogeneous and discontinuous with consequences for usability. Finally, we review Bell's virtual trackball and discuss studies of the usability of virtual trackballs.

We present an exploration of reading patterns and usability in visualizations of electronic documents. Twenty subjects wrote essays and answered questions about scientific documents using an overview+detail, a fisheye, and a linear interface. We study reading patterns by progression maps that visualize the progression of subjects' reading activity, and by visibility maps that show for how long different parts of the document are visible. The reading patterns help explain differences in usability between the interfaces and show how interfaces affect the way subjects read. With the overview+detail interface, subjects get higher grades for their essays. All but one of the subjects prefer this interface. With the fisheye interface, subjects use more time on gaining an overview of the document and less time on reading the details. Thus, they read the documents faster, but display lower incidental learning. We also show how subjects only briefly have visible the parts of the document that are not initially readable in the fisheye interface, even though they express a lack of trust in the algorithm underlying the fisheye interface. When answering questions, the overview is used for jumping directly to answers in the document and to already-visited parts of the document. However, subjects are slower at answering questions with the overview+detail interface. From the visualizations of the reading activity, we find that subjects using the overview+detail interface often explore the document further even when a satisfactory answer to the given question has already been read. Thus, overviews may grab subjects' attention and possibly distract them.

The literature on information visualization establishes the usability of interfaces with an overview of the information space, but for zoomable user interfaces, results are mixed. We compare zoomable user interfaces with and without an overview to understand the navigation patterns and usability of these interfaces. Thirty-two subjects solved navigation and browsing tasks on two maps. We found no difference between interfaces in subjects' ability to solve tasks correctly. Eighty percent of the subjects preferred the interface with an overview, stating that it supported navigation and helped keep track of their position on the map. However, subjects were faster with the interface without an overview when using one of the two maps. We conjecture that this difference was due to the organization of that map in multiple levels, which rendered the overview unnecessary by providing richer navigation cues through semantic zooming. The combination of that map and the interface without an overview also improved subjects' recall of objects on the map. Subjects who switched between the overview and the detail windows used more time, suggesting that integration of overview and detail windows adds complexity and requires additional mental and motor effort.

Reading of electronic documents is becoming increasingly important as more information is disseminated electronically. We present an experiment that compares the usability of a linear, a fisheye, and an overview+detail interface for electronic documents. Using these interfaces, 20 subjects wrote essays and answered questions about scientific documents. Essays written using the overview+detail interface received higher grades, while subjects using the fisheye interface read documents faster. However, subjects used more time to answer questions with the overview+detail interface. All but one subject preferred the overview+detail interface. The most common interface in practical use, the linear interface, is found to be inferior to the fisheye and overview+detail interfaces regarding most aspects of usability. We recommend using overview+detail interfaces for electronic documents, while fisheye interfaces mainly should be considered for time-critical tasks.

Usability comprises the aspects effectiveness, efficiency, and satisfaction. The correlations between these aspects are not well understood for complex tasks. We present data from an experiment where 87 subjects solved 20 information retrieval tasks concerning programming problems. The correlation between efficiency, as indicated by task completion time, and effectiveness, as indicated by quality of solution, was negligible. Generally, the correlations among the usability aspects depend in a complex way on the application domain, the user's experience, and the use context. Going through three years of CHI Proceedings, we find that 11 out of 19 experimental studies involving complex tasks account for only one or two aspects of usability. When these studies make claims concerning overall usability, they rely on risky assumptions about correlations between usability aspects. Unless domain specific studies suggest otherwise, effectiveness, efficiency, and satisfaction should be considered independent aspect of usability and all be included in usability testing.