Scrolling is ubiquitous in our daily computing experience. We explore how single-handed microgestures can be used for scrolling. Based on an analysis of the basic components necessary for scrolling, we selected 3 microgestures: Tap, Hold and Drag. Considering both rate and position controls, we designed 4 microgesture-based scrolling techniques adapted to these 3 microgestures. We contrasted these 4 techniques in a laboratory experiment with 24 participants who performed 2 tasks: a reciprocal selection task, where participants scrolled the view to reach and select a target; and a counting task, where participants scrolled the view to count image occurrences. Our results suggest that the technique based on Drag microgestures with rate control is the most effective for scrolling operations, regardless of the task. This work demonstrates that microgestures, with their advantages for frequent everyday tasks, offer a promising approach to continuous and efficient scrolling control.

While HCI research acknowledges that prior knowledge can be transferred from familiar interfaces to unfamiliar ones, we lack an understanding of which interface characteristics support this process. To adress this issue, we conducted three experiments to identify the interface characteristics that influence the perception of dis/similarities between software interfaces. The first, which involves a card-sorting activity, identifies seven intrinsic characteristics of interface. The second, conducted via an online pairwise comparison survey, identifies three characteristics inherent to the interface’s display context. Finally, the third experiment contrasts the ten identified characteristics to determine their respective influence on perceived interface dis/similarities. Altogether, our results provide actionable guidance for understanding how users perceive differences between interfaces and how such perceptions may inform or facilitate analogical transfer of knowledge from familiar to unfamiliar interfaces.

Critical reading is a primary way through which researchers develop their critical thinking skills. While exchanging thoughts and opinions with peers can strengthen critical reading, junior researchers often lack access to peers who can offer diverse perspectives. To address this gap, we designed an in-situ thought exchange interface informed by peer feedback from a formative study (N=8) to support junior researchers’ critical paper reading. We evaluated the effects of thought exchanges under three conditions (no-agent, single-agent, and multi-agent) with 46 junior researchers over two weeks. Our results showed that incorporating agent-mediated thought exchanges during paper reading significantly improved participants’ critical thinking scores compared to the no-agent condition. In the single-agent condition, participants more frequently made reflective annotations on the paper content. In the multi-agent condition, participants engaged more actively with agents’ responses. Our qualitative analysis further revealed that participants compared and analyzed multiple perspectives in the multi-agent condition. This work contributes to understanding in-situ AI-based support for critical paper reading through thought exchanges and offers design implications for future research.

Everyday users are confronted with increasingly complex technologies, be that novel introductions or ever-changing and expanding familiar interfaces. This creates the challenge of users needing to perpetually discover new functionality and interactivity, which is rarely addressed when novel technology is proposed. A possible contributing factor for this is the variety of, often inconsistently defined, associated concepts and a consequent lack of clarity on how to address these problems. We examine usage and definitions of discoverability as well as related concepts and offer clarifications where definitions are ambiguous while highlighting focus areas and limitations. In doing so, we provide a clear definition of discoverability, underscore the separation of system, feature and interaction discoverability and elucidate how other concepts focused on initial interactions differ.

Figure 1: Interaction signifiers are embedded in animated transitions to temporarily expose users to swipe-revealed hidden widgets, or swhidgets, as a means to increase their discoverability.The swhidget interaction is hidden in the application view (middle).A user then swipes an email away from the edge of the touch screen to reveal the hidden buttons.

Video tutorials are the main medium to learn novel software skills. However, the User Interface (UI) presented in a video tutorial may differ from the learner’s UI because of customizations or differences in software versions. We investigate the frictions resulting from such differences on a learners’ ability to reproduce a task demonstrated in a video tutorial. Through a morphological analysis, we first identify 13 types of “interface differences" that differ in terms of availability, reachability and spatial location of features in the interface. To better assess the frictions resulting from each of these differences, we then conduct a laboratory study with 26 participants instructed to reproduce a vector graphics editing task. Our results highlight interesting UI comparison behaviors, and illustrate various approaches employed to visually locate features.

We characterize and demonstrate how the principles of direct manipulation can improve interaction with large language models. This includes: continuous representation of generated objects of interest; reuse of prompt syntax in a toolbar of commands; manipulable outputs to compose or control the effect of prompts; and undo mechanisms. This idea is exemplified in DirectGPT, a user interface layer on top of ChatGPT that works by transforming direct manipulation actions to engineered prompts. A study shows participants were 50% faster and relied on 50% fewer and 72% shorter prompts to edit text, code, and vector images compared to baseline ChatGPT. Our work contributes a validated approach to integrate LLMs into traditional software using direct manipulation. Data, code, and demo available at https://osf.io/3wt6s.

We prototype and evaluate a desktop input and output device in the form of an architect desk lamp. The bulb is replaced with a pico laser projector, a button replaces the switch, and the mechanical design allows it to remain in position between user manipulations. By also tracking lamp position and orientation, we explore novel interaction techniques to extend and augment conventional devices in a physical desktop environment. Content can be transferred from devices to the surrounding environment, and the representation of content can be adapted to surfaces, objects, and other devices using the lamp projection target and lamp proximity. A user study with the prototype and semi-structured interviews examine proposed interactions and consider potential scenarios and applications. Based on the results, we propose further design considerations for direct manipulation systems to extend and augment desktop computing.

We explore the use of vibrotactile haptic cues for representing microgestures. We built a four-axes haptic device for providing vibrotactile cues mapped to all four fingers. We also designed six patterns, inspired by six most commonly studied microgestures. The patterns can be played independently on each axis of the device. We ran an experiment with 36 participants testing three different device locations (fingers, back of the hand, and forearm) for pattern and axis recognition. For all three device locations, participants interpreted the patterns with similar accuracy. We also found that they were better at distinguishing the axes when the device is placed on the fingers. Hand and Forearm device locations remain suitable alternatives but involve a greater trade-off between recognition rate and expressiveness. We report the recognition rates obtained for the different patterns, axes and their combinations per device location. These results per device location are important, as constraints of various kinds, such as hardware, context of use and user activities, influence device location. We discuss this choice of device location by improving literature microgesture-based scenarios with haptic feedback or feedforward.

Hand microgestures are promising for mobile interaction with wearable devices. However, they will not be adopted if practitioners cannot communicate to users the microgestures associated with the commands of their applications. This requires unambiguous representations that simultaneously show the multiple microgestures available to control an application. Using a systematic approach, we evaluate how these representations should be designed and contrast 4 conditions depending on the microgestures (tap-swipe and tap-hold) and fingers (index and index-middle) considered. Based on the results, we design a simultaneous representation of microgestures for a given set of 14 application commands. We then evaluate the usability of the representation for novice users and the suitability of the representation for small screens compared with a baseline. Finally, we formulate 8 recommendations based on the results of all the experiments. In particular, redundant graphical and textual representations of microgestures should only be displayed for novice users.

Pointing-based interaction interferences are situations wherein GUI elements appear, disappear, or change shortly before being selected, and too late for the user to inhibit their movement. Their cause lays in the design of most GUIs, for which any user event on an interactive element unquestionably reflects the user’s intention—even one millisecond after that element has changed. Previous work indicate that interferences can cause frustration and sometimes severe consequences. This article investigates new default behaviors for GUI elements that aim to prevent the occurrences of interferences or to mitigate their consequences. We present a design space of the advantages and technical requirements of these behaviors and demonstrate in a controlled study how simple rules can reduce the occurrences of so-called “ Pop-up -style” interferences and user frustration. We then discuss their application to various forms of interaction interferences. We conclude by addressing the feasibility and trade-offs of implementing these behaviors in existing systems.

Visual content must be labeled to facilitate navigation and retrieval, or provide ground truth data for supervised machine learning approaches. The efficiency of labeling techniques is crucial to produce numerous qualitative labels, but existing techniques remain sparsely evaluated. We systematically evaluate the efficiency of tagging and browsing tasks in relation to the number of images displayed, interaction modes, and the image visual complexity. Tagging consists in focusing on a single image to assign multiple labels (image-oriented strategy), and browsing in focusing on a single label to assign to multiple images (label-oriented strategy). In a first experiment, we focus on the nudges inducing participants to adopt one of the strategies (n=18). In a second experiment, we evaluate the efficiency of the strategies (n=24). Results suggest an image-oriented strategy (tagging task) leads to shorter annotation times, especially for complex images, and participants tend to adopt it regardless of the conditions they face.

Extracting underlying data from rasterized charts is tedious and inaccurate; values might be partially occluded or hard to distinguish, and the quality of the image limits the precision of the data being recovered. To address these issues, we introduce a semi-automatic system leveraging vector charts to extract the underlying data easily and accurately. The system is designed to make the most of vector information by relying on a drag-and-drop interface combined with selection, filtering, and previsualization features. A user study showed that participants spent less than 4 minutes to accurately recover data from charts published at CHI with diverse styles, thousands of data points, a combination of different encodings, and elements partially or completely occluded. Compared to other approaches relying on raster images, our tool successfully recovered all data, even when hidden, with a 78% lower relative error.

Documents often have paragraphs packed with numbers that are difficult to extract, compare, and interpret. To help readers make sense of data in text, we introduce the concept of Charagraphs: dynamically generated interactive charts and annotations for in-situ visualization, comparison, and manipulation of numeric data included within text. Three Charagraph characteristics are defined: leveraging related textual information about data; integrating textual and graphical representations; and interacting at different contexts. We contribute a document viewer to select in-text data; generate and customize Charagraphs; merge and refine a Charagraph using other in-text data; and identify, filter, compare, and sort data synchronized between text and visualization. Results of a study show participants can easily create Charagraphs for diverse examples of data-rich text, and when answering questions about data in text, participants were more correct compared to only reading text.

Inferential statistics are typically reported using p-values (NHST) or confidence intervals on effect sizes (estimation). This is done using a range of styles, but some readers have preferences about how statistics should be presented and others have limited familiarity with alternatives. We propose a system to interactively translate statistical reporting styles in existing documents, allowing readers to switch between interval estimates, p-values, and standardized effect sizes, all using textual and graphical reports that are dynamic and user customizable. Forty years of CHI papers are examined. Using only the information reported in scientific documents, equations are derived and validated on simulated datasets to show that conversions between p-values and confidence intervals are accurate. The system helps readers interpret statistics in a familiar style, compare reports that use different styles, and even validate the correctness of reports. Code and data: https://osf.io/x4ue7

The representations of microgestures are essentials for researchers presenting their results through academic papers and system designers proposing tutorials to novice users. However, those representations remain disparate and inconsistent. As a first attempt to investigate how to best graphically represent microgestures, we created 21 designs, each depicting static and dynamic versions of 4 commonly used microgestures (tap, swipe, flex and hold). We first studied these designs in a quantitative online experiment with 45 participants. We then conducted a qualitative laboratory experiment in Augmented Reality with 16 participants. Based on the results, we provide design guidelines on which elements of a microgesture should be represented and how. In particular, it is recommended to represent the actuator and the trajectory of a microgesture. Also, although preferred by users, dynamic representations are not considered better than their static counterparts for depicting a microgesture and do not necessarily result in a better user recognition.

Touch screens supporting different inputs such as 'Tap', 'Dwell', 'Double Tap' and 'Force Press' are omnipresent in modern devices and yet this variety of interaction opportunities is rarely communicated to the user. Without visual signifiers, these potentially useful inputs remain unknown or underutilised. We propose a design space of visual signifier characteristics that may impact the perception of in-place one finger inputs. We generated 36 designs and investigated their perception in an online survey (N=32) and an interactive experiment (N=24). The results suggest that visual signifiers increase the perception of input possibilities beyond 'Tap', and reduce perceived mental effort for participants, who also prefer added visual signifiers over a baseline. Our work informs how future touch-based interfaces could be designed to better communicate in-place single finger input possibilities.

We advocate for the usage of hotkeys on touch-based devices by capitalising on soft keyboards through four studies. First, we evaluated visual designs and recommended icons with command names for novices while letters with command names for experts. Second, we investigated the discoverability by asking crowdworkers to use our prototype, with some tasks only doable upon successfully discovering the technique. Discovery rates were high regardless of conditions that vary the familiarity and saliency of modifier keys. However, familiarity with desktop hotkeys boosted discoverability. Our third study focused on how prior knowledge of hotkeys could be leveraged and resulted in a 5% selection time improvement and identified the role of spatial memory in retention. Finally, we compared our soft keyboard layout with a grid layout similar to FastTap. The latter offered a 12–16% gain on selection speed, but at a high cost in terms of screen estate and low spatial stability.

Static illustrations are ubiquitous means to represent interaction scenarios. Across papers and reports, these visuals demonstrate people’s use of devices, explain systems, or show design spaces. Creating such figures is challenging, and very little is known about the overarching strategies for visually representing interaction scenarios. To mitigate this task, we contribute a unified taxonomy of design elements that compose such figures. In particular, we provide a detailed classification of Structural and Interaction strategies, such as composition, visual techniques, dynamics, representation of users, and many others – all in context of the type of scenarios. This taxonomy can inform researchers’ choices when creating new figures, by providing a concise synthesis of visual strategies, and revealing approaches they were not aware of before. Furthermore, to support the community for creating further taxonomies, we also provide three open-source software facilitating the coding process and visual exploration of the coding scheme.

Pointing transfer functions remain predominantly expressed in pixels per input counts, which can generate different visual pointer behaviors with different input and output devices; we show in a first controlled experiment that even small hardware differences impact pointing performance with functions defined in this manner. We also demonstrate the applicability of “hardware-independent” transfer functions defined in physical units. We explore two methods to maintain hardware-independent pointer performance in operating systems that require hardware-dependent definitions: scaling them to the resolutions of the input and output devices, or selecting the OS acceleration setting that produces the closest visual behavior. In a second controlled experiment, we adapted a baseline function to different screen and mouse resolutions using both methods, and the resulting functions provided equivalent performance. Lastly, we provide a tool to calculate equivalent transfer functions between hardware setups, allowing users to match pointer behavior with different devices, and researchers to tune and replicate experiment conditions. Our work emphasizes, and hopefully facilitates, the idea that operating systems should have the capability to formulate pointing transfer functions in physical units, and to adjust them automatically to hardware setups.

People increasingly access cross-device applications from their smartphones while on the go. Yet, they do not fully use the mobile versions for complex tasks, preferring the desktop version of the same application. We conducted a survey (N=77) to identify challenges when switching back and forth between devices. We discovered significant cross-device learnability issues, including that users often find exploring the mobile version frustrating, which leads to prematurely giving up on using the mobile version. Based on the findings, we created four design concepts as video prototypes to explore how to support cross-device learnability. The concepts vary in four key dimensions: the device involved, automation, temporality, and learning approach. Interviews (N=20) probing the design concepts identified individual differences affecting cross-device learning preferences, and that users are more motivated to use cross-device applications when offered the right cross-device learnability support. We conclude with future design directions for supporting seamless cross-device learnability.

Delayed display of menu items is a core design component of marking menus, arguably to prevent visual distraction and foster the use of mark mode. We investigate these assumptions, by contrasting the original marking menu design with immediately-displayed marking menus. In three controlled experiments, we fail to reveal obvious and systematic performance or usability advantages to using delay and mark mode. Only in very constrained settings – after significant training and only two items to learn – did traditional marking menus show a time improvement of about 260~ms. Otherwise, we found an overall decrease in performance with delay, whether participants exhibited practiced or unpracticed behaviour. Our final study failed to demonstrate that an immediately-displayed menu interface is more visually disrupting than a delayed menu. These findings inform the costs and benefits of incorporating delay in marking menus, and motivate guidelines for situations in which its use is desirable.

Documents such as presentations, instruction manuals, and research papers are disseminated using various file formats, many of which barely support the incorporation of interactive content. To address this lack of interactivity, we present Chameleon, a system-wide tool that combines computer vision algorithms used for image identification with an open database format to allow for the layering of dynamic content. Using Chameleon, static documents can be easily upgraded by layering user-generated interactive content on top of static images, all while preserving the original static document format and without modifying existing applications. We describe the development of Chameleon, including the design and evaluation of vision-based image replacement algorithms, the new document-creation pipeline as well as a user study evaluating Chameleon.

Touch-based devices, despite their mainstream availability, do not support a unified and efficient command selection mechanism, available on every platform and application. We advocate that hotkeys, conventionally used as a shortcut mechanism on desktop computers, could be generalized as a command selection mechanism for touch-based devices, even for keyboard-less applications. In this paper, we investigate the performance and usage of soft keyboard shortcuts or hotkeys (abbreviated SoftCuts) through two studies comparing different input methods across sitting, standing and walking conditions. Our results suggest that SoftCuts not only are appreciated by participants but also support rapid command selection with different devices and hand configurations. We also did not find evidence that walking deters their performance when using the Once input method.

We study interaction interferences, situations where an unexpected change occurs in an interface immediately before the user performs an action, causing the corresponding input to be misinterpreted by the system. For example, a user tries to select an item in a list, but the list is automatically updated immediately before the click, causing the wrong item to be selected. First, we formally define interaction interferences and discuss their causes from behavioral and system-design perspectives. Then, we report the results of a survey examining users' perceptions of the frequency, frustration, and severity of interaction interferences. We also report a controlled experiment, based on state-of-the-art experimental protocols from neuroscience, that explores the minimum time interval, before clicking, below which participants could not refrain from completing their action. Finally, we discuss our findings and their implications for system design, paving the way for future work.

Revealing a hidden widget with a dedicated sliding gesture is a common interaction design in today's handheld devices. Such "Swhidgets" (for swipe-revealed hidden widgets) provide a fast (and sometime unique) access to some commands. Interestingly, swhidgets do not follow conventional design guidelines in that they have no explicit signifiers, and users have to discover their existence before being able to use them. In this paper, we discuss the benefits of this signifierless design and investigate how iOS users deal with this type of widgets. We report on the results of a laboratory study and an online survey, investigating iOS users' experience with swhidgets. Our results suggest that swhidgets~are moderately but unevenly known by participants, yet the awareness and the discovery issues of this design is worthy of further discussion.

End-to-end latency corresponds to the temporal difference between a user input and the corresponding output from a system. It has been shown to degrade user performance in both direct and indirect interaction. If it can be reduced to some extend, latency can also be compensated through software compensation by trying to predict the future position of the cursor based on previous positions, velocities and accelerations. In this paper, we propose a hybrid hardware and software prediction technique specifically designed for partially compensating end-to-end latency in indirect pointing. We combine a computer mouse with a high frequency accelerometer to predict the future location of the pointer using Euler based equations. Our prediction method results in more accurate prediction than previously introduced prediction algorithms for direct touch. A controlled experiment also revealed that it can improve target acquisition time in pointing tasks.

Despite the ubiquity of touch-based input and the availability of increasingly computationally powerful touchscreen devices, there has been comparatively little work on enhancing basic canonical gestures such as swipe-to-pan and pinch-to-zoom. In this paper, we introduce transient pan and zoom, i.e. pan and zoom manipulation gestures that temporarily alter the view and can be rapidly undone. Leveraging typical touchscreen support for additional contact points, we design our transient gestures such that they co-exist with traditional pan and zoom interaction. We show that our transient pan-and-zoom reduces repetition in multi-level navigation and facilitates rapid movement between document states. We conclude with a discussion of user feedback, and directions for future research.

Text selection on touch devices can be a difficult task for users. Letters and words are often too small to select directly, and the enhanced interaction techniques provided by the OS -- magnifiers, selection handles, and methods for selecting at the character, word, or sentence level -- often lead to as many usability problems as they solve. The introduction of force-sensitive touchscreens has added another enhancement to text selection (using force for different selection modes); however, these modes are difficult to discover and many users continue to struggle with accurate selection. In this paper we report on an investigation of the design of touch-based and force-based text selection mechanisms, and describe two novel text-selection techniques that provide improved discoverability, enhanced visual feedback, and a higher performance ceiling for experienced users. Two evaluations show that one design successfully combined support for novices and experts, was never worse than the standard iOS technique, and was preferred by participants.

Large displays are becoming commonplace at work, at home, or in public areas. However, interaction at a distance -- anything greater than arms-length -- remains cumbersome, restricts simultaneous use, and requires specific hardware augmentations of the display: touch layers, cameras, or dedicated input devices. Yet a rapidly increasing number of people carry smartphones and smartwatches, devices with rich input capabilities that can easily be used as input devices to control interactive systems. We contribute (1) the results of a survey on possession and use of smart devices, and (2) the results of a controlled experiment comparing seven distal pointing techniques on phone or watch, one- and two-handed, and using different input channels and mappings. Our results favor using a smartphone as a trackpad, but also explore performance tradeoffs that can inform the choice and design of distal pointing techniques for different contexts of use.

Operating systems support autoscroll to allow users to scroll a view while in dragging mode: the user moves the pointer near the window's edge to trigger an "automatic" scrolling whose rate is typically proportional to the distance between the pointer and the window's edge. This approach suffers from several problems, especially when the window is maximized, resulting in a very limited space around it. Another problem is that for some operations, such as object drag-and-drop, the source and destination might be located in different windows, making it complicated for the computer to understand user's intention. In this paper, we present ForceEdge, a novel autoscroll technique relying on touch surfaces with force-sensing capabilities to alleviate the problems related to autoscroll. We report on the results of three controlled experiments showing that it improves over macOS and iOS systems baselines for top-to-bottom select and move tasks.

We propose a novel perspective on the design of toolbar buttons that aims to increase keyboard shortcut accessibility. IconHK implements this perspective by blending visual cues that convey keyboard shortcut information into toolbar buttons without denaturing the pictorial representation of their command. We introduce three design strategies to embed the hotkey, a visual encoding to convey the modifiers, and a magnification factor that determines the blending ratio between the pictogram of the button and the visual representation of the keyboard shortcut. Two studies examine the benefits of IconHK for end-users and provide insights from professional designers on the practicality of our approach for creating iconsets. Building on these insights, we develop a tool to assist designers in applying the IconHK design principle.

HCI researchers lack low latency and robust systems to support the design and development of interaction techniques using finger identification. We developed a low cost prototype using piezo based vibration sensors attached to each finger. By combining the events from an input device with the information from the vibration sensors we demonstrate how to achieve low latency and robust finger identification. Our prototype was evaluated in a controlled experiment, using two keyboards and a touchpad, showing recognition rates of 98.2% for the keyboard and, for the touchpad, 99.7% for single touch and 94.7% for two simultaneous touches. These results were confirmed in an additional laboratory style experiment with ecologically valid tasks. Last we present new interactions techniques made possible using this technology.

We investigate the potential benefits of actuated devices for the desktop workstation which remains the most used environment for daily office works. A formative study reveals that the desktop workstation is not a fixed environment because users manually change the position and the orientation of their devices. Based on these findings, we present the LivingDesktop, an augmented desktop workstation with devices (mouse, keyboard, monitor) capable of moving autonomously. We describe interaction techniques and applications illustrating how actuated desktop workstations can improve ergonomics, foster collaboration, leverage context and reinforce physicality. Finally, the findings of a scenario evaluation are (1) the perceived usefulness of ergonomics and collaboration applications; (2) how the LivingDesktop inspired our participants to elaborate novel accessibility and social applications; (3) the location and user practices should be considered when designed actuated desktop devices.

Edge-scrolling allows users to scroll a viewport while simultaneously dragging near or beyond a window's edge. Common implementations rely on rate control, mapping the distance between the pointer and the edge of the viewport to the scrolling velocity. While ubiquitous in operating systems, edge-scrolling has received little attention, even though previous works suggest that (1) rate control may be suboptimal for isotonic pointing devices like mice and trackpads and (2) space beyond the window's edge might be scarce, limiting scrolling control. To address these problems, we developed Push-edge scrolling (and Slide-edge scrolling, its inertial variant), two novel position-based techniques that allow scrolling by "pushing" against the viewport edge. A controlled experiment shows that our techniques reduce overshoots and offer performance improvements by up to 13% over traditional edge-scrolling.

As news is increasingly accessed on smartphones and tablets, the need for personalising news app interactions is apparent. We report a series of three studies addressing key issues in the development of adaptive news app interfaces. We first surveyed users' news reading preferences and behaviours; analysis revealed three primary types of reader. We then implemented and deployed an Android news app that logs users' interactions with the app. We used the logs to train a classifier and showed that it is able to reliably recognise a user according to their reader type. Finally we evaluated alternative, adaptive user interfaces for each reader type. The evaluation demonstrates the differential benefit of the adaptation for different users of the news app and the feasibility of adaptive interfaces for news apps.

Touch-based tablet UIs provide few shortcut mechanisms for rapid command selection; as a result, command selection on tablets often requires slow traversal of menus. We developed a new selection technique for multi-touch tablets, called FastTap, that uses thumb-and-finger touches to show and choose from a spatially-stable grid-based overlay interface. FastTap allows novices to view and inspect the full interface, but once item locations are known, FastTap allows people to select commands with a single quick thumb-and-finger tap. The interface helps users develop expertise, since the motor actions carried out as a novice rehearse the expert behavior. A controlled study showed that FastTap was significantly faster (by 33% per selection overall) than marking menus, both for novices and experts, and without reduction in accuracy or subjective preference. Our work introduces a new and efficient selection mechanism that supports rapid command execution on touch tablets, for both novices and experts.

Keyboard shortcuts allow fast interaction, but they are known to be infrequently used, with most users relying heavily on traditional pointer-based selection for most commands. We describe the goals, design, and evaluation of ExposeHK, a new interface mechanism that aims to increase hotkey use. ExposeHK's four key design goals are: 1) enable users to browse hotkeys; 2) allow non-expert users to issue hotkey commands as a physical rehearsal of expert performance; 3) exploit spatial memory to assist non-expert users in identifying hotkeys; and 4) maximise expert performance by using consistent shortcuts in a flat command hierarchy. ExposeHK supports these objectives by displaying hotkeys overlaid on their associated commands when a modifier key is pressed. We evaluated ExposeHK in three empirical studies using toolbars, menus, and a tabbed \'18ribbon' toolbar. Results show that participants used more hotkeys, and used them more often, with ExposeHK than with other techniques; they were faster with ExposeHK than with either pointing or other hotkey methods; and they strongly preferred ExposeHK. Our research shows that ExposeHK can substantially improve the user's transition from a \'18beginner mode' of interaction to a higher level of expertise.

We propose Augmented Letters, a new technique aimed at augmenting gesture-based techniques such as Marking Menus [9] by giving them natural, mnemonic associations. Augmented Letters gestures consist of the initial of command names, sketched by hand in the Unistroke style, and affixed with a straight tail. We designed a tentative touch device interaction technique that supports fast interactions with large sets of commands, is easily discoverable, improves user's recall at no speed cost, and supports fluid transition from novice to expert mode. An experiment suggests that Augmented Letters outperform Marking Menu in terms of user recall.

Relative spatial consistency - that is, the stable arrangement of objects in a 2D presentation - provides several benefits for interactive interfaces. Spatial consistency allows users to develop memory of object locations, reducing the time needed for visual search, and because spatial memory is long lasting and has a large capacity these performance benefits are enduring and scalable. This suggests that spatial consistency could be used as a fundamental principle for the design of interfaces. However, there are many display situations where the standard presentation is altered in some way: e.g., a window is moved to a new location, scaled, or rotated on a mobile or tabletop display. It is not known whether the benefits of spatial organization are robust to these common kinds of view transformation. To assess these effects, we tested user performance with a spatial interface that had been transformed in several ways, including different degrees of translation, rotation, scaling, and perspective change. We found that performance was not strongly affected by the changes, except in the case of large rotations. To demonstrate the value of spatial consistency over existing mechanisms for dealing with view changes, we compared user performance with a spatially-stable presentation (using scaling) with that of a 'reflowing' presentation (widely used in current interfaces). This study showed that spatial stability with scaling dramatically outperforms reflowing. This research provides new evidence of spatial consistency's value in interface design: it is robust to the view transformations that occur in typical environments, and it provides substantial performance advantages over traditional methods.

Applications typically provide ways for expert users to increase their performance, such as keyboard shortcuts or customization, but these facilities are frequently ignored. To help address this problem, we introduce skillometers -- lightweight displays that visualize the benefits available through practicing, adopting a better technique, or switching to a faster mode of interaction. We present a general framework for skillometer design, then discuss the design and implementation of a real-world skillometer intended to increase hotkey use. A controlled experiment shows that our skillometer successfully encourages earlier and faster learning of hotkeys. Finally, we discuss general lessons for future development and deployment of skillometers.

Touch scrolling systems use a transfer function to transform gestures on a touch sensitive surface into scrolling output. The design of these transfer functions is complex as they must facilitate precise direct manipulation of the underlying content as well as rapid scrolling through large datasets. However, researchers' ability to refine them is impaired by: (1) limited understanding of how users express scrolling intentions through touch gestures; (2) lack of knowledge on proprietary transfer functions, causing researchers to evaluate techniques that may misrepresent the state of the art; and (3) a lack of tools for examining existing transfer functions. To address these limitations, we examine how users express scrolling intentions in a human factors experiment; we describe methods to reverse engineer existing `black box' transfer functions, including use of an accurate robotic arm; and we use the methods to expose the functions of Apple iOS and Google Android, releasing data tables and software to assist replication. We discuss how this new understanding can improve experimental rigour and assist iterative improvement of touch scrolling.

This paper presents S-Notebook, a tool that makes it possible to "extend" mobile devices with augmented paper. Paper is used to overcome the physical limitations of mobile devices by offering additional space to annotate digital files and to easily create relationships between them. S-Notebook allows users to link paper annotations or drawings to anchors in digital files without having to learn pre-defined pen gestures. The systems stores meta data such as spatial or temporal location of anchors in the document as well as the zoom level of the view. Tapping on notes with the digital pen make appear the corresponding documents as displayed when the notes were taken. A given piece of augmented paper can contain notes associated to several documents, possibliy at several locations. The annotation space can thus serve as a simple way to relate various pieces of one or several digital documents between them. When the user shares his notes, the piece of paper becomes a tangible token that virtually contains digital information.

International audience