Some Augmented Reality applications require tracking physical objects to anchor virtual elements to them. Despite significant progress in computer vision, achieving robust tracking of manipulated objects remains challenging, notably due to occlusions caused by the hands. Yet the hands carry valuable information: the way an object is grasped and moved is reflected in their shape and motion. We explore the following question: can we infer changes to an object’s position and orientation from the shape and movements of the hand manipulating it? We investigate this general approach, which we call Object-from-Hand (ObHa), building three probes, drawing on insights from experimental psychology research on grasping, on our own empirical studies, and on an analysis of the extensive HOT3D dataset of everyday object manipulations. We then discuss the approach’s potential either as a complement to vision-based pose tracking solutions or as a coarse standalone pose tracking solution.

Co-located collaboration often raises challenges related to physical constraints because of e.g., stationary display setups or limited possibilities of movement. We introduce DoubleMe, an Augmented Reality system that generates virtual duplicates of collaborators’ workspaces, comprising both their displays and avatars. With DoubleMe, users maintain the layout of their own physical workspace and can position the duplicate of their collaborator’s workspace nearby. This approach alleviates spatial constraints by enabling one user to join another’s workspace without leaving their own. We report on two experiments examining the effectiveness of this approach. The first experiment investigates how avatar appearance and interaction influence user comfort and relationship dynamics. The second experiment assesses the performance benefits of duplicates over traditional co-located setups for collaborative tasks. Our findings suggest that the addition of duplicates to physical presence can enhance co-located collaboration while improving comfort.

International audience

Interaction in Augmented Reality primarily relies on raycast pointing and mid-air touch. An alternative consists of using the non-dominant hand as a touch-sensitive surface, enabling more comfortable, less fatiguing input. AR UI design guidelines have so far discouraged this alternative because of poor hand tracking performance when the hands overlap, favoring touchpads in the air near the hand, rather than on the hand. But significant improvements to the hand tracking capabilities of recent commodity headsets suggest that on-hand pads may now be feasible. We develop an on-hand touchpad prototype and conduct two studies that involve both discrete input and continuous control tasks. The first study compares such on-hand pads to baseline in-air and on-object pads, showing comparable performance despite some limitations in tracking accuracy. The second study quantifies the advantage of on-hand and in-air pads over on-object pads during transitions between touchpad input and other physical hand activities.

Composers use music notation programs throughout their creative process. Those programs are essentially elaborate structured document editors that enable composers to create high-quality scores by enforcing musical notation rules. They effectively support music engraving, but impede the more creative stages in the composition process because of their lack of flexibility. Composers thus often combine these desktop tools with other mediums such as paper. Interactive surfaces that support pen and touch input have the potential to address the tension between the contradicting needs for structure and flexibility. We interview nine professional composers. We report insights about their thought process and creative intentions, and rely on the “Cognitive Dimensions of Notations” framework to capture the frictions they experience when materializing those intentions on a score. We then discuss how interactive surfaces could increase flexibility by temporarily breaking the structure when manipulating the notation.

TriPad enables opportunistic touch interaction in Augmented Reality using hand tracking only. Users declare the surface they want to appropriate with a simple hand tap gesture. They can then use this surface at will for direct and indirect touch input. TriPad only involves analyzing hand movements and postures, without the need for additional instrumentation, scene understanding or machine learning. TriPad thus works on a variety of flat surfaces, including glass. It also ensures low computational overhead on devices that typically have a limited power budget. We describe the approach, and report on two user studies. The first study demonstrates the robustness of TriPad’s hand movement interpreter on different surface materials. The second study compares TriPad against direct mid-air AR input techniques on both discrete and continuous tasks and with different surface orientations. TriPad achieves a better speed-accuracy trade-off overall, improves comfort and minimizes fatigue.

Spreadsheet programs for interactive surfaces have limited manipulations capabilities and are often frustrating to use. One key reason is that the spreadsheet grid creates a layer that intercepts most user input events, making it difficult to reach the cell values that lie underneath. We conduct an analysis of commercial spreadsheet programs and an elicitation study to understand what users can do and what they would like to do with spreadsheets on interactive surfaces. Informed by these, we design interaction techniques that leverage the precision of the pen to mitigate friction between the different layers. These enable more operations by direct manipulation on and through the grid, targeting not only cells and groups of cells, but values and substrings within and across cells as well. We prototype these interaction techniques and conduct a qualitative study with information workers who perform a variety of spreadsheet operations on their own data.

Large vertical surfaces such as wall displays allow users to work with a very large and high-resolution workspace. Such displays promote physical navigation: users can step close to the display to see details, but also move away to get a wider view of the workspace. In terms of input, current solutions usually combine direct touch on the wall with input on a handheld device, disconnecting close and distant input rather than treating it as a continuum. We present SurfAirs, which are physical controllers that users can manipulate on screen (surface input), in the air (mid-air input), and transition from the surface to the air during a single manipulation (hybrid input). We report on two user studies that compare SurfAirs’ performance with bare-hand input for both mid-air and hybrid input. Participants prefer and perform better with SurfAirs.

Mobile devices such as smartphones and tablets have limited display size and input capabilities that make a variety of tasks challenging. Coupling the mobile device with Augmented Reality eyewear such as smartglasses can help address some of these challenges. In the specific context of digital content annotation tasks, this combination has the potential to enhance the user experience on two fronts. First, annotations can be offloaded into the air around the mobile device, freeing precious screen real-estate. Second, as smartglasses often come equipped with a variety of sensors including a camera, users can annotate documents with pictures or videos of their environment, captured on the spot, hands-free, and from the wearer's perspective. We present AnnotAR, a prototype that we use as a research probe to assess the viability of this approach to digital content annotation. We use AnnotAR to gather users' preliminary feedback in a laboratory setting, and to showcase how it could support real-world use cases.

Tangibles can enrich interaction with digital surfaces. Among others, they support eyes-free control or increase awareness of other users’ actions. Tangibles have been studied in combination with horizontal surfaces such as tabletops, but not with vertical screens such as wall displays. The obvious obstacle is gravity: tangibles cannot be placed on such surfaces without falling. We present WallTokens, easy-to-fabricate tangibles to interact with a vertical surface. A WallToken is a passive token whose footprint is recognized on a tactile surface. It is equipped with a push-handle that controls a suction cup. This makes it easy for users to switch between sliding the token or attaching it to the wall. We describe how to build such tokens and how to recognize them on a tactile surface. We report on a study showing the benefits of WallTokens for manipulating virtual objects over multi-touch gestures. This project is a step towards enabling tangible interaction in a wall display context.

We contribute a detailed investigation of AR-enhanced widgets for smartphones, where AR technology is not only used to offload widgets from the phone to the air around it, but to give users more control on input precision as well. Such widgets have the obvious benefit of freeing up screen real-estate on the phone, but their other potential benefits remain largely theoretical. Their limitations are not well understood, most particularly in terms of input performance. We compare different AR-enhanced widget designs against their state-of-the-art touch-only counterparts with a series of exploratory studies in which participants had to perform three tasks: command trigger, parameter value adjustment, and precise 2D selection. We then derive guidelines from our empirical observations.

Expressive design environments enable visualization designers not only to specify chart types and visual mappings, but also to customize individual graphical marks, as they would in a vector graphics drawing tool. Prior work has mainly investigated how to support the expressive design of a wide range of charts generated from tabular data: bar charts, scatterplots, maps, etc. We focus here on an expressive design environment for node-link diagrams generated from multivariate networks. Such data structures raise specific challenges and opportunities in terms of visual design and interactive authoring. We discuss those specificities and describe the user-centered design process that led to Graphies, a prototype environment for expressive node-link diagram authoring. We then report on a study in which participants successfully reproduced several expressive designs, and created their own designs as well.

Interacting efficiently and comfortably with Augmented Reality (AR) headsets remains a major issue. We investigate the concept of mid-air pads as an alternative to gaze or direct hand input to control a cursor in windows anchored in the environment. ARPads allow users to control the cursor displayed in the headset screen through movements on a mid-air plane, which is not spatially aligned with the headset screen. We investigate a design space for ARPads, which takes into account the position of the pad relative to the user's body, and the orientation of the pad relative to that of the headset screen. Our study suggests that 1) indirect input can achieve the same performance as direct input while causing less fatigue than hand raycast, 2) an ARPad should be attached to the wrist or waist rather than to the thigh, and 3) the ARPad and the screen should have the same orientation.

During sensemaking, people annotate insights: underlining sentences in a document or circling regions on a map. They jot down their hypotheses: drawing correlation lines on scatterplots or creating personal legends to track patterns. We present ActiveInk, a system enabling people to seamlessly transition between exploring data and externalizing their thoughts using pen and touch. ActiveInk enables the natural use of pen for active reading behaviors, while supporting analytic actions by activating any of these ink strokes. Through a qualitative study with eight participants, we contribute observations of active reading behaviors during data exploration and design principles to support sensemaking.

When editing or reviewing a document, people directly overlay ink marks on content. For instance, they underline words, or circle elements in a fgure. These overlay marks often accompany in-context annotations in the form of handwritten footnotes and marginalia. People tend to put annotations close to the content that elicited them, but have to compose with the often-limited whitespace. We introduce SpaceInk, a design space of pen+touch techniques that make room for in-context annotations by dynamically refowing documents. We identify representative techniques in this design space, spanning both new ones and existing ones. We evaluate them in a user study, with results that inform the design of a prototype system. Our system lets users concentrate on capturing feeting thoughts, streamlining the overall annotation process by enabling the fuid inverleaving of space-making gestures with freeform ink.

Before-and-after image pairs show how entities in a given region have evolved over a specific period of time. Satellite images are a major source of such data, that capture how natural phenomena or human activity impact a geographical area. These images are used both for data analysis and to illustrate the resulting findings to diverse audiences. The simple techniques used to display them, including juxtaposing, swapping and monolithic blending, often fail to convey the underlying phenomenon in a meaningful manner. We introduce Baia, a framework to create advanced animated transitions, called animation plans, between before-and-after images. Baia relies on a pixel-based transition model that gives authors much expressive power, while keeping animations for common types of changes easy to create thanks to predefined animation primitives. We describe our model, the associated animation editor, and report on two user studies. In the first study, advanced transitions enabled by Baia were compared to monolithic blending, and perceived as more realistic and better at focusing viewer's attention on a region of interest than the latter. The second study aimed at gathering feedback about the usability of Baia's animation editor.

TouchTokens were introduced recently as a means to design low-cost tangible interfaces. The technique consists in recognizing multi-touch patterns associated with specific tokens, and works on any touch-sensitive surface, with passive tokens that can be made out of any material. TouchTokens have so far been limited to a few basic geometrical shapes only, which puts a significant practical limit to how tailored token sets can be. In this article, we introduce TouchTokenBuilder and TouchTokenTracker that, taken together, aim at facilitating the development of tailor-made tangible interfaces. TouchTokenBuilder is an application that assists interface designers in creating token sets using a simple direct-manipulation interface. TouchTokenTracker is a library that enables tracking the tokens' full geometry. We report on experiments with those tools, showing the strengths and limitations of tangible interfaces with passive tokens.

Network edge data attributes are usually encoded using color, opacity, stroke thickness and stroke pattern, or some combination thereof. In addition to these static variables, it is also possible to animate dynamic particles flowing along the edges. This opens a larger design space of animated edge textures, featuring additional visual encodings that have potential not only in terms of visual mapping capacity but also playfulness and aesthetics. Such animated edge textures have been used in several commercial and design-oriented visualizations, but to our knowledge almost always in a relatively ad hoc manner. We introduce a design space and Web-based framework for generating animated edge textures, and report on an initial evaluation of particle properties - particle speed, pattern and frequency - in terms of visual perception.

Multi-scale navigation interfaces were originally designed to enable single users to explore large visual information spaces on desktop workstations. These interfaces can also be quite useful on tabletops. However, their adaptation to co-located multi-user contexts is not straightforward. The literature describes different interfaces, that only offer a limited subset of navigation actions. In this paper, we first identify a comprehensive set of actions to effectively support multi-scale navigation. We report on a guessability study in which we elicited user-defined gestures for triggering these actions, showing that there is no natural design solution, but that users heavily rely on the now-ubiquitous slide, pinch and turn gestures. We then propose two interface designs based on this set of three basic gestures: one involves two-hand variations on these gestures, the other combines them with widgets. A comparative study suggests that users can easily learn both, and that the gesture-based, visually-minimalist design is a viable option, that saves display space for other controls.

TouchTokens are passive tokens that can be recognized on any capacitive surface based on the spatial configuration of the fingers that hold them. However, interaction with these tokens is confined to the basic two-state model of touch interaction as the system only knows the tokens' position and cannot detect tokens that are not touched. We increase the expressive power of TouchTokens by introducing laser-cut lattice hinges in their design, so as to make them flexible. A new recognizer, that analyzes the micro-movements of the fingers that hold the tokens, enables the system to detect when a token is left on the surface rather than taken off it. It can also detect bend events that can be mapped to command triggers, and a squeezed state that can be used for quasi-modal interaction.

GIS software applications and other mapping tools enable users to correlate data from multiple layers and gain insight from the resulting visualizations. However, most of these applications only feature basic, monolithic layer compositing techniques. These techniques do not always support users effectively in their tasks, as we observed during interviews with GIS experts. We introduce MapMosaic, a novel approach based on dynamic visual compositing that enables users to interactively create and manipulate local composites of multiple vector and raster map layers, taking into account the semantics and attribute values of objects and fields in the compositing process. We evaluate MapMosaic ’s interaction model against that of QGIS (a widely used desktop GIS) and MAPublisher (a professional cartography tool) using the ‘Cognitive Dimensions’ framework and through an analytical comparison, showing that MapMosaic ’s model is more flexible and can support users more effectively in their tasks. We also report on feedback obtained from experts, which further confirms the potential of this highly dynamic approach to map layer compositing.

TouchTokens make it possible to easily build interfaces that combine tangible and gestural input using passive tokens and a regular multi-touch surface. The tokens constrain users' grasp, and thus, the relative spatial configuration of fingers on the surface, theoretically making it possible to design algorithms that can recognize the resulting touch patterns. We performed a formative user study to collect and analyze touch patterns with tokens of varying shape and size. The analysis of this pattern collection showed that individual users have a consistent grasp for each token, but that this grasp is user-dependent and that different grasp strategies can lead to confounding patterns. We thus designed a second set of tokens featuring notches that constrain users' grasp. Our recognition algorithm can classify the resulting patterns with a high level of accuracy (>95%) without any training, enabling application designers to associate rich touch input vocabularies with command triggers and parameter controls.

Geovisualization applications typically organize data into layers. These layers hold different types of geographical features, describe different characteristics of the same features, or represent those features at different points in time. Layers can be composited in various ways, most often employing a juxtaposition or superimposition strategy, to produce maps that users can explore interactively. From an HCI perspective, one of the main challenges is to design interactive compositions that optimize the legibility of the resulting map and that ease layer comparison. We characterize five representative techniques, and empirically evaluate them using a set of real-world maps in which we purposefully introduce six types of differences amenable to inter-layer visual comparison. We discuss the merits of these techniques in terms of visual interference, user attention and scanning strategy. Our results can help inform the design of map-based visualizations for supporting geo-analysis tasks in many application areas.

Drag-and-drop has become ubiquitous, both on desktop computers and touch-sensitive surfaces. It is used to move and edit the geometry of elements in graphics editors, to adjust parameters using controllers such as sliders, or to manage views (e.g., moving and resizing windows, panning maps). Reverting changes made via a drag-and-drop usually entails performing the reciprocal drag-and-drop action. This can be costly as users have to remember the previous position of the object and put it back precisely. We introduce the D n D −1 model that handles all past locations of graphical objects. We redesign the Dwell-and-Spring widget to interact with this history, and explain how applications can implement D n D −1 to enable users to perform reciprocal drag-and-drop to any past location for both individual objects and groups of objects. We report on two user studies, whose results show that users understand D n D −1 , and that Dwell-and-Spring enables them to interact with this model effectively.

Millions of people go to the Web to search for geographical itineraries. Inspecting those map itineraries remains tedious because they seldom fit on screen, requiring much panning & zooming to see details. Focus+context techniques address this problem by displaying routes at a scale that allows them to fully fit on screen: users see the entire route at once, and perform magnified steering using a lens to navigate along the path, revealing additional detail. Navigation based on magnified steering has been shown to outperform pan & zoom for large steering tasks. Yet, this task remains challenging, in part because paths have a tendency to "slip off" the side of the lens. RouteLenses automatically adjust their position based on the geometry of the path that users steer through. RouteLenses make it easier for users to follow a route, yet do not constrain movements too strictly, leaving them free to move the lens away from the path to explore its surroundings.

Touchscreen interaction currently relies on a limited set of multi-touch gestures and a wide range of graphical widgets that are often difficult to manipulate and consume much screen real-estate. Many tasks remain tedious to perform on touchscreens: selecting text over multiple views, manipulating different degrees of freedom of a graphical object, invoking a command and setting its parameter values in a row. We propose a design space of simple multi-touch gestures that designers of user interfaces can systematically explore to propose more gestures to users. We further consider a set of 32 gestures for tablet-sized devices, by proposing an incremental recognition engine that works with current hardware technology, and empirically testing the usability of those gestures. In our experiment, individual gestures are recognized with an average accuracy of ~90%, and users successfully achieve some of the transitions between gestures without the use of explicit delimiters. The goal of our contribution is to assist designers in optimizing the use of the rich multi-touch input channel for the activation of discrete and continuous controls, and enable fluid transitions between controls.

Substantial amount of research in Psychology has studied how people manipulate objects in the physical world. This work has unveiled that people show strong signs of prospective motor planning, i.e., they choose initial grasps that avoid uncomfortable end postures and facilitate object manipulation. Interactive tabletops allow their users great flexibility in the manipulation of virtual objects but to our knowledge previous work has never examined whether prospective motor control takes place in this context. To test this, we ran three experiments. We systematically studied how users adapt their grasp when asked to translate and rotate virtual objects on a multitouch tabletop. Our results demonstrate that target position and orientation significantly affect the orientation of finger placement on the object. We analyze our results in the light of the most recent model of planning for manipulating physical objects and identify their implications for the design of tabletop interfaces. \

Our goal is to enhance navigation in mobile interfaces with quick command gestures that do not make use of explicit mode-switching actions. TilTouch gestures extend the vocabulary of navigation interfaces by combining motion tilt with directional touch. We consider sixteen directional TilTouch gestures that rely on tilt and touch movements along the four main compass directions. An experiment explores their effectiveness for both one-handed and two-handed use. Results identify the best combinations of TilTouch gestures in terms of performance, motor coordination, and user preferences.

Virtual navigation on a mobile touchscreen is usually performed using finger gestures: drag and flick to scroll or pan, pinch to zoom. While easy to learn and perform, these gestures cause significant occlusion of the display. They also require users to explicitly switch between navigation mode and edit mode to either change the viewport's position in the document, or manipulate the actual content displayed in that viewport, respectively. SidePress augments mobile devices with two continuous pressure sensors co-located on one of their sides. It provides users with generic bidirectional navigation capabilities at different levels of granularity, all seamlessly integrated to act as an alternative to traditional navigation techniques, including scrollbars, drag-and-flick, or pinch-to-zoom. We describe the hardware prototype, detail the associated interaction vocabulary for different applications, and report on two laboratory studies. The first shows that users can precisely and efficiently control SidePress; the second, that SidePress can be more efficient than drag-and-flick touch gestures when scrolling large documents.

The Power-up Button is a physical button that combines pressure and proximity sensing to enable gestural interaction with one thumb. Combined with a gesture recognizer that takes the hand's anatomy into account, the Power-up Button can recognize six different mid-air gestures performed on the side of a mobile device. This gives it, for instance, enough expressive power to provide full one-handed control of interface widgets displayed on screen. This technology can complement touch input, and can be particularly useful when interacting eyes-free. It also opens up a larger design space for widget organization on screen: the button enables a more compact layout of interface components than what touch input alone would allow. This can be useful when, e.g., filling the numerous fields of a long Web form, or for very small devices.

In graphical user interfaces, direct manipulation consists in incremental actions that should be reversible. Typical examples include manipulating geometrical shapes in a vector graphics editor, navigating a document using a scrollbar, or moving and resizing windows on the desktop. As in many such cases, there will not be any mechanism to undo them, requiring users to manually revert to the previous state using a similar sequence of direct manipulation actions. The associated motor and cognitive costs can be high. We argue that proper and consistent mechanisms to support undo in this context are lacking, and present Dwell-and-Spring, an interaction technique that uses the metaphor of springs to enable users to undo direct manipulations. A spring widget pops up whenever the user dwells during a press-drag-release interaction, giving her the opportunity to either cancel the current manipulation or undo the last one. The technique is generic and can easily be implemented on top of existing applications to complement the traditional undo command. Empirical evaluation shows that users quickly adopt it as soon as they discover it.

Gesture-based interfaces provide expert users with an efficient form of interaction but they require a learning effort for novice users. To address this problem, some on-line guiding techniques display all available gestures in response to partial input. However, partial input recognition algorithms are scale dependent while most gesture recognizers support scale independence (i.e., the same shape at different scales actually invokes the same command). We propose an algorithm for estimating the scale of any partial input in the context of a gesture recognition system and illustrate how it can be used to improve users' experience with gesture-based systems.

Focus+context interfaces provide in-place magnification of a region of the display, smoothly integrating the focus of attention into its surroundings. Two representations of the data exist simultaneously at two different scales, providing an alternative to classical pan & zoom for navigating multi-scale interfaces. For many practical applications however, the magnification range of focus+context techniques is too limited. This paper addresses this limitation by exploring the quantization problem: the mismatch between visual and motor precision in the magnified region. We introduce three new interaction techniques that solve this problem by

Focus+context interaction techniques based on the metaphor of lenses are used to navigate and interact with objects in large information spaces. They provide in-place magnification of a region of the display without requiring users to zoom into the representation and consequently lose context. In order to avoid occlusion of its immediate surroundings, the magnified region is often integrated in the context using smooth transitions based on spatial distortion. Such lenses have been developed for various types of representations using techniques often tightly coupled with the underlying graphics framework. We describe a representation-independent solution that can be implemented with minimal effort in different graphics frameworks, ranging from 3D graphics to rich multiscale 2D graphics combining text, bitmaps, and vector graphics. Our solution is not limited to spatial distortion and provides a unified model that makes it possible to define new focus+context interaction techniques based on lenses whose transition is defined by a combination of dynamic displacement and compositing functions. We present the results of a series of user evaluations that show that one such new lens, the speed-coupled blending lens, significantly outperforms all others.

This paper investigates using stroke gestures as shortcuts to menu selection. We first experimentally measured the performance and ease of learning of stroke shortcuts in comparison to keyboard shortcuts when there is no mnemonic link between the shortcut and the command. While both types of shortcuts had the same level of performance with enough practice, stroke shortcuts had substantial cognitive advantages in learning and recall. With the same amount of practice, users could successfully recall more shortcuts and make fewer errors with stroke shortcuts than with keyboard shortcuts. The second half of the paper focuses on UI development support and articulates guidelines for toolkits to implement stroke shortcuts in a wide range of software applications. We illustrate how to apply these guidelines by introducing the Stroke Shortcuts Toolkit (SST) which is a library for adding stroke shortcuts to Java Swing applications with just a few lines of code.

International audience

Focus + context techniques such as fisheye lenses are used to navigate and manipulate objects in multi-scale worlds. They provide in-place magnification of a region without requiring users to zoom the whole representation and consequently lose context. Their adoption is however hindered by usability problems mostly due to the nature of the transition between focus and context. Existing transitions are often based on a physical metaphor (magnifying glass, fisheye, rubber sheet), and are almost always achieved through a single dimension: space. We investigate how other dimensions, namely time and translucence, can be used to achieve more efficient transitions. We present an extension to Carpendale's framework for unifying presentation space accommodating these new dimensions. We define new lenses in that space, called Sigma lenses, and compare them to existing lenses through experiments based on a generic task: focus targeting. Results show that one new lens, the Speed-coupled flattening lens, significantly outperforms all others.

Abstract This article describes SwingStates, a Java toolkit designed to facilitate the development of graphical user interfaces and bring advanced interaction techniques to the Java platform. SwingStates is based on the use of finite‐state machines specified directly in Java to describe the behavior of interactive systems. State machines can be used to redefine the behavior of existing Swing widgets or, in combination with a new canvas widget that features a rich graphical model, to create brand new widgets. SwingStates also supports arbitrary input devices to implement novel interaction techniques based, for example, on bi‐manual or pressure‐sensitive input. We have used SwingStates in several Master's‐level classes over the past two years and have developed a benchmark approach to evaluate the toolkit in this context. The results demonstrate that SwingStates can be used by non‐expert developers with little training to successfully implement advanced interaction techniques. Copyright © 2007 John Wiley & Sons, Ltd.

Touchstone is an open-source experiment design platform designed to help establish a solid research foundation for HCI in the area of novel interaction techniques. Touchstone includes a design platform for exploring alternative designs of controlled laboratory experiments, a run platform for running subjects and a limited analysis platform for advice and access to on-line statistics packages. Designed for HCI researchers and their students, Touchstone facilitates the process of creating new experiments, as well as replicating and extending experiments in the research literature. We tested Touchstone by designing two controlled experiments. One illustrates how to create a new experiment from scratch. The other replicates and extends a previous study of multiscale pointing interaction techniques: OrthoZoom was fastest, followed by bi-manual Pan & Zoom; SDAZ and traditional Pan & Zoom were consistently slower.

A number of experimental studies based on domain-specific tasks have evaluated the efficiency of navigation techniques for searching multi-scale worlds. The discrepancies among their results call for a more generic framework similar in spirit to Fitts' reciprocal pointing task, but adapted to a task that significantly differs from pure pointing. We introduce such a framework based on an abstract task and evaluate how four multi-scale navigation techniques perform in one particular multi-scale world configuration. Experimental findings indicate that, in this context, pan & zoom combined with an overview is the most efficient technique of all four, and that focus + context techniques perform better than classical pan & zoom. We relate these findings to more realistic situations, discuss their applicability, and how the framework can be used to cover a broad range of situations.

This article introduces the OrthoZoom Scroller, a novel interaction technique that improves target acquisition in very large one-dimensional spaces. The OrthoZoom Scroller requires only a mouse to perform panning and zooming in a 1D space. Panning is performed along the slider dimension while zooming is performed along the orthogonal one. We present a controlled experiment showing that the OrthoZoom Scroller is about twice as fast as Speed Dependant Automatic Zooming to perform pointing tasks whose index of difficulty is in the 10-30 bits range. We also present an application to browse large textual documents with the OrthoZoom Scroller that uses semantic zooming and snapping on the structure.

This article describes SwingStates, a library that adds state machines to the Java Swing user interface toolkit. Unlike traditional approaches, which use callbacks or listeners to define interaction, state machines provide a powerful control structure and localize all of the interaction code in one place. SwingStates takes advantage of Java's inner classes, providing programmers with a natural syntax and making it easier to follow and debug the resulting code. SwingStates tightly integrates state machines, the Java language and the Swing toolkit. It reduces the potential for an explosion of states by allowing multiple state machines to work together. We show how to use SwingStates to add new interaction techniques to existing Swing widgets, to program a powerful new Canvas widget and to control high-level dialogues.