Large curved displays are ideal for viewing 360° content, such as 3D maps, but typically restrict users to a 180° viewport, leaving information off-screen. Since users naturally direct their heads toward regions on-screen before interacting, head movements offer a promising alternative for workspace manipulation to bring off-screen content into view. We explore rate control functions (linear, sigmoid, polynomial) and zone control functions (continuous, friction, interrupted, additive) to translate head rotations into workspace control, enabling users to access off-screen content. Polynomial rate control emerges as the best choice, achieving the fastest trial times and highest subjective ratings. Using a map navigation task, our second study demonstrates that users perform better with the polynomial head-based technique than with the industry-standard controller-based methods, click-and-drag and joystick-push, for 360° workspace navigation. Based on these findings, we provide guidelines to inform the design of future 360° workspace navigation techniques for large curved displays.

We explore the value of Hand Proximate User Interfaces (HPUIs) for in-situ interactive head-mounted systems, i.e. systems designed to support a user’s primary activity. HPUIs are unencumbered, single-handed, and have tactile and proprioceptive affordances. This allows novice and expert users to rely on the visual cues available in the UI, but to then gradually interact eyes-free such that they minimize interrupting the user’s core task. Prior work on HPUI falls short of validating this premise. We address this gap with comparative analysis in the context of a compound task, where one hand is involved in a primary task and the second interacts with a secondary but supporting task (e.g. selecting items from a menu). We compare HPUI with mid-air direct interactions, a common form of interacting with head-mounted displays. The results show that HPUI performs similarly to mid-air but with better eyes-free affordances and user preferences.

Many smartwatches have a digital crown as a complementary input modality to the touchscreen. While the crown eliminates issues of touchscreen occlusion and imprecision, i.e., the 'fat-finger' problem, the crown input is limited as it only supports bi-directional rotations along a single dimension. We present TiltCrown, a crown prototype that increases a smartwatch's input bandwidth through an isometric joystick. TiltCrown supports angular tilt, rotations and button press events for touchless smartwatch interaction. In a user study we explore how accurate and how fast users can perform item selection tasks using TiltCrown.

In recent years, around device input has emerged as a complement to standard touch input, albeit in limited tasks and contexts, such as for item selection or map navigation. We push the boundaries for around device interactions to facilitate an entire smartphone application: browsing through large information lists to make a decision. To this end, we present AirPanes, a novel technique that allows two-handed in-air interactions, conjointly with touch input to perform analytic tasks, such as making a purchase decision. AirPanes resolves the inefficiencies of having to switch between multiple views or panes in common smartphone applications. We explore the design factors that make AirPanes efficient. In a controlled study, we find that AirPanes is on average 50% more efficient that standard touch input for an analytic task. We offer recommendations for implementing AirPanes in a broad range of applications.

We investigate how the use of metaphors supports the intuitiveness of gesture input vocabularies for Unmanned Aerial Vehicle (UAV) navigation. We compare gesture sets constituting a single metaphor to gesture sets that are based on mixed metaphors in terms of their respective intuitiveness. To this end, we implemented a 3D simulator in order to check how well novice users steer a UAV without knowing the valid gestures, using only a hint on the underlying metaphor. We compared their task completion time (indirect assessment of intuitiveness) with the one achieved after studying a gesture set that consists of gestures from several metaphors. We analyzed users' feedback reflected in questionnaires (direct assessment of intuitiveness) to further compare single metaphor gesture sets with mixed metaphors gesture sets. The results of the study support our hypothesis that a metaphor-based approach is an expedient means for gesture-based UAV navigation.

Motivated by a rise in the variety and number of mobile devices that users carry, we investigate scenarios when operating these devices in a spatially interlinked manner can lead to interfaces that generate new advantages. Our exploration is focused on the design of SAMMI, a spatially-aware multi-device interface to assist with analytic map navigation tasks, where, in addition to browsing the workspace, the user has to make a decision based on the content embedded in the map. We focus primarily on the design space for spatially interlinking a smartphone with a smartwatch. As both smart devices are spatially tracked, the user can browse information by moving either device in the workspace. We identify several design factors for SAMMI and through a first study we explore how best to combine these for efficient map navigation. In a second study we compare SAMMI to the common Flick-&-Pinch gestures for an analytic map navigation task. Our results reveal that SAMMI is an efficient spatial navigation interface, and by means of an additional spatially tracked display, can facilitate quick information retrieval and comparisons. We finally demonstrate other potential use cases for SAMMI that extend beyond map navigation to facilitate interaction with spatial workspaces.

Several research groups have demonstrated advantages of extending a mobile device's input vocabulary with in-air gestures. Such gestures show promise but are not yet being integrated onto commercial devices. One reason for this might be the uncertainty about users' perceptions regarding the social acceptance of such around-device gestures. In three studies, performed in public settings, we explore users' and spectators' attitudes about using around-device gestures in public. The results show that people are concerned about others' reactions. They are also sensitive and selective regarding where and in front of whom they would feel comfortable using around-device gestures. However, acceptance and comfort are strongly linked to gesture characteristics, such as, gesture size, duration and in-air position. Based on our findings we present recommendations for around-device input designers and suggest new approaches for evaluating the social acceptability of novel input methods.

Most interfaces in the field of image and video search use a two-dimensional grid interface, which presents image thumbnails in a left-to-right arrangement that can be browsed from top to bottom. This grid interface, however, has several drawbacks that become particularly apparent when performing interactive search tasks for target items in large collections of images or videos. Therefore, we propose to use 3-D interfaces as an alternative to the grid interface for interactive known-item search in visual data as they can partially overcome these drawbacks. In this paper, we first summarize our ideas and discuss design aspects of a 3-D ring and a 3-D globe interface. Next, we present results from four different user studies, where we evaluated the performance of these interfaces for known-item search tasks in image collections. Our results from these studies show that the proposed 3-D interfaces allow for significantly faster visual target search on desktop computers with mouse interaction as well as on tablet devices. The interfaces also achieve better subjective ratings. However, our evaluation also shows that on smartphones with 3.5-in screens an improvement over the grid interface in terms of visual search time is only possible in collections with more than 200 images.

Previous in-depth analyses of the cursor paths taken by young children when they point at screen targets have shown that the fine-tuning movements that are necessary to accurately position the cursor over a small target can be very troublesome. We present two mouse pointing experiments with four-year-olds; the first re-evaluates the effect of PointAssist, a technique designed to help children performing fine-tuning movements by tracking the cursor and manipulating the cursor control-display gain in problem situations. The results partially confirm previously reported results showing that PointAssist can improve children's pointing accuracy when pointing at small targets. The second experiment investigates the effect of various (constant) cursor control-display gains on children's pointing performance. The results suggest a preference for higher gains.

Exploring information content on mobile devices can be tedious and time consuming. We present Around-Device Binning, or AD-Binning, a novel mobile user interface that allows users to off-load mobile content in the space around the device. We informed our implementation of AD-Binning by exploring various design factors, such as the minimum around-device target size, suitable item selection methods, and techniques for placing content in off-screen space. In a task requiring exploration, we find that AD-Binning improves browsing efficiency by avoiding the minute selection and flicking mechanisms needed for on-screen interaction. We conclude with design guidelines for off screen content storage and browsing.

Smartphones are useful personal assistants and omnipresent communication devices. However, collaboration is not among their strengths. With the advent of embedded projectors this might change. We conducted a study with 56 participants to find out if map navigation and spatial memory performance among users and observers can be improved by using a projector phone with a peephole interface instead of a smartphone with its touchscreen interface. Our results show that users performed map navigation equally well on both interfaces. Spatial memory performance, however, was 41% better for projector phone users. Moreover, observers of the map navigation on the projector phone were 25% more accurate when asked to recall locations of points of interest after they watched a user performing map navigation.

Smartphones and tablets are popular devices. As lightweight, compact devices with built-in high-quality cameras, they are ideal to carry around and to use for snapshot photography. As the number of photos accumulate on the device quickly finding a particular photo can be tedious using the default grid-based photo browser installed on the device. In this paper we investigate user performance in a photo browsing task on an iPad and an iPod Touch device. We present results from two user experiments comparing the standard grid interface to a pan-and-zoom able grid, a 3D-globe and a 3D-ring. In particular we are interested in how the interfaces perform with large photo collections (100 to 400 photos). The results show most promise for the pan-and-zoom grid and that the performance with the standard grid interface quickly deteriorates with large collections.

Interactive image and video search tools typically use a grid-like arrangement of thumbnails for preview purpose. Such a display, which is commonly known as storyboard, provides limited flexibility at interactive search and it does not optimally exploit the available screen estate. In this paper we design and evaluate alternatives to the common two-dimensional storyboard. We take advantage of 3D graphics in order to present image thumbnails in cylindrical arrangements. Through a user study we evaluate the performance of these interfaces in terms of visual search time and subjective performance.

Peephole pointing is a promising interaction technique for large workspaces that contain more information than can be appropriately displayed on a single screen. In peephole pointing a window to the virtual workspace is moved in space to reveal additional content. In 2008, two different models for peephole pointing were discussed. Cao, Li and Balakrishnan proposed a two-component model, whereas Rohs and Oulasvirta investigated a similar model, but concluded that Fitts' law is sufficient for predicting peephole pointing performance. We present a user study performed with a handheld projector showing that Cao et al.'s model only outperforms Fitts' law in prediction accuracy when different peephole sizes are used and users have no prior knowledge of target location. Nevertheless, Fitts' law succeeds under the conditions most likely to occur. Additionally, we show that target overshooting is a key characteristic of peephole pointing and present the implementation of an orientation aware handheld projector that enables peephole interaction without instrumenting the environment.

Default image browsing interfaces on touch-based mobile devices provide limited support for image search tasks. To facilitate fast and convenient searches we propose an alternative interface that takes advantage of 3D graphics and arranges images on a rotatable globe according to color similarity. In a user study we compare the new design to the iPad's image browser. Results collected from 24 participants show that for color-sorted image collections the globe can reduce search time by 23% without causing more errors and that it is perceived as being fun to use and preferred over the standard browsing interface by 70% of the participants.

We investigate whether users' visual search performance in a commonly used grid-like arrangement of images (i.e., a storyboard) can be improved by using a similarity-based sorting of images. We propose a simple but efficient algorithm for sorting image based on their color similarity. The algorithm generates an intuitive arrangement of images and allows for general application with several different layouts (e.g., storyboard, simple row/column, 3D globe/cylinder). In difference to previous work, which rarely present results from user studies, we perform a fair user study and compare an interface with color sorted images to an interface with images positioned in a random order. Both interfaces use exactly the same screen estate and interaction means. Results show that users are 20% faster with the sorted interface.

We present an intuitive user interface for the exploration of images on mobile multi-touch devices. Our interface uses a novel cylindrical 3D visualization of visually sorted images as well as touch gestures and tilting operations to support mobile users in interactive browsing of images by providing convenient navigation/interaction and intuitive visualization capabilities.

The limited viewport size of mobile devices requires that users continuously acquire information that lies beyond the edge of the screen. Recent hardware solutions are capable of continually tracking a user's finger around the device. This has created new opportunities for interactive solutions, such as direct off-screen pointing: the ability to directly point at objects that are outside the viewport. We empirically characterize user performance with direct off-screen pointing when assisted by target cues. We predict time and accuracy outcomes for direct off-screen pointing with existing and derived models. We validate the models with good results (R2 ≥ 0.9) and reveal that direct off-screen pointing takes up to four times longer than pointing at visible targets, depending on the desired accuracy tradeoff. Pointing accuracy degrades logarithmically with target distance. We discuss design implications in the context of several real-world applications.

We consider different hierarchical menu and toolbar-like interface designs from a theoretical perspective and show how a model based on visual search time, pointing time, decision time and expertise development can assist in understanding and predicting interaction performance. Three hierarchical menus designs are modelled -- a traditional pull-down menu, a pie menu and a novel Square Menu with its items arranged in a grid -- and the predictions are validated in an empirical study. The model correctly predicts the relative performance of the designs -- both the eventual dominance of Square Menus compared to traditional and pie designs and a performance crossover as users gain experience. Our work shows the value of modelling in HCI design, provides new insights about performance with different hierarchical menu designs, and demonstrates a new high-performance menu type.

In this paper we show how a model centered analysis of the usage of the mouse click interaction action in graphical user interfaces can be used to create a new menu system. The analysis identifies a possible new usage of the click action in cascading pull-down menus which can make it easier for the user during menu navigation and selection. A new menu system which is easy to implement, the ""Jumping Menu"", is introduced. The new menu system warps the screen cursor to the right into open sub-menu levels when a mouse click is detected inside a parent item. The Jumping Menu was compared with standard pull-down menus and force enhanced menus in a user experiment. The results show that the Jumping Menu and a force enhanced menu can facilitate menu interaction and that they are promising alternatives to conventional menu systems. Based on the results, a prediction model for selection times in Jumping Menus is developed.

Selecting a menu item in a cascading pull-down menu is a frequent but time consuming and complex GUI task. This paper describes an approach aimed to support the user during selection in cascading pull-down menus when using an indirect pointing device. By enhancing such a cascading pull-down menu with "force fields", the cursor is attracted toward a certain direction, e.g. toward the right hand side within a menu item, which opens up a sub-menu, making the cursor steering task easier and faster. The experiment described here shows that the force fields can decrease selection times, on average by 18%, when a mouse, a track point, or touch pad is used as input device. The results also suggest that selection times in cascading pull-down menus can be modeled using a combination of Fitts' law and the steering law. The proposed model proved to hold for all three devices, in both standard and in enhanced cascading pull-down menus, with correlations better than r2=0.90.