Live and pre-recorded video tutorials are an effective means for teaching physical skills such as cooking or prototyping electronics. A dedicated cameraperson following an instructor’s activities can improve production quality. However, instructors who do not have access to a cameraperson’s help often have to work within the constraints of static cameras. We present Stargazer, a novel approach for assisting with tutorial content creation with a camera robot that autonomously tracks regions of interest based on instructor actions to capture dynamic shots. Instructors can adjust the camera behaviors of Stargazer with subtle cues, including gestures and speech, allowing them to fluidly integrate camera control commands into instructional activities. Our user study with six instructors, each teaching a distinct skill, showed that participants could create dynamic tutorial videos with a diverse range of subjects, camera framing, and camera angle combinations using Stargazer.

Online synchronous tutoring allows for immediate engagement between instructors and audiences over distance. However, tutoring physical skills remains challenging because current telepresence approaches may not allow for adequate spatial awareness, viewpoint control of the demonstration activities scattered across an entire work area, and the instructor’s sufficient awareness of the audience. We present Asteroids, a novel approach for tangible robotic telepresence, to enable workbench-scale physical embodiments of remote people and tangible interactions by the instructor. With Asteroids, the audience can actively control a swarm of mini-telepresence robots, change camera positions, and switch to other robots’ viewpoints. Demonstrators can perceive the audiences’ physical presence while using tangible manipulations to control the audience’s viewpoints and presentation flow. We conducted an exploratory evaluation for Asteroids with 12 remote participants in a model-making tutorial scenario with an architectural expert demonstrator. Results suggest our unique features benefitted participants’ engagement, sense of presence, and understanding.

Remotely controlled camera drones can support live, dynamic, and interactive virtual tours for travelers to overcome distance, expense, and health barriers. Yet, assigning one drone to one traveler may incur unnecessary waste of resources, and an abundance of concurrent drones raises safety concerns. While sharing the input and output of a single drone among multiple concurrent users can alleviate these limitations, standard control sharing protocols, such as turn-taking, are often inefficient. We present Constellation, a multi-user drone control system that synthesizes diverse user goals and generates efficient flight paths for the group. It supports point-of-interest specification on both static 3D environmental maps and live camera views. The generated paths minimize all users’ total extra waiting time. A web-based study with 16 participants show that Constellation could help groups navigate to their points-of-interest faster in comparison to the turn-taking baseline.

An increasingly popular way of experiencing remote places is by viewing 360° virtual tour videos, which show the surrounding view while traveling through an environment. However, finding particular locations in these videos can be difficult because current interfaces rely on distorted frame previews for navigation. To alleviate this usability issue, we propose Route Tapestries, continuous orthographic-perspective projection of scenes along camera routes. We first introduce an algorithm for automatically constructing Route Tapestries from a 360° video, inspired by the slit-scan photography technique. We then present a desktop video player interface using a Route Tapestry timeline for navigation. An online evaluation using a target-seeking task showed that Route Tapestries allowed users to locate targets 22% faster than with YouTube-style equirectangular previews and reduced the failure rate by 75% compared to a more conventional row-of-thumbnail strip preview. Our results highlight the value of reducing visual distortion and providing continuous visual contexts in previews for navigating 360°virtual tour videos.

The 2020 CHCCS/SCDHM Achievement Award from the Canadian Human-Computer Communications Society is presented to Dr. Ravin Balakrishnan. This award recognizes his significant and varied contributions in the areas of Human Computer Interaction (HCI), Information and Communications Technology for Development, and Interactive Computer Graphics. Ravin's work has had a tremendous impact on real-world applications. His research includes early innovations in areas such as 3D user interfaces, large display input, multitouch gestures, freehand input, and pen-based computing, which has informed and inspired techniques and technologies that are now commonplace in commercial products. a conversation between Ravin Balakrishnan and Prof. Tovi Grossman (University of Toronto) that took place in April, 2020.

Camera drones, a rapidly emerging technology, offer people the ability to remotely inspect an environment with a high degree of mobility and agility. However, manual remote piloting of a drone is prone to errors. In contrast, autopilot systems can require a significant degree of environmental knowledge and are not necessarily designed to support flexible visual inspections. Inspired by camera manipulation techniques in interactive graphics, we designed StarHopper, a novel touch screen interface for efficient object-centric camera drone navigation, in which a user directly specifies the navigation of a drone camera relative to a specified object of interest. The system relies on minimal environmental information and combines both manual and automated control mechanisms to give users the freedom to remotely explore an environment with efficiency and accuracy. A lab study shows that StarHopper offers an efficiency gain of 35.4% over manual piloting, complimented by an overall user preference towards our object-centric navigation system.

Past work in augmented reality has shown that temperature-associated AR stimuli can induce warming and cooling sensations in the user, and prior work in psychology suggests that a person’s body temperature can influence that person’s sense of subjective perception of duration. In this paper, we present a user study to evaluate the relationship between temperature-associated virtual stimuli presented on an AR-HMD and the user’s sense of subjective perception of duration and temperature. In particular, we investigate two independent variables: the apparent temperature of the virtual stimuli presented to the participant, which could be hot or cold, and the location of the stimuli, which could be in direct contact with the user, in indirect contact with the user, or both in direct and indirect contact simultaneously. We investigate how these variables affect the users’ perception of duration and perception of body and environment temperature by having participants make prospective time estimations while observing the virtual stimulus and answering subjective questions regarding their body and environment temperatures. Our work confirms that temperature-associated virtual stimuli are capable of having significant effects on the users’ perception of temperature, and highlights a possible limitation in the current augmented reality technology in that no secondary effects on the users’ perception of duration were observed.

Online language lessons have adopted live broadcasted videos to provide more real-time interactive experiences between language teachers and learners. However, learner interactions are primarily limited to the built-in text chat in the live stream. Using text alone, learners cannot get feedback on important aspects of a language, such as speaking skills, that are afforded only by offering richer types of interactions. We present results from a 2-week in-the-wild study, in which we investigate the use of text, audio, video, image, and stickers as interaction tools for language teachers and learners in live streaming. Our language teacher explored three different teaching strategies over four live streamed English lessons, while nine students watched and interacted using multimodal tools. The findings reveal that multimodal communication yields instant feedback and increased engagement, but its use is dependent on factors such as group size, surroundings, time, and online identity.

Selecting an item of interest on smartwatches can be tedious and time-consuming as it involves a series of swipe and tap actions. We present PageFlip, a novel method that combines into a single action multiple touch operations such as command invocation and value selection for efficient interaction on smartwatches. PageFlip operates with a page flip gesture that starts by dragging the UI from a corner of the device. We first design PageFlip by examining its key design factors such as corners, drag directions and drag distances. We next compare PageFlip to a functionally equivalent radial menu and a standard swipe and tap method. Results reveal that PageFlip improves efficiency for both discrete and continuous selection tasks. Finally, we demonstrate novel smartwatch interaction opportunities and a set of applications that can benefit from PageFlip.

Machine readable passive tags for tagging physical objects are ubiquitous today. We propose Motion Codes, a passive tagging mechanism that is based on the kinesthetic motion of the user's hand. Here, the tag comprises of a visual pattern that is displayed on a physical surface. To scan the tag and receive the encoded information, the user simply traces their finger over the pattern. The user wears an inertial motion sensing (IMU) ring on the finger that records the traced pattern. We design two motion code schemes, Asterisk and Obelisk that rely on directional vector data processed from the IMU. We evaluate both schemes for the effects of orientation, size, and data density on their accuracies. We further conduct an in-depth analysis of the sources of motion deviations in the ring data as compared to the ground truth finger movement data. Overall, Asterisk achieves a 95% accuracy for an information capacity of 16.8 million possible sequences.

Current freehand interactions with large displays rely on point & select as the dominant paradigm. However, constant hand movement in air for pointer navigation leads to hand fatigue quickly. We introduce summon & select, a new model for freehand interaction where, instead of navigating to the control, the user summons it into focus and then manipulates it. Summon & select solves the problems of constant pointer navigation, need for precise selection, and out-of-bounds gestures that plague point & select. We describe the design and conduct two studies to evaluate the design and compare it against point & select in a multi-button selection study. The results show that summon & select is significantly faster and has less physical and mental demand than point & select.

Squeezing sensations are one of the most common and intimate forms of human contact. In this paper, we investigate HapticClench, a device that generates squeezing sensations using shape memory alloys. We define squeezing feedback in terms of it perceptual properties and conduct a psychophysical evaluation of HapticClench. HapticClench is capable of generating up to four levels of distinguishable load and works well in distracted scenarios. HapticClench has a high spatial acuity and can generate spatial patterns on the wrist that the user can accurately recognize. We also demonstrate the use of HapticClench for communicating gradual progress of an activity, and for generating squeezing sensations using rings and loose bracelets.

Improv Remix is a mixed-reality system for live onstage video editing, using whole-body interaction, as an extension of improvised theatre. This work documents the process of analyzing an art form, then building technology that supports its extension. We tested the potential for video integration in improv in a exploratory workshop, then determined features and interaction techniques through iterative development with improvisors. We demonstrated the final iteration in a public showcase. Our contributions are (1) the documentation of the process, (2) a basic set of always-on interaction techniques tailored for performers standing adjacent to a large display, (3) methods to remix stage performance video, and, (4) a collection of creative use cases for the system, with an analysis of how the system extends improvisation.

Fast and accurate access to keys for text entry remains an open question for miniature screens. Existing works typically use a cumbersome two-step selection process, first to zero-in on a particular zone and second to make the key selection. We introduce DualKey, a miniature screen text entry technique with a single selection step that relies on finger identification. We report on the results of a 10 day longitudinal study with 10 participants that evaluated speed, accuracy, and learning. DualKey outperformed the existing techniques on long-term performance with a speed of 19.6 WPM. We then optimized the keyboard layout for reducing finger switching time based on the study data. A second 10 day study with eight participants showed that the new sweqty layout improved upon DualKey even further to 21.59 WPM for long-term speed, was comparable to existing techniques on novice speed and outperformed existing techniques on novice accuracy rate.

Tactile displays have predominantly been used for information transfer using patterns or as assistive feedback for interactions. With recent advances in hardware for conveying increasingly rich tactile information that mirrors visual information, and the increasing viability of wearables that remain in constant contact with the skin, there is a compelling argument for exploring tactile interactions as rich as visual displays. Direct Manipulation underlies much of the advances in visual interactions. In this work, we introduce the concept of a Direct Manipulation-enabled Tactile display (DMT). We define the concepts of a tactile screen, tactile pixel, tactile pointer, and tactile target which enable tactile pointing, selection and drag & drop. We build a proof of concept tactile display and study its precision limits. We further develop a performance model for DMTs based on a tactile target acquisition study. Finally, we study user performance in a real-world DMT menu application. The results show that users are able to use the application with relative ease and speed.

We present Storeoboard, a system for stereo-cinematic conceptualization, via storyboard sketching directly in stereo. The resurgence of stereoscopic media has motivated filmmakers to evolve a new stereo-cinematic vocabulary, as many principles for stereo 3D film are unique. Concepts like plane separation, parallax position, and depth budgets are missing from early planning due to the 2D nature of existing storyboards. Storeoboard is the first of its kind, allowing filmmakers to explore, experiment and conceptualize ideas in stereo early in the film pipeline, develop new stereo-cinematic constructs and foresee potential difficulties. Storeoboard is the design outcome of interviews and field work with directors, stereographers, and storyboard artists. We present our design guidelines and implementation of a tool combining stereo-sketching, depth manipulations and storyboard features into a coherent and novel workflow. We report on feedback from storyboard artists, industry professionals and the director of a live action, feature film on which Storeoboard was deployed.

The lack of dedicated multitasking interface features in smartphones has resulted in users attempting a sequential form of multitasking via frequent app switching. In addition to the obvious temporal cost, it requires physical and cognitive effort which increases multifold as the back and forth switching becomes more frequent. We propose porous interfaces, a paradigm that combines the concept of translucent windows with finger identification to support efficient multitasking on small screens. Porous interfaces enable partially transparent app windows overlaid on top of each other, each of them being accessible simultaneously using a different finger as input. We design porous interfaces to include a broad range of multitasking interactions with and between windows, while ensuring fidelity with the existing smartphone interactions. We develop an end-to-end smartphone interface that demonstrates porous interfaces. In a qualitative study, participants found porous interfaces intuitive, easy, and useful for frequent multitasking scenarios.

Haptic learning of gesture shortcuts has never been explored. In this paper, we investigate haptic learning of a freehand semaphoric finger tap gesture shortcut set using haptic rings. We conduct a two-day study of 30 participants where we couple haptic stimuli with visual and audio stimuli, and compare their learning performance with wholly visual learning. The results indicate that with <30 minutes of learning, haptic learning of finger tap semaphoric gestures is comparable to visual learning and maintains its recall on the second day.

Virtual Reality (VR) narratives have the unprecedented potential to connect with an audience through presence, placing viewers within the narrative. The onset of consumer VR has resulted in an explosion of interest in immersive storytelling. Planning narratives for VR, however, is a grand challenge due to its unique affordances, its evolving cinematic vocabulary, and most importantly the lack of supporting tools to explore the creative process in VR.

Video authoring activity typically consists of three phases: planning (pre-production), capture (production) and processing (post-production). The status quo is that these phases occur separately, and the latter two have a significant amount of "slack time", where the camera operator is watching the scene unfold during capture, and the editor is re-watching and navigating through recorded footage during post-production. While this process is well suited to creating polished or professional video, video clips produced by casual video makers as seen in online forums could benefit from some editing without the overhead of current authoring tools. We introduce LACES, a tablet-based system enabling simple video manipulations in the midst of filming. Seamless in-situ integration of video capture and manipulation forms a novel workflow, allowing greater spontaneity and exploration of video creation.

With the ever increasing amount of digital information, users desire more screen real estate to process daily desktop computing work, and might well benefit from using a large high-resolution display for information management. Unfortunately, we know very little about users' behaviors when using such a display for daily computing, and current user interfaces are mainly designed for normal-sized desktop monitors, which might not well suit a large display. In this article, we first present a longitudinal study that investigates how users manage overflowing digital information on a wall-sized display in a personal desktop computing context by comparing it with single and dual desktop monitors. Results showed users' unanimous preferences of working on a large display and revealed large-display users' unique activity patterns of managing windows. Guided by the study results, we designed a set of interaction techniques that provide greater flexibility in managing multiple windows. They include facile methods for selecting, moving, and resizing multiple windows using an active window boundary called Fringe, rearranging selected windows using multi- and single-window marking menus, packing/unpacking the selected windows using easily activated icons, and freely adjusting the order of overlapping windows with a Jab-to-Lift operation. We coherently integrated these techniques with traditional operations in a large-display window management prototype called WallTop. Two rounds of usability testing showed that users can quickly and easily learn the new interaction techniques and apply them to realistic window management tasks on a large display with increased efficiency.

We explore the design space for using object motion trajectories to create and edit visual elements in various media across space and time. We introduce a suite of pen-based techniques that facilitate fluid stylization, annotation and editing of space-time content such as video, slide presentations and 2D animation, utilizing pressure and multi-touch input. We implemented and evaluated these techniques in DirectPaint, a system for creating free-hand painting and annotation over video.

Speech certainly has advantages as an input modality for smartphone applications, especially in scenarios where using touch or keyboard entry is difficult, on increasingly miniaturized devices where useable keyboards are difficult to accommodate, or in scenarios where only small amounts of text need to be input, such as when entering SMS texts or responding to a CAPTCHA challenge. In this paper, we propose two new alternative ways to design CAPTCHAs in which the user says the answer instead of typing it with (a) output stimuli provided visually (SeeSay) or (b) auditorily (HearSay). Our user study results show that SeeSay CAPTCHA requires less time to be solved and users prefer it over current text-based CAPTCHA methods.

Web applications designed for map exploration in local neighborhoods have become increasingly popular and important in everyday life. During the information-seeking process, users often revisit previously viewed locations, repeat earlier searches, or need to memorize or manually mark areas of interest. To facilitate rapid returns to earlier views during map exploration, we propose a novel algorithm to automatically generate map bookmarks based on a user's interaction. TrailMap, a web application based on this algorithm, is developed, providing a fluid and effective neighborhood exploration experience. A one-week study is conducted to evaluate TrailMap in users' everyday web browsing activities. Results showed that TrailMap's implicit bookmarking mechanism is efficient for map exploration and the interactive and visual nature of the tool is intuitive to users.

Many datasets, such as scientific literature collections, contain multiple heterogeneous facets which derive implicit relations, as well as explicit relational references between data items. The exploration of this data is challenging not only because of large data scales but also the complexity of resource structures and semantics. In this paper, we present PivotSlice, an interactive visualization technique which provides efficient faceted browsing as well as flexible capabilities to discover data relationships. With the metaphor of direct manipulation, PivotSlice allows the user to visually and logically construct a series of dynamic queries over the data, based on a multi-focus and multi-scale tabular view that subdivides the entire dataset into several meaningful parts with customized semantics. PivotSlice further facilitates the visual exploration and sensemaking process through features including live search and integration of online data, graphical interaction histories and smoothly animated visual state transitions. We evaluated PivotSlice through a qualitative lab study with university researchers and report the findings from our observations and interviews. We also demonstrate the effectiveness of PivotSlice using a scenario of exploring a repository of information visualization literature.

Global crowdsourcing platforms could offer new employment opportunities to low-income workers in developing countries. However, the impact to date has been limited because poor communities usually lack access to computers and the Internet.

Touchscreen phones tend to require constant visual attention, thus not allowing eyes-free interaction. For users with visual impairment, or when occupied with another task that requires a user's visual attention, these phones can be difficult to use. Recently, marks initiating from the bezel, the physical touch-insensitive frame surrounding a touchscreen display, have been proposed as a method for eyes-free interaction. Due to the physical form factor of the mobile device, it is possible to access different parts of the bezel eyes-free. In this paper, we first studied the performance of different bezel menu layouts. Based on the results, we designed a bezel-based text entry application to gain insights into how bezel menus perform in a real-world application. From a longitudinal study, we found that the participants achieved 9.2 words per minute in situations requiring minimal visual attention to the screen. After only one hour of practice, the participants transitioned from novice to expert users. This shows that bezel menus can be adopted for realistic applications.

A discourse parser is a natural language processing system which can represent the organization of a document based on a rhetorical structure tree-one of the key data structures enabling applications such as text summarization, question answering and dialogue generation. Computational linguistics researchers currently rely on manually exploring and comparing the discourse structures to get intuitions for improving parsing algorithms. In this paper, we present DAViewer, an interactive visualization system for assisting computational linguistics researchers to explore, compare, evaluate and annotate the results of discourse parsers. An iterative user-centered design process with domain experts was conducted in the development of DAViewer. We report the results of an informal formative study of the system to better understand how the proposed visualization and interaction techniques are used in the real research environment.

The need for pattern discovery in long time-series data led researchers to develop interactive visualization tools and analytical algorithms for gaining insight into the data. Most of the literature on time-series data visualization either focus on a small number of tasks or a specific domain. We propose KronoMiner, a tool that embeds new interaction and visualization techniques as well as analytical capabilities for the visual exploration of time-series data. The interface's design has been iteratively refined based on feedback from expert users. Qualitative evaluation with an expert user not involved in the design process indicates that our prototype is promising for further research.

Verifying that a live human is interacting with an automated speech based system is needed in some applications such as biometric authentication. In this paper, we present a method to verify that the user is human. Simply stated, our method asks the user to repeat a sentence. The reply is analyzed to verify that it is the requested sentence and said by a human, not a speech synthesis system. Our method is taking advantage of both speech synthesizer and speech recognizer limitations to detect computer programs, which is new, and potentially more accessible, way to develop CAPTCHA systems. Using an acoustic model trained on voices of over 1000 users, our system can verify the user’s answer with 98% accuracy and with 80% success in distinguishing humans from computers. Index Terms: Accessibility, CAPTCHA, Speech Recognition, Speech Synthesis

The developing world faces infrastructural challenges in providing Western-style educational computing technologies, but on the other hand observes very high cell phone penetration. However, the use of mobile technology has not been extensively explored in the context of collaborative learning. New projection and display technologies for mobile devices raise the important question of whether to use single or multiple displays in these environments. In this paper, we explore two mobile-based techniques for using co-located collaborative game-play to supplement ESL (English as a Second Language) education in a developing region: (1) Mobile Single Display Groupware: a pico-projector connected to a cell phone, with a handheld controller for each child to interact, and (2) Mobile Multiple Display Groupware: a phone for each child. We explore the types of interaction that occur in both of these conditions and the impact on learning outcomes.

We explore the potential of using camera phones and pico projectors in rapid creation and presentation of digital content in a development context. A camera phone based content authoring application was designed and deployed with three different user populations in the domains of classroom education and health care. Our findings show that despite the variations in education levels, cultural background, and technology exposure, users successfully created and presented different forms of digital content using the camera phone and pico projector.

Elasticurves present a novel approach to neaten sketches in real-time, resulting in curves that combine smoothness with user-intended detail. Inspired by natural variations in stroke speed when drawing quickly or with precision, we exploit stroke dynamics to distinguish intentional fine detail from stroke noise. Combining inertia and stroke dynamics, elasticurves can be imagined as the trace of a pen attached to the user by an oscillation-free elastic band. Sketched quickly, the elasticurve spatially lags behind the stroke, smoothing over stroke detail, but catches up and matches the input stroke at slower speeds. Connectors, such as lines or circular-arcs link the evolving elasticurve to the next input point, growing the curve by a responsiveness fraction along the connector. Responsiveness is calibrated, to reflect drawing skill or device noise. Elasticurves are theoretically sound and robust to variations in stroke sampling. Practically, they neaten digital strokes in real-time while retaining the modeless and visceral feel of pen on paper.

Visual representations of time-series are useful for tasks such as identifying trends, patterns and anomalies in the data. Many techniques have been devised to make these visual representations more scalable, enabling the simultaneous display of multiple variables, as well as the multi-scale display of time-series of very high resolution or that span long time periods. There has been comparatively little research on how to support the more elaborate tasks associated with the exploratory visual analysis of timeseries, e.g., visualizing derived values, identifying correlations, or discovering anomalies beyond obvious outliers. Such tasks typically require deriving new time-series from the original data, trying different functions and parameters in an iterative manner. We introduce a novel visualization technique called ChronoLenses, aimed at supporting users in such exploratory tasks. ChronoLenses perform on-the-fly transformation of the data points in their focus area, tightly integrating visual analysis with user actions, and enabling the progressive construction of advanced visual analysis pipelines.

Tiled-monitor large displays are widely used in various application domains. However, how their interior bezels affect user performance and behavior has not been fully understood. We conducted three controlled experiments to investigate effects of tiled-monitor interior bezels on visual search, straight-tunnel steering, and target selection tasks. The conclusions of our paper are: 1) interior bezels do not affect visual search time nor error rate; however, splitting objects across bezels is detrimental to search accuracy, 2) interior bezels are detrimental to straight-tunnel steering, but not to target selection. In addition, we discuss how inte-rior bezels affect user behaviors, and suggest guidelines for effectively using tiled-monitor large displays and designing user interfaces suited to them.

We present a study which investigated how and why users of Mozilla Firefox use multiple tabs and windows during web browsing. The detailed web browsing usage of 21 participants was logged over a period of 13 to 21 days each, and was supplemented by qualitative data from diary entries and interviews. Through an examination of several measures of their tab usage, we show that our participants had a strong preference for the use of tabs rather than multiple windows. We report the reasons they cited for using tabs, and the advantages over multiple windows. We identify several common tab usage patterns which browsers could explicitly support. Finally, we look at how tab usage affects web page revisitation. Most of our participants switched tabs more often than they used the back button, making tab switching the second most important navigation mechanism in the browser, after link clicking.

We define occlusion-aware interfaces as interaction techniques which know what area of the display is currently occluded, and use this knowledge to counteract potential problems and/or utilize the hidden area. As a case study, we describe the Occlusion-Aware Viewer, which identifies important regions hidden beneath the hand and displays them in a non-occluded area using a bubble-like callout. To determine what is important, we use an application agnostic image processing layer. For the occluded area, we use a user configurable, real-time version of Vogel et al.'s [21] geometric model. In an evaluation with a simultaneous monitoring task, we find the technique can successfully mitigate the effects of occlusion, although issues with ambiguity and stability suggest further refinements. Finally, we present designs for three other occlusion-aware techniques for pop-ups, dragging, and a hidden widget.

A recent trend in interface design for classrooms in developing regions has many students interacting on the same display using mice. Text entry has emerged as an important problem preventing such mouse-based singledisplay groupware systems from offering compelling interactive activities. We explore the design space of mouse-based text entry and develop 13 techniques with novel characteristics suited to the multiple mouse scenario. We evaluated these in a 3-phase study over 14 days with 40 students in 2 developing region schools. The results show that one technique effectively balanced all of our design dimensions, another was most preferred by students, and both could benefit from augmentation to support collaborative interaction. Our results also provide insights into the factors that create an optimal text entry technique for single-display groupware systems.

RearType is a text input system for mobile devices such as Tablet PCs, using normal keyboard keys but on the reverse side of the device. The standard QWERTY layout is split and rotated so that hands gripping the device from either side have the usual keys under the fingers. This frees up the front of the device, maximizing the use of the display for visual output, eliminating the need for an onscreen keyboard and the resulting hand occlusion, and providing tactile and multi-finger text entry - with potential for knowledge transfer from QWERTY. Using a prototype implementation which includes software visualization of the keys to assist with learning, we conducted a study to explore the initial learning curve for RearType. With one hour's training, RearType typing speed was an average 15 WPM, and was not statistically different to a touchscreen keyboard.

We present the satellite cursor - a novel technique that uses multiple cursors to improve pointing performance by reducing input movement. The satellite cursor associates every target with a separate cursor in its vicinity for pointing, which realizes the MAGIC (manual and gaze input cascade) pointing method without gaze tracking. We discuss the problem of visual clutter caused by multiple cursors and propose several designs to mitigate it. Two controlled experiments were conducted to evaluate satellite cursor performance in a simple reciprocal pointing task and a complex task with multiple targets of varying layout densities. Results show the satellite cursor can save significant mouse movement and consequently pointing time, especially for sparse target layouts, and that satellite cursor performance can be accurately modeled by Fitts' Law.

One possibility presented by novel communication technologies is the ability for remotely located experts to provide guidance to others who are performing difficult technical tasks in the real world, such as medical procedures or engine repair. In these scenarios, video views and other visual information seem likely to be useful in the ongoing negotiation of shared understanding, or common ground, but actual results with experimental systems have been mixed. One difficulty in designing these systems is achieving a balance between close-up shots that allow for discussion of detail and wide shots that allow for orientation or establishing a mutual point of focus in a larger space. Achieving this balance can be difficult without disorienting or overloading task participants. In this article we present results from two experiments involving three automated camera control systems for remote repair tasks. Results show that a system providing both detailed and overview information was superior to systems providing only one or the other in terms of performance but that some participants preferred the detail-only system.

We examine the usability and performance of Tablet PC direct pen input with a conventional graphical user interface (GUI). We use a qualitative observational study design with 16 participants divided into 4 groups: 1 mouse group for a baseline control and 3 Tablet PC groups recruited according to their level of experience. The study uses a scripted scenario of realistic tasks and popular office applications designed to exercise standard GUI components and cover typical interactions such as parameter selection, object manipulation, text selection, and ink annotation. We capture a rich set of logging data including 3D motion capture, video taken from the participants' point-of-view, screen capture video, and pen events such as movement and taps. To synchronize, segment, and annotate these logs, we used our own custom analysis software. We find that pen participants make more errors, perform inefficient movements, and express frustration during many tasks. Our observations reveal overarching problems with direct pen input: poor precision when tapping and dragging, errors caused by hand occlusion, instability and fatigue due to ergonomics and reach, cognitive differences between pen and mouse usage, and frustration due to limited input capabilities. We believe these to be the primary causes of nontext errors, which contribute to user frustration when using a pen with a conventional GUI. Finally, we discuss how researchers could address these issues without sacrificing the consistency of current GUIs and applications by making improvements at three levels: hardware, base interaction, and widget behavior.

With the ever increasing amount of digital information, users desire more screen real estate to process their daily computing work, and might well benefit from using a wall-size large high-resolution display instead of a desktop one. Unfortunately, we know very little about users' behaviors when using such a display for daily computing. We present a week-long study that investigates large display use in a personal desktop computing context by comparing it with single and dual desktop monitor use. Results show users' unanimous preference for using a large display: it facilitates multi-window and rich information tasks, enhances users' awareness of peripheral applications, and offers a more"immersive experience. Further, the data reveals distinct usage patterns in partitioning screen real estate and managing windows on a large display. Detailed analysis of these results provides insights into designing interaction techniques and window management systems more suited to a large display.

Multimodal interfaces with little or no text have been shown to be useful for users with low literacy. However, this research has not differentiated between the needs of the fully illiterate and semiliterate - those who have basic literacy but cannot read and write fluently. Text offers a fast and unambiguous mode of interaction for literate users and the exposure to text may allow for incidental improvement of reading skills. We conducted two studies that explore how semiliterate users with very little education might benefit from a combination of text and audio as compared to illiterate and literate users. Results show that semiliterate users reduced their use of audio support even during the first hour of use and over several hours this reduction was accompanied by a gain in visual word recognition; illiterate users showed no similar improvement. Semiliterate users should thus be treated differently from illiterate users in interface design.

People's ability to accurately locate target objects in images is severely affected by the prevalence of the sought objects. This negative effect greatly impacts critical real world tasks, such as baggage screening and cell slide pathology, in which target objects are rare. We present three novel image presentation techniques that are designed to improve visual search. Our techniques rely on the images being broken into image segments, which are then recombined or displayed in novel ways. The techniques and their underlying design reasoning are described in detail, and three experiments are presented that provide initial evidence that these techniques lead to better search performance in a simulated cell slide pathology task.

The Codex is a dual-screen tablet computer, about the size of a 4"x 6 day planner, with a self-supporting binding and embedded sensors. The device can be oriented in a variety of postures to support different nuances of individual work, ambient display, or collaboration with another user. In the context of a pen-operated note taking application, we demonstrate interaction techniques that support a fluid division of labor for tasks and information across the two displays while minimizing disruption to the primary experience of authoring notes.

Mischief is a system for classroom interaction that allows multiple children to use individual mice and cursors to interact with a single large display [20]. While the system can support large groups of children, it is unclear how children's performance is affected as group size increases. We explore this question via a study involving two tasks, with children working in group sizes ranging from 1 to 32. The first required reciprocal selection of two on-screen targets, resembling a swarm pointing scenario that might be used in educational applications. The second, a more temporally and spatially distributed pointing task, had children entering different words by selecting characters on an on-screen keyboard. Results indicate that performance is significantly affected by group size only when targets are small. Further, group size had a smaller effect when pointing was spatially and temporally distributed than when everyone was concurrently aiming at the same targets.

We present results from an experiment examining the area occluded by the hand when using a tablet-sized direct pen input device. Our results show that the pen, hand, and forearm can occlude up to 47% of a 12 inch display. The shape of the occluded area varies between participants due to differences in pen grip rather than simply anatomical differences. For the most part, individuals adopt a consistent posture for long and short selection tasks. Overall, many occluded pixels are located higher relative to the pen than previously thought. From the experimental data, a five-parameter scalable circle and pivoting rectangle geometric model is presented which captures the general shape of the occluded area relative to the pen position. This model fits the experimental data much better than the simple bounding box model often used implicitly by designers. The space of fitted parameters also serves to quantify the shape of occlusion. Finally, an initial design for a predictive version of the model is discussed.

Featherweight multimedia devices combine audio with non-electronic visual displays (e.g., paper). Because of their low cost, customizability, durability, storage capacity, and energy efficiency, they are well-suited for education and information dissemination among illiterate and semi-literate people. We present a taxonomy of featherweight multimedia devices and also derive design recommendations from our experiences deploying featherweight multimedia in the agriculture and health domains in India. We found that with some initial guidance, illiterate users can quickly learn to use and enjoy the device, especially if they are taught by peers.

Non-governmental organizations (NGOs) working in disadvantaged communities have a variety of data-collection and analysis needs, for example, for performing surveys or monitoring programs. Because much of this data collection occurs in environments with insufficient IT support and infrastructure, and among populations not always comfortable with technology, paper forms rather than electronic methods remain the predominant means for data collection.

We present EverybodyLovesSketch, a gesture-based 3D curve sketching system for rapid ideation and visualization of 3D forms, aimed at a broad audience. We first analyze traditional perspective drawing in professional practice. We then design a system built upon the paradigm of ILoveSketch, a 3D curve drawing system for design professionals. The new system incorporates many interaction aspects of perspective drawing with judicious automation to enable novices with no perspective training to proficiently create 3D curve sketches. EverybodyLovesSketch supports a number of novel interactions: tick-based sketch plane selection, single view definition of arbitrary extrusion vectors, multiple extruded surface sketching, copy-and-project of 3D curves, freeform surface sketching, and an interactive perspective grid. Finally, we present a study involving 49 high school students (with no formal artistic training) who each learned and used the system over 11 days, which provides detailed insights into the popularity, power and usability of the various techniques, and shows our system to be easily learnt and effectively used, with broad appeal.

Previous work has demonstrated the viability of applying offline analysis to interpret forearm electromyography (EMG) and classify finger gestures on a physical surface. We extend those results to bring us closer to using muscle-computer interfaces for always-available input in real-world applications. We leverage existing taxonomies of natural human grips to develop a gesture set covering interaction in free space even when hands are busy with other objects. We present a system that classifies these gestures in real-time and we introduce a bi-manual paradigm that enables use in interactive systems. We report experimental results demonstrating four-finger classification accuracies averaging 79% for pinching, 85% while holding a travel mug, and 88% when carrying a weighted bag. We further show generalizability across different arm postures and explore the tradeoffs of providing real-time visual feedback.

Until recently, most of our computing was carried out in environments with a single display. In contrast, modern computing environments are evolving into multi-display environments. Simple dual-mon...

We present work that explores whether the asynchronous peer-to-peer communication capabilities of email can be made accessible to illiterate populations in the developing world. Building on metaphors from traditional communication systems such as postal mail, and relevant design principles established by previous research into text-free interfaces, we designed and evaluated a prototype asynchronous communication application built on standard email protocols. We considered different message formats -- text, freeform ink, audio, and video + audio -- and via iterative usage and design sessions, determined that video + audio was the most viable. Design alternatives for authentication processes were also explored. Our prototype was refined over three usability iterations, and the final version evaluated in a two-stage study with 20 illiterate users from an urban slum in Bangalore, India. Our results are mixed: On the one hand, the results show that users can understand the concept of video mail. They were able to successfully complete tasks ranging from account setup to login to viewing and creating mail, but required assistance from an online audio assistant. On the other hand, there were some surprising challenges such as a consistent difficulty understanding the notion of asynchronicity. The latter suggests that more work on the paradigm is required before the benefits of email can be brought to illiterate users.

Peephole interaction occurs when a spatially aware display is moved and acts as a viewport to reveal different parts of the virtual space that cannot all fit within the display at once. We investigate pointing within this peephole metaphor, where the targets may not be initially visible on the display, but are dynamically revealed by moving the display. We develop and experimentally validate a quantitative model for peephole pointing. Our results indicate that the model accurately accounts for peephole pointing for a variety of display sizes, both with and without users' having prior knowledge of the target location.

We present a method for browsing videos by directly dragging their content. This method brings the benefits of direct manipulation to an activity typically mediated by widgets. We support this new type of interactivity by: 1) automatically extracting motion data from videos; and 2) a new technique called relative flow dragging that lets users control video playback by moving objects of interest along their visual trajectory. We show that this method can outperform the traditional seeker bar in video browsing tasks that focus on visual content rather than time.

We present a pair of experiments that explore the effects of tactile-feedback and direct vs. indirect pen input on pointing and crossing selection tasks. While previous work has demonstrated the validity of crossing as a useful selection mechanism for pen-based computing, those experiments were conducted using an indirect input device -- one in which the pen-input and display were separated. We investigate users' performance with pointing and crossing interfaces controlled via not only an indirect input device, but also a direct input device -- one in which the pen-input and display are co-located. Results show that direct input significantly outperforms indirect input for crossing selection, but the two modalities are essentially equivalent in pointing selection. A small amount of tactile feedback is shown to be beneficial for both pointing and crossing selection, most noticeably in crossing tasks when using direct input where visual feedback is often occluded by a hand or stylus.

Volumetric displays possess a number of unique properties which potentially make them particularly suitable for collaborative 3D applications. Because such displays have only recently become available, interaction techniques for collaborative usage have yet to be explored. In this paper, we initiate this exploration. We present a prototype collaborative 3D model viewing application, which served as a platform for our explorations. We outline three design goals, discuss the key interaction issues which were encountered, and describe a suite of new techniques in detail. In initial user observation sessions, we found that our techniques allowed users to successfully complete a variety of 3D tasks. Furthermore, interviews with experts in potential usage domains indicated that the techniques we developed can serve as a baseline for future collaborative applications for volumetric displays.

Video recordings of meetings are often monotonous and tedious to watch. In this paper, we report on the design, implementation and evaluation of an automated meeting capture system that applies television production principles to capture and present videos of small group meetings in a compelling manner. The system uses inputs from a motion capture system and microphones to drive multiple pan-tilt-zoom cameras and uses heuristics to frame shots and cut between them. An evaluation of the system indicates that its performance approaches that of a professional crew while requiring significantly fewer human resources.

We explore the feasibility of muscle-computer interfaces (muCIs): an interaction methodology that directly senses and decodes human muscular activity rather than relying on physical device actuation or user actions that are externally visible or audible. As a first step towards realizing the mu-CI concept, we conducted an experiment to explore the potential of exploiting muscular sensing and processing technologies for muCIs. We present results demonstrating accurate gesture classification with an off-the-shelf electromyography (EMG) device. Specifically, using 10 sensors worn in a narrow band around the upper forearm, we were able to differentiate position and pressure of finger presses, as well as classify tapping and lifting gestures across all five fingers. We conclude with discussion of the implications of our results for future muCI designs.

As large displays become prevalent in public spaces, they could be employed to create novel game experiences for the public. We present an exploratory study of an ad-hoc multi-player game played on such public displays. The game, Flashlight Jigsaw, was deployed in a shared lab space and a public atrium for two weeks in total. Through interviews supported by observations and system logs we explored the experiences and behaviors of players and spectators. We also investigated the interrelationship between public display games and the spaces they are deployed in. The research resulted in findings regarding game play, communication, social interaction, spectatorship, and space and location around such a game. We use our findings to develop design implications for future public display games.

We present ILoveSketch, a 3D curve sketching system that captures some of the affordances of pen and paper for professional designers, allowing them to iterate directly on concept 3D curve models. The system coherently integrates existing techniques of sketch-based interaction with a number of novel and enhanced features. Novel contributions of the system include automatic view rotation to improve curve sketchability, an axis widget for sketch surface selection, and implicitly inferred changes between sketching techniques. We also improve on a number of existing ideas such as a virtual sketchbook, simplified 2D and 3D view navigation, multi-stroke NURBS curve creation, and a cohesive gesture vocabulary. An evaluation by a professional designer shows the potential of our system for deployment within a real design process.

Sphere is a multi-user, multi-touch-sensitive spherical display in which an infrared camera used for touch sensing shares the same optical path with the projector used for the display. This novel configuration permits: (1) the enclosure of both the projection and the sensing mechanism in the base of the device, and (2) easy 360-degree access for multiple users, with a high degree of interactivity without shadowing or occlusion. In addition to the hardware and software solution, we present a set of multi-touch interaction techniques and interface concepts that facilitate collaborative interactions around Sphere. We designed four spherical application concepts and report on several important observations of collaborative activity from our initial Sphere installation in three high-traffic locations.

Current pen input mainly utilizes the position of the pen tip, and occasionally, a button press. Other possible device parameters, such as rolling the pen around its longitudinal axis, are rarely used. We explore pen rolling as a supporting input modality for pen-based interaction. Through two studies, we are able to determine 1) the parameters that separate intentional pen rolling for the purpose of interaction from incidental pen rolling caused by regular writing and drawing, and 2) the parameter range within which accurate and timely intentional pen rolling interactions can occur. Building on our experimental results, we present an exploration of the design space of rolling-based interaction techniques, which showcase three scenarios where pen rolling interactions can be useful: enhanced stimulus-response compatibility in rotation tasks [7], multi-parameter input, and simplified mode selection.

Abstract We present an interface for 3D object manipulation in which standard transformation tools are replaced with transient 3D widgets invoked by sketching context‐dependent strokes. The widgets are automatically aligned to axes and planes determined by the user's stroke. Sketched pivot‐points further expand the interaction vocabulary. Using gestural commands, these basic elements can be assembled into dynamic, user‐constructed 3D transformation systems. We supplement precise widget interaction with techniques for coarse object positioning and snapping. Our approach, which is implemented within a broader sketch‐based modeling system, also integrates an underlying “widget history” to enable the fluid transfer of widgets between objects. An evaluation indicates that users familiar with 3D manipulation concepts can be taught how to efficiently use our system in under an hour.

We theoretically and empirically examine the impact of control display (CD) gain on mouse pointing performance. Two techniques for modifying CD gain are considered: constant gain (CG) where CD gain is uniformly adjusted by a constant multiplier, and pointer acceleration (PA) where CD gain is adjusted using a nonuniform function depending on movement characteristics. Both CG and PA are evaluated at various levels of relationship between mouse and cursor movement: from low levels, which have a near one-to-one mapping, through to high levels that aggressively amplify mouse movement. We further derive a model predicting the modification in motor-space caused by pointer acceleration. Experiments are then conducted on a standard desktop display and on a very large high-resolution display, allowing us to measure performance in high index of difficulty tasks where the effect of clutching may be pronounced. The evaluation apparatus was designed to minimize device quantization effects and used accurate 3D motion tracking equipment to analyze users' limb movements. On both displays, and in both gain techniques, we found that low levels of CD gain had a marked negative effect on performance, largely because of increased clutching and maximum limb speeds. High gain levels had relatively little impact on performance, with only a slight increase in time when selecting very small targets at high levels of constant gain. On the standard desktop display, pointer acceleration resulted in 3.3% faster pointing than constant gain and up to 5.6% faster with small targets. This supported the theoretical prediction of motor-space modification but fell short of the theoretical potential, possibly because PA caused an increase in target overshooting. Both techniques were accurately modeled by Fitts' law in all gain settings except for when there was a significant amount of clutching. From our results, we derive a usable range of CD gain settings between thresholds of speed and accuracy given the capabilities of a pointing device, display, and the expected range of target widths and distances.

This paper reports on an exploratory study of the effects of input configuration on group behavior and performance in a collaborative task performed by a collocated group using a large display. Twelve groups completed a mixed-motive negotiation task under two conditions: a single, shared mouse and one mouse per person. Results suggest that the multiple mouse condition allowed for more parallel work, but the quality of discussion was higher in the single mouse condition. Moreover, participants were more likely to act in their own best interest in the multiple mouse condition.

We investigate the differences -- in terms of bothquantitative performance and subjective preference -- between direct-touch and mouse input for unimanual andbimanual tasks on tabletop displays. The results of twoexperiments show that for bimanual tasks performed ontabletops, users benefit from direct-touch input. However,our results also indicate that mouse input may be moreappropriate for a single user working on tabletop tasksrequiring only single-point interaction.

Hotkeys are extremely useful in leveraging expert performance, but learning them is a slow process. This paper investigates alternative menu designs that can motivate and help users remember associations between menu commands and hotkeys. Building upon previous work on paired-associate learning, we suggest that the transition to expert use can be accelerated by manipulating feedback and cost associated with menu selection. We evaluate five designs in a pilot study and then two of the most promising ones in a formal experiment, showing that the speed of hotkey learning can indeed be significantly increased with little modifications to the standard menu/hotkey paradigm.

We investigate pointing at graphical targets of arbitrary shapes. We first describe a previously proposed probabilistic Fitts' law model [7] which, unlike previous models that only account for rectangular targets, has the potential to handle arbitrary shapes. Three methods of defining the centers of arbitrarily shaped targets for use within the model are developed. We compare these methods of defining target centers, and validate the model using a pointing experiment in which the targets take on various shapes. Results show that the model can accurately account for the varying target shapes. We discuss the implications of our results to interface design.

Volumetric displays, which provide a 360° view of imagery illuminated in true 3D space, are a promising platform for interactive 3D applications. However, presenting text in volumetric displays can be a challenge, as the text may not be oriented towards the user. This is especially problematic with multiple viewers, as the text could, for example, appear forwards to one user, and backwards to another. In a first experiment we determined the effects of 3D rotations on text readability. Based on the results, we developed and evaluated a new technique which optimizes text orientation for multiple viewers. This technique provided 33% faster group reading times in a collaborative experimental task.

Selections and actions in GUI's are often separated -- i.e. an action or command typically follows a selection. This sequence imposes a lower bound on the interaction time that is equal to or greater than the sum of its parts. In this paper, we introduce pressure marks -- pen strokes where the variations in pressure make it possible to indicate both a selection and an action simultaneously. We propose a series of design guidelines from which we develop a set of four basictypes of pressure marks. We first assess the viability of this set through an exploratory study that looks at the way users draw straight and lasso pressure marks of different sizes and orientations. We then present the results of a quantitative experiment that shows that users perform faster selection-action interactions with pressure marks than with a combination of lassos and pigtails. Based on these results, we present and discuss a number of interaction designs that incorporate pressure marks.

Using a stylus on a tablet computer to acquire small targets can be challenging. In this paper we present pointing lenses -- interaction techniques that help users acquire and select targets by presenting them with an enlarged visual and interaction area. We present and study three pointing lenses for pen-based systems and find that our proposed Pressure-Activated Lens is the top overall performer in terms of speed, accuracy and user preference. In addition, our experimental results not only show that participants find all pointing lenses beneficial for targets smaller than 5 pixels, but they also suggest that this benefit may extend to larger targets as well.

We present an experimental study of automatic camera control in the performance of collaborative remote repair tasks using video-mediated communication. Twelve pairs of participants, one "helper" and one "worker," completed a series of Lego puzzle tasks using both a static camera and an automatic camera system that was guided in part by tracking the worker's hand position. Results show substantial performance benefits for the automatic system, particularly for complex tasks. The implications of these results are discussed, along with some lessons for the use of motion tracking as a driver for camera control.

Information shown on a tabletop display can appear distorted when viewed by a seated user. Even worse, the impact of this distortion is different depending on the location of the information on the display. In this paper, we examine how this distortion affects the perception of the basic graphical elements of information visualization shown on displays at various angles. We first examine perception of these elements on a single display, and then compare this to perception across displays, in order to evaluate the effectiveness of various elements for use in a tabletop and multi-display environment. We found that the perception of some graphical elements is more robust to distortion than others. We then develop recommendations for building data visualizations for these environments.

We present the design and evaluation of earPod: an eyes-free menu technique using touch input and reactive auditory feedback. Studies comparing earPod with an iPod-like visual menu technique on reasonably-sized static menus indicate that they are comparable in accuracy. In terms of efficiency (speed), earPod is initially slower, but outperforms the visual technique within 30 minutes of practice. Our results indicate that earPod is potentially a reasonable eyes-free menu technique for general use, and is a particularly exciting technique for use in mobile device interfaces.

Recent research on handheld projector interaction has expanded the display and interaction space of handheld devices by projecting information onto the physical environment around the user, but has mainly focused on single-user scenarios. We extend this prior single-user research to co-located multi-user interaction using multiple handheld projectors. We present a set of interaction techniques for supporting co-located collaboration with multiple handheld projectors, and discuss application scenarios enabled by them.

We present an experiment that compares volumetric displays to existing 3D display techniques in three tasks that require users to perceive depth in 3D scenes. Because they generate imagery in true 3D space, volumetric displays allow viewers to use their natural physiological mechanisms for depth perception, without requiring special hardware such as head trackers or shutter glasses. However, it is unclear from the literature as to whether these displays are actually better than the status-quo for enabling the perception of 3D scenes, thus motivating the present study. Our results show that volumetric displays enable significantly better user performance in a simple depth judgment task, and better performance in a collision judgment task, but in its current form does not enhance user comprehension of more complex 3D scenes.

Tables have historically played a key role in many real-time collaborative environments, often referred to as "war rooms". Today, these environments have been transformed by computational technology into spaces with large vertical displays surrounded by numerous desktop computers. However, despite significant research activity in the area of tabletop computing, very little is known about how to best integrate a digital tabletop into these multi-surface environments. In this paper, we identify various design requirements for the implementation of a system intended to support such an environment. We then present a set of designs that demonstrate how an interactive tabletop can be used in a real-time operations center to facilitate collaborative situation-assessment and decision-making.

We explore making virtual desktops behave in a more physically realistic manner by adding physics simulation and using piling instead of filing as the fundamental organizational structure. Objects can be casually dragged and tossed around, influenced by physical characteristics such as friction and mass, much like we would manipulate lightweight objects in the real world. We present a prototype, called BumpTop, that coherently integrates a variety of interaction and visualization techniques optimized for pen input we have developed to support this new style of desktop organization.

We present Hover Widgets, a new technique for increasing the capabilities of pen-based interfaces. Hover Widgets are implemented by using the pen movements above the display surface, in the tracking state. Short gestures while hovering, followed by a pen down, access the Hover Widgets, which can be used to activate localized interface widgets. By using the tracking state movements, Hover Widgets create a new command layer which is clearly distinct from the input layer of a pen interface. In a formal experiment Hover Widgets were found to be faster than a more traditional command activation technique, and also reduced errors due to divided attention.

Searching audio data can potentially be facilitated by the use of automatic speech recognition (ASR) technology to generate text transcripts which can then be easily queried. However, since current ASR technology cannot reliably generate 100% accurate transcripts, additional techniques for fluid browsing and searching of the audio itself are required. We explore the impact of transcripts of various qualities, dichotic presentation, and time-compression on an audio search task. Results show that dichotic presentation and reasonably accurate transcripts can assist in the search process, but suggest that time-compression and low accuracy transcripts should be used carefully.

In many environments, it is often the case that input is made to displays that are positioned non-traditionally relative to one or more users. This typically requires users to perform interaction tasks under transformed input-display spatial mappings, and the literature is unclear as to how such transformations affect performance. We present two experiments that explore the impact of display space position and input control space orientation on user's subjective preference and objective performance in a docking task. Our results provide guidelines as to optimal display placement and control orientation in collaborative computing environments with one or more shared displays.

Visual search is the subject of countless psychology studies in which people search for target items within a scene. The bulk of this literature focuses on the individual with the goal of understanding the human perceptual system. In life, visual search is performed not only by individuals, but also by groups -- a team of doctors may study an x-ray and a team of analysts may study a satellite photograph. In this paper, we examine the issues one should consider when searching as a group. We present the details of an experiment designed to investigate the impact of group size on visual search performance, and how different display configurations affected that performance. We asked individuals, pairs, and groups of four people to participate in a baggage screening task in which these teams searched simulated x-rays for prohibited items. Teams conducted these searches on single monitors, a row of four monitors, and on a single horizontal display. Our findings suggest that groups commit far fewer errors in visual search tasks, although they may perform slower than individuals under certain conditions. The interaction between group size and display configuration turned out to be an important factor as well.

This paper reports on an exploratory experimental study of the relationships between physical movement and desired visual information in the performance of video-mediated collaborative tasks in the real world by geographically distributed groups. Twenty-three pairs of participants (one "helper" and one "worker") linked only by video and audio participated in a Lego construction task in one of three experimental conditions: a fixed scene camera, a helper-controlled pan-tilt-zoom camera, and a dedicated operator-controlled camera. "Worker" motion was tracked in 3-D space for all three conditions, as were all camera movements. Results suggest performance benefits for the operator-controlled condition, and the relationships between camera position/movement and worker action are explored to generate preliminary theoretical and design implications.

Managing large amounts of dynamic visual information involves understanding changes happening out of the user's sight. In this paper, we show how current software does not adequately support users in this task, and motivate the need for a more general approach. We propose an image-based storage, visualization, and implicit interaction paradigm called mnemonic rendering that provides better support for handling visual changes. Once implemented on a system, mnemonic rendering techniques can benefit all applications. We explore its rich design space and discuss its expected benefits as well as limitations based on feedback from users of a small-screen and a wall-size prototype.

The recent trend towards miniaturization of projection technology indicates that handheld devices will soon have the ability to project information onto any surface, thus enabling interfaces that are not possible with current handhelds. We explore the design space of dynamically defining and interacting with multiple virtual information spaces embedded in a physical environment using a handheld projector and a passive pen tracked in 3D. We develop techniques for defining and interacting with these spaces, and explore usage scenarios.

We present HybridPointing, a technique that lets users easily switch between absolute and relative pointing with a direct input device such as a pen. Our design includes a new graphical element, the Trailing Widget, which remains "close at hand" but does not interfere with normal cursor operation. The use of visual feedback to aid the user's understanding of input state is discussed, and several novel visual aids are presented. An experiment conducted on a large, wall-sized display validates the benefits of HybridPointing under certain conditions. We also discuss other situations in which HybridPointing may be useful. Finally, we present an extension to our technique that allows for switching between absolute and relative input in the middle of a single drag-operation.

Volumetric displays, which display imagery in true 3D space, are a promising platform for the display and manipulation of 3D data. To fully leverage their capabilities, appropriate user interfaces and interaction techniques must be designed. In this paper, we explore 3D selection techniques for volumetric displays. In a first experiment, we find a ray cursor to be superior to a 3D point cursor in a single target environment. To address the difficulties associated with dense target environments we design four new ray cursor techniques which provide disambiguation mechanisms for multiple intersected targets. Our techniques showed varied success in a second, dense target experiment. One of the new techniques, the depth ray, performed particularly well, significantly reducing movement time, error rate, and input device footprint in comparison to the 3D point cursor.

We explore the design space of a two-sided interactive touch table, designed to receive touch input from both the top and bottom surfaces of the table. By combining two registered touch surfaces, we are able to offer a new dimension of input for co-located collaborative groupware. This design accomplishes the goal of increasing the relative size of the input area of a touch table while maintaining its direct-touch input paradigm. We describe the interaction properties of this two-sided touch table, report the results of a controlled experiment examining the precision of user touches to the underside of the table, and a series of application scenarios we developed for use on inverted and two-sided tables. Finally, we present a list of design recommendations based on our experiences and observations with inverted and two-sided tables.

We present the design and evaluation of the vacuum, a new interaction technique that enables quick access to items on areas of a large display that are difficult for a user to reach without significant physical movement. The vacuum is a circular widget with a user controllable arc of influence that is centered at the widget's point of invocation and spans out to the edges of the display. Far away objects residing inside this influence arc are brought closer to the widget's centre in the form of proxies that can be manipulated in lieu of the original. We conducted two experiments which compare the vacuum to direct picking and an existing technique called drag-and-pick [2]. Results show that the vacuum outperforms existing techniques when selecting multiple targets in a sequence, performs similarly to existing techniques when selecting single targets located moderately far away, and slightly worse with single targets located very far away in the presence of distracter targets along the path.

We present the bubble cursor - a new target acquisition technique based on area cursors. The bubble cursor improves upon area cursors by dynamically resizing its activation area depending on the proximity of surrounding targets, such that only one target is selectable at any time. We also present two controlled experiments that evaluate bubble cursor performance in 1D and 2D target acquisition tasks, in complex situations with multiple targets of varying layout densities. Results show that the bubble cursor significantly outperforms the point cursor and the object pointing technique [7], and that bubble cursor performance can be accurately modeled and predicted using Fitts' law.

Designing interfaces for interactive handheld projectors is an exiting new area of research that is currently limited by two problems: hand jitter resulting in poor input control, and possible reduction of image resolution due to the needs of image stabilization and warping algorithms. We present the design and evaluation of a new interaction technique, called zoom-and-pick, that addresses both problems by allowing the user to fluidly zoom in on areas of interest and make accurate target selections. Subtle design features of zoom-and-pick enable pixel-accurate pointing, which is not possible in most freehand interaction techniques. Our evaluation results indicate that zoom-and-pick is significantly more accurate than the standard pointing technique described in our previous work.

We explore the idea of using vision-based hand tracking over a constrained tabletop surface area to perform multi-finger and whole-hand gestural interactions with large displays from a distance. We develop bimanual techniques to support a variety of asymmetric and symmetric interactions, including fast targeting and navigation to all parts of a large display from the comfort of a desk and chair, as well as techniques that exploit the ability of the vision-based hand tracking system to provide multi-finger identification and full 2D hand segmentation. We also posit a design that allows for handling multiple concurrent users.

High precision parameter manipulation tasks typically require adjustment of the scale of manipulation in addition to the parameter itself. This paper introduces the notion of Zoom Sliding, or Zliding, for fluid integrated manipulation of scale (zooming) via pressure input while parameter manipulation within that scale is achieved via x-y cursor movement (sliding). We also present the Zlider (Figure 1), a widget that instantiates the Zliding concept. We experimentally evaluate three different input techniques for use with the Zlider in conjunction with a stylus for x-y cursor positioning, in a high accuracy zoom and select task. Our results marginally favor the stylus with integrated isometric pressure sensing tip over bimanual techniques which separate zooming and sliding controls over the two hands. We discuss the implications of our results and present further designs that make use of Zliding.

We explore the design space of freehand pointing and clicking interaction with very large high resolution displays from a distance. Three techniques for gestural pointing and two for clicking are developed and evaluated. In addition, we present subtle auditory and visual feedback techniques to compensate for the lack of kinesthetic feedback in freehand interaction, and to promote learning and use of appropriate postures.

We investigate and model two-dimensional pointing where the target distance and size vary as does the angle of movement. We first study the spread of hits in a rapid approximate pointing task at varied distances and movement angles. Consistent with the literature, our results show that the spread of hits along the movement direction deviate more than the spread of hits in the direction perpendicular to movement, and both spreads increase with distance. Based on the distribution of this spread of hits, we propose and validate a new probabilistic model that describes two-dimensional pointing. Unlike previous models, our model accounts for more variables of two-dimensional pointing and can be generalized to any target shape, size, orientation, location, and dimension. In contrast to previous work, which suggests that target height has minimal impact on performance when it is larger than the width, our results show that, even when height is greater than width, it can significantly impact movement time.

Recently, there has been renewed interest in techniques for facilitating the selection of user interface widgets or other on-screen targets with a pointing device. We report research into using target expansion for facilitating selection. Widgets that expand or grow in response to the user's focus of attention allow for a reduced initial size which can help optimize screen space use and may be easier to select than targets that do not expand. However, selection performance could plausibly suffer from a decreased initial widget size. We describe an experiment in which users select a single, isolated target button that expands just before it is selected. Our results show that users benefit from target expansion even if the target only begins expanding after 90% of the distance to the target has been travelled. Furthermore, our results suggest that, for sufficiently large ID values, users are able to take approximately full advantage of the expanded target size. For interfaces with multiple expanding widgets, however, subtle problems arise due to the collisions or overlap that may occur between adjacent expanding widgets. We give a detailed examination of the issues involved in both untiled and tiled multiple expanding targets and present various design strategies for improving their performance.

Volumetric displays provide interesting opportunities and challenges for 3D interaction and visualization, particularly when used in a highly interactive manner. We explore this area through the design and implementation of techniques for interactive direct manipulation of objects with a 3D volumetric display. Motion tracking of the user's fingers provides for direct gestural interaction with the virtual objects, through manipulations on and around the display's hemispheric enclosure. Our techniques leverage the unique features of volumetric displays, including a 360° viewing volume that enables manipulation from any viewpoint around the display, as well as natural and accurate perception of true depth information in the displayed 3D scene. We demonstrate our techniques within a prototype 3D geometric model building application.

We introduce the concept of the Temporal Thumbnail, used to quickly convey information about the amount of time spent viewing specific areas of a virtual 3D model. Temporal Thumbnails allow for large amounts of time-based information collected from model viewing sessions to be rapidly visualized by collapsing the time dimension onto the space of the model, creating a characteristic impression of the overall interaction. We describe three techniques that implement the Temporal Thumbnail concept and present a study comparing these techniques to more traditional video and storyboard representations. The results suggest that Temporal Thumbnails have potential as an effective technique for quickly analyzing large amounts of viewing data. Practical and theoretical issues for visualization and representation are also discussed.

We investigate pointing in true 3D environments where the target size varies in three spatial dimensions. We also study the effect of the user's physical movement angle on pointing performance. Results show that target size dimension along the primary axis of movement has a greater impact on performance than the other two dimensions. Movement angle also significantly affects performance, and changes the relative impact of the three target dimensions. Building upon recent results in the modeling of bivariate pointing, we propose and validate a new model that describes pointing at trivariate targets. This model also accounts for movement angle, and outperforms previously published models.

Current user interface widgets typically assume that the input device can only provide x-y position and binary button press information. Other inputs such as the continuous pressure data provided by styluses on tablets are rarely used. We explore the design space of using the continuous pressure sensing capabilities of styluses to operate multi-state widgets. We present the results of a controlled experiment that investigates human ability to perform discrete target selection tasks by varying a stylus' pressure, with full or partial visual feedback. The experiment also considers different techniques for confirming selection once the target is acquired. Based on the experimental results, we discuss implications for the design of pressure sensitive widgets. A taxonomy of pressure widgets is presented, along with a set of initial concept sketches of various pressure widget designs.

We present an image guided pen-based suggestive interface for sketching 3D wireframe models. Rather than starting from a blank canvas, existing 2D images of similar objects serve as a guide to the user. Image based filters enable attraction, smoothing, and resampling of input curves, and allows for their selective application using pinning and gluing techniques. New input strokes also invoke suggestions of relevant geometry that can be used, reducing the need to explicitly draw all parts of the new model. All suggestions appear in-place with the model being built, in the user's focal attention space. A curve matching algorithm seamlessly augments basic suggestions with more complex ones from a database populated with previously used geometry. The interface also incorporates gestural command input, and interaction techniques for camera controls that enable smooth transitions between orthographic and perspective views.

The numeric keypads on mobile phones generally consist of 12 keys (0-9, *, #). Ambiguity arises when the 36-character alpha-numeric English alphabet is mapped onto this smaller number of keys. In this paper, we first present a taxonomy of the various techniques for resolving this ambiguity, dividing them into techniques that use consecutive actions to first select a character grouping and then a character from within that grouping, and those that use concurrent actions to achieve the same end. We then present the design and implementation of a chording approach to text entry that uses concurrent key presses. We conducted a controlled experiment that compared this chording technique to one-handed and two-handed versions of the commonly used MultiTap technique. The results show that the concurrent chording technique significantly outperforms both versions of the consecutive action MultiTap technique.

Volumetric displays provide interesting opportunities and challenges for 3D interaction and visualization, particularly when used in a highly interactive manner. We explore this area through the design and implementation of techniques for interactive direct manipulation of objects with a 3D volumetric display. Motion tracking of the user's fingers provides for direct gestural interaction with the virtual objects, through manipulations on and around the display's hemispheric enclosure. Our techniques leverage the unique features of volumetric displays, including a 360° viewing volume that enables manipulation from any viewpoint around the display, as well as natural and accurate perception of true depth information in the displayed 3D scene. We demonstrate our techniques within a prototype 3D geometric model building application.

We develop design principles and an interaction framework for sharable, interactive public ambient displays that support the transition from implicit to explicit interaction with both public and personal information. A prototype system implementation that embodies these design principles is described. We use novel display and interaction techniques such as simple hand gestures and touch screen input for explicit interaction and contextual body orientation and position cues for implicit interaction. Techniques are presented for subtle notification, self-revealing help, privacy controls, and shared use by multiple people each in their own context. Initial user feedback is also presented, and future directions discussed.

We present a variant of hierarchical marking menus where items are selected using a series of inflection-free simple marks, rather than the single "zig-zag" compound mark used in the traditional design. Theoretical analysis indicates that this simple mark approach has the potential to significantly increase the number of items in a marking menu that can be selected efficiently and accurately. A user experiment is presented that compares the simple and compound mark techniques. Results show that the simple mark technique allows for significantly more accurate and faster menu selections overall, but most importantly also in menus with a large number of items where performance of the compound mark technique is particularly poor. The simple mark technique also requires significantly less physical input space to perform the selections, making it particularly suitable for small footprint pen-based input devices. Visual design alternatives are also discussed.

A passive wand tracked in 3D using computer vision techniques is explored as a new input mechanism for interacting with large displays. We demonstrate a variety of interaction techniques and visual widgets that exploit the affordances of the wand, resulting in an effective interface for large scale display interaction. The lack of any buttons or other electronics on the wand presents a challenge that we address by developing a set of gestures and postures to track and enable command input.

Current interfaces for manipulating curves typically use a standard point cursor to indirectly adjust curve parameters. We present an interface for far more direct manipulation of curves using a specialized high degree-of-freedom curve input device, called ShapeTape. This device allows us to directly control the shape and position of a virtual curve widget. We describe the design and implementation of a variety of interaction techniques that use this curve widget to create and manipulate other virtual curves in 2D and 3D space. The input device is also used to sense a set of user gestures for invoking commands and tools. The result is an effective alternate user interface for curve manipulation that can be used in 2D and 3D graphics applications.

A passive wand tracked in 3D using computer vision techniques is explored as a new input mechanism for interacting with large displays. We demonstrate a variety of interaction techniques that exploit the affordances of the wand, resulting in an effective interface for large scale interaction. The lack of any buttons or other electronics on the wand presents a challenge that we address by developing a set of postures and gestures to track state and enable command input. We also describe the use of multiple wands, and posit designs for more complex wands in the future.

We explore a variety of interaction and visualization techniques for fluid navigation, segmentation, linking, and annotation of digital videos. These techniques are developed within a concept prototype called LEAN that is designed for use with pressure-sensitive digitizer tablets. These techniques include a transient position+velocity widget that allows users not only to move around a point of interest on a video, but also to rewind or fast forward at a controlled variable speed. We also present a new variation of fish-eye views called twist-lens, and incorporate this into a position control slider designed for the effective navigation and viewing of large sequences of video frames. We also explore a new style of widgets that exploit the use of the pen's pressure-sensing capability, increasing the input vocabulary available to the user. Finally, we elaborate on how annotations referring to objects that are temporal in nature, such as video, may be thought of as links, and fluidly constructed, visualized and navigated.

TiltText, a new technique for entering text into a mobile phone is described. The standard 12-button text entry keypad of a mobile phone forces ambiguity when the 26- letter Roman alphabet is mapped in the traditional manner onto keys 2-9. The TiltText technique uses the orientation of the phone to resolve this ambiguity, by tilting the phone in one of four directions to choose which character on a particular key to enter. We first discuss implementation strategies, and then present the results of a controlled experiment comparing TiltText to MultiTap, the most common text entry technique. The experiment included 10 participants who each entered a total of 640 phrases of text chosen from a standard corpus, over a period of about five hours. The results show that text entry speed including correction for errors using TiltText was 23% faster than MultiTap by the end of the experiment, despite a higher error rate for TiltText. TiltText is thus amongst the fastest known language-independent techniques for entering text into mobile phones.

Recent advances in sensing technology have enabled a new generation of tabletop displays that can sense multiple points of input from several users simultaneously. However, apart from a few demonstration techniques [17], current user interfaces do not take advantage of this increased input bandwidth. We present a variety of multifinger and whole hand gestural interaction techniques for these displays that leverage and extend the types of actions that people perform when interacting on real physical tabletops. Apart from gestural input techniques, we also explore interaction and visualization techniques for supporting shared spaces, awareness, and privacy. These techniques are demonstrated within a prototype room furniture layout application, called RoomPlanner.

Many traditional techniques for "looking inside" volumetric data involve removing portions of the data, for example using various cutting tools, to reveal the interior. This allows the user to see hidden parts of the data, but has the disadvantage of removing potentially important surrounding contextual information. We explore an alternate strategy for browsing that uses deformations, where the user can cut into and open up, spread apart, or peel away parts of the volume in real time, making the interior visible while still retaining surrounding context. We consider various deformation strategies and present a number of interaction techniques based on different metaphors. Our designs pay special attention to the semantic layers that might compose a volume (e.g. the skin, muscle, bone in a scan of a human). Users can apply deformations to only selected layers, or apply a given deformation to a different degree to each layer, making browsing more flexible and facilitating the visualization of relationships between layers. Our interaction techniques are controlled with direct, "in place" manipulation, using pop-up menus and 3D widgets, to avoid the divided attention and awkwardness that would come with panels of traditional widgets. Initial user feedback indicates that our techniques are valuable, especially for showing portions of the data spatially situated in context with surrounding data.

Previous systems have explored the challenges of designing an interface for automotive styling which combine the metaphor of 2D drawing using physical tape with the simultaneous creation and management of a corresponding virtual 3D model. These systems have been limited to only 2D planar curves while typically the principal characteristic curves of an automotive design are three dimensional and non-planar. We present a system which addresses this limitation. Our system allows a designer to construct these non-planar 3D curves by drawing a series of 2D curves using the 2D tape drawing technique and interaction style. These results are generally applicable to the interface design of 3D modeling applications and also to the design of arm's length interaction on large scale display systems

There exist several user interface widgets that dynamically grow in size in response to the user's focus of attention. Some of these, such as icons in toolbars, expand to facilitate their selection - allowing for a reduced initial size in an attempt to optimize screen space use. However, selection performance may be degraded by this decreased initial widget size. We describe an experiment which explores the effects of varying parameters of expansion techniques in a selection task. Our results suggest that Fitts' law can model and predict performance in such tasks. They also indicate that performance is governed by the target's final size, not its initial one. Further, performance is dependent on the target's final size even when the target only begins expanding as late as after 90% of the movement towards the target has already been completed. These results indicate that expanding widgets can be used without sacrificing performance

This paper describes StyleCam, an approach for authoring 3D viewing experiences that incorporate stylistic elements that are not available in typical 3D viewers. A key aspect of StyleCam is that it allows the author to significantly tailor what the user sees and when they see it. The resulting viewing experience can approach the visual richness and pacing of highly authored visual content such as television commercials or feature films. At the same time, StyleCam allows for a satisfying level of interactivity while avoiding the problems inherent in using unconstrained camera models. The main components of StyleCam are camera surfaces which spatially constrain the viewing camera; animation clips that allow for visually appealing transitions between different camera surfaces; and a simple, unified, interaction technique that permits the user to seamlessly and continuously move between spatial-control of the camera and temporal-control of the animated transitions. Further, the user's focus of attention is always kept on the content, and not on extraneous interface widgets. In addition to describing the conceptual model of StyleCam, its current implementation, and an example authored experience, we also present the results of an evaluation involving real users.

We present an alternate interface for 3D modeling for use on large scale displays. The interface integrates several concepts specifically selected and enhanced for large scale interaction. These include 2D construction planes spatially integrated in a 3D volume, enhanced orthographic views, smooth transitions between 2D and 3D views, tape drawing as the primary curve and line creation technique, visual viewpoint markers, and continuous two-handed interaction.

We present experimental work that explores the factors governing symmetric bimanual interaction in a two-handed task that requires the user to track a pair of targets, one target with each hand. A symmetric bimanual task is a two-handed task in which each hand is assigned an identical role. In this context, we explore three main experimental factors. We vary the distance between the pair of targets to track: as the targets become further apart, visual diversion increases, forcing the user to divide attention between the two targets. We also vary the demands of the task by using both a slow and a fast tracking speed. Finally, we explore visual integration of sub-tasks: in one condition, the two targets to track are connected by a line segment which visually links the targets, while in the other condition there is no connecting line. Our results indicate that all three experimental factors affect the degree of parallelism, which we quantify using a new metric of bimanual parallelism. However, differences in tracking error between the two hands are affected only by the visual integration factor.

We explore the use of the non-dominant hand to control a virtual camera while the dominant hand performs other tasks in a virtual 3D scene. Two experiments and an informal study are presented which evaluate this interaction style by comparing it to the status-quo unimanual interaction. In the first experiment, we find that for a target selection task, performance using the bimanual technique was 20% faster. Experiment 2 compared performance in a more complicated object docking task. Performance advantages are shown, however, only after practice. Free-form 3D painting was explored in the user study. In both experiments and in the user study participants strongly preferred the bimanual technique. The results also indicate that user preferences concerning bimanual interaction may be driven by factors other than simple time-motion performance advantages.

Rotating a piece of paper while drawing is an integral and almost subconscious part of drawing with pencil and paper. In a similar manner, the advent of lightweight pen-based computers allow digital artwork to be rotated while drawing by rotating the entire computer. Given this type of manipulation we explore the implications for the user interface to support artwork orientation. First we describe an exploratory study to further motivate our work and characterize how artwork is manipulated while drawing. After presenting some possible UI approaches to support artwork orientation, we define a new solution called a rotating user interface (RUIs). We then discuss design issues and requirements for RUIs based on our exploratory study.

We explore a new input device and a set of interaction techniques to facilitate direct manipulation of curves and surfaces. The input device, called ShapeTapeTM, is a continuous bend and twist sensitive strip that encourages manipulations that use both hands and, at times, all 10 fingers. We explore this input and interaction design space through a set of usage scenarios for creating and editing curves and surfaces as well as consider general interactions such as command access and camera controls. This investigation is carried out by extending Alias|wavefront’s modeling and animation package, Maya. CR

Tape drawing is the art of creating sketches on large scale upright surfaces using black photographic tape. Typically used in the automotive industry, it is an important part of the automotive design process that is currently not computerized. We analyze and describe the unique aspects of tape drawing, and use this knowledge to design and implement a digital tape drawing system. Our system retains the fundamental interaction and visual affordances of the traditional media while leveraging the power of the digital media. Aside from the practical aspect of our work, the interaction techniques developed have interesting implications for current theories of human bimanual interaction.

We present experimental work which explores how the match (or mismatch) between the input space of the hands and the output space of a graphical display influences two-handed input performance. During interaction with computers, a direct correspondence between the input and output spaces is often lacking. Not only are the hands disjoint from the display space, but the reference frames of the hands may in fact be disjoint from one another if two separate input devices (e.g. two mice) are used for two-handed input. In general, we refer to the workspace and origin within which the hands operate as kinesthetic reference frames. Our goal is to better understand how an interface designer's choice of kinesthetic reference frames influences a user's ability to coordinate two-handed movements, and to explore how the answer to this question may depend on the availability of visual feedback. Understanding this issue has implications for the design of two-handed interaction techniques and input devices, as well as for the reference principle of Guiard's Kinematic Chain model of human bimanual action. Our results suggest that the Guiard reference principle is robust with respect to variances in the kinesthetic reference frames as long as appropriate visual feedback is present.

Article The PadMouse: facilitating selection and spatial positioning for the non-dominant hand Share on Authors: Ravin Balakrishnan Dept. of Computer Science, University of Toronto, Toronto, Ontario, Canada M5S 3G4 and Aliaslwavefront, 210 King Street East, Toronto, Ontario, Canada M5A 1J7 Dept. of Computer Science, University of Toronto, Toronto, Ontario, Canada M5S 3G4 and Aliaslwavefront, 210 King Street East, Toronto, Ontario, Canada M5A 1J7View Profile , Pranay Patel Aliaslwavefront, 210 King Street East, Toronto, Ontario, Canada M5A 1J7 Aliaslwavefront, 210 King Street East, Toronto, Ontario, Canada M5A 1J7View Profile Authors Info & Claims CHI '98: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsJanuary 1998 Pages 9–16https://doi.org/10.1145/274644.274646Online:01 January 1998Publication History 29citation817DownloadsMetricsTotal Citations29Total Downloads817Last 12 Months20Last 6 weeks1 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

Article Free Access Share on The Rockin'Mouse: integral 3D manipulation on a plane Authors: Ravin Balakrishnan Dept. of Computer Science, University of Toronto, Toronto, Ontario, Canada M5S 1A1 and Alias/Wavefront, 110 Richmond Street East, Toronto, Ontario, Canada M5C 1P1 Dept. of Computer Science, University of Toronto, Toronto, Ontario, Canada M5S 1A1 and Alias/Wavefront, 110 Richmond Street East, Toronto, Ontario, Canada M5C 1P1View Profile , Thomas Baudel Alias/Wavefront, 110 Richmond Street East, Toronto, Ontario, Canada M5C 1P1 Alias/Wavefront, 110 Richmond Street East, Toronto, Ontario, Canada M5C 1P1View Profile , Gordon Kurtenbach Alias/Wavefront, 110 Richmond Street East, Toronto, Ontario, Canada M5C 1P1 Alias/Wavefront, 110 Richmond Street East, Toronto, Ontario, Canada M5C 1P1View Profile , George Fitzmaurice Alias/Wavefront, 110 Richmond Street East, Toronto, Ontario, Canada M5C 1P1 Alias/Wavefront, 110 Richmond Street East, Toronto, Ontario, Canada M5C 1P1View Profile Authors Info & Claims CHI '97: Proceedings of the ACM SIGCHI Conference on Human factors in computing systemsMarch 1997 Pages 311–318https://doi.org/10.1145/258549.258778Published:27 March 1997Publication History 84citation1,262DownloadsMetricsTotal Citations84Total Downloads1,262Last 12 Months49Last 6 weeks5 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my Alerts New Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

Recent work in computer input control has sought to maximize the use of the fingers in the operation of computer pointing devices.The main rationale is the hypothesis that the muscle groups conmolling the fingers have a higher bandwidth than those controlling other segments of the human upper limb.Evidence which supports this, however, is inconclusive.We conducted an experiment to determine the relative bandwidths of the fingers, wrist, and forearm and found that the fingers do not necessarily outperform the other limb segments.Our results indicate that the bandwidth of the unsupported index finger is approximately 3.0 bits/s while the wrist and forearm have bandwidths of about 4.1 bits/s.We also show that the thumb and index finger working together in a pinch grip have an information processing rate of about 4.5 bits/s.Other factors which influence the relative performance of the different limbs in manipulation tasks are considered.

This article reports the results from three experimental studies of reaching behavior in a head-coupled stereo display system with a hand-tracking subsystem for object selection. It is found that lag in the head-tracking system is relatively unimportant in predicting performance, whereas lag in the hand-tracking system is critical. The effect of hand lag can be modeled by means of a variation on Fitts' Law with the measured system lag introduced as a multiplicative variable to the Fitts' Law index of difficulty. This means that relatively small lags can cause considerable degradation in performance if the targets are small. Another finding is that errors are higher for movement in and out of the screen, as compared to movements in the plane of the screen, and there is a small (10%) time penalty for movement in the Z direction in all three experiments. Low frame rates cause a degradation in performance; however, this can be attributed to the lag which is caused by low frame rates, particularly if double buffering is used combined with early sampling of the hand-tracking device.