Histopathology laboratories devote considerable time and effort to sorting tissue sample slides. We observed slide sorting in a typical urban hospital to understand the existing workflow and explore how it might be supported by an interactive computer support system. We observed 8.5 hours of slide sorting activity through a video camera mounted above a laboratory workbench. Through detailed video analysis, we characterised the process, examined which activities took the most time, and explored design considerations. We found that a very large proportion (23.5%) of the slide sorting time involved managing paper documents. We suggest that an interactive computer support system could automatically detect which slides are sorted into which folders and digitally list additional slides to include with these sets; this would support the workflow of technicians, while eliminating paper management and manual barcode reading operations, leading to time savings of approximately 30%. Additional recommendations for the design of such a support system include focusing on case management (e.g. how many slides belong to each case, whether a complete case will fit within the current folder, and which slides associated with a case are still missing), supporting recovery from disruptions, and enabling a flexible rather than a highly scripted workflow.

The study of surgeons' eye movements is an innovative way of assessing skill and situation awareness, in that a comparison of eye movement strategies between expert surgeons and novices may show differences that can be used in training.

We introduce eSeeTrack, an eye-tracking visualization prototype that facilitates exploration and comparison of sequential gaze orderings in a static or a dynamic scene. It extends current eye-tracking data visualizations by extracting patterns of sequential gaze orderings, displaying these patterns in a way that does not depend on the number of fixations on a scene, and enabling users to compare patterns from two or more sets of eye-gaze data. Extracting such patterns was very difficult with previous visualization techniques. eSeeTrack combines a timeline and a tree-structured visual representation to embody three aspects of eye-tracking data that users are interested in: duration, frequency and orderings of fixations. We demonstrate the usefulness of eSeeTrack via two case studies on surgical simulation and retail store chain data. We found that eSeeTrack allows ordering of fixations to be rapidly queried, explored and compared. Furthermore, our tool provides an effective and efficient mechanism to determine pattern outliers. This approach can be effective for behavior analysis in a variety of domains that are described at the end of this paper.

Abstract Visual fixation on one's tool(s) takes much attention away from one's primary task. Following the belief that the best tools ‘disappear’ and become invisible to the user, we present a study comparing visual fixations (eye gaze within locations on a graphical display) and performance for mouse, pen, and physical slider user interfaces. Participants conducted a controlled, yet representative, color matching task that required user interaction representative of many data exploration tasks such as parameter exploration of medical or fuel cell data. We demonstrate that users may spend up to 95% fewer visual fixations on physical sliders versus standard mouse and pen tools without any loss in performance for a generalized visual performance task.

Spatialization displays use a geographic metaphor to arrange non-spatial data. For example, spatializations are commonly applied to document collections so that document themes appear as geographic features such as hills. Many common spatialization interfaces use a 3-D landscape metaphor to present data. However, it is not clear whether 3-D spatializations afford improved speed and accuracy for user tasks compared to similar 2-D spatializations. We describe a user study comparing users' ability to remember dot displays, 2-D landscapes, and 3-D landscapes for two different data densities (500 vs. 1000 points). Participants' visual memory was statistically more accurate when viewing dot displays and 3-D landscapes compared to 2-D landscapes. Furthermore, accuracy remembering a spatialization was significantly better overall for denser spatializations. These results are of benefit to visualization designers who are contemplating the best ways to present data using spatialization techniques.

As emphasis is placed on developing mobile, educational, and other applications that minimize cognitive load on users, it is becoming more essential to explore interfaces based on implicit engagement techniques so users can remain focused on their tasks. In this research, data were collected with 12 pairs of students who solved complex math problems using a tutorial system that they engaged over 100 times per session entirely implicitly via speech amplitude or pen pressure cues. Results revealed that users spontaneously, reliably, and substantially adapted these forms of communicative energy to designate and repair an intended interlocutor in a computer-mediated group setting. Furthermore, this behavior was harnessed to achieve system engagement accuracies of 75-86%, with accuracies highest using speech amplitude. However, students had limited awareness of their own adaptations. Finally, while continually using these implicit engagement techniques, students maintained their performance level at solving complex mathematics problems throughout a one-hour session.

The present paper reports on the design and performance of a novel dual-Wizard simulation infrastructure that has been used effectively to prototype next-generation adaptive and implicit multimodal interfaces for collaborative groupwork. This high-fidelity simulation infrastructure builds on past development of single-wizard simulation tools for multiparty multimodal interactions involving speech, pen, and visual input [1]. In the new infrastructure, a dual-wizard simulation environment was developed that supports (1) real-time tracking, analysis, and system adaptivity to a user's speech and pen paralinguistic signal features (e.g., speech amplitude, pen pressure), as well as the semantic content of their input. This simulation also supports (2) transparent user training to adapt their speech and pen signal features in a manner that enhances the reliability of system functioning, i.e., the design of mutually-adaptive interfaces. To accomplish these objectives, this new environment also is capable of handling (3) dynamic streaming digital pen input. We illustrate the performance of the simulation infrastructure during longitudinal empirical research in which a user-adaptive interface was designed for implicit system engagement based exclusively on users' speech amplitude and pen pressure [2]. While using this dual-wizard simulation method, the wizards responded successfully to over 3,000 user inputs with 95-98% accuracy and a joint wizard response time of less than 1.0 second during speech interactions and 1.65 seconds during pen interactions. Furthermore, the interactions they handled involved naturalistic multiparty meeting data in which high school students were engaged in peer tutoring, and all participants believed they were interacting with a fully functional system. This type of simulation capability enables a new level of flexibility and sophistication in multimodal interface design, including the development of implicit multimodal interfaces that place minimal cognitive load on users during mobile, educational, and other applications.

Physical controls such as knobs, sliders, and buttons are experiencing a revival as many computing systems progress from personal computing architectures towards ubiquitous computing architectures. We demonstrate a process for measuring and comparing visceral emotional responses of a physical control to performance results of a target acquisition task. In our user study, participants experienced mechanical and rendered friction, inertia, and detent dynamics as they turned a haptic knob towards graphical targets of two different widths and amplitudes. Together, this process and user study provide novel affect- and performance-based design guidance to developers of physical controls for emerging ubiquitous computing environments. Our work bridges extensive human factors work in mechanical systems that peaked in the 1960's, to contemporary trends, with a goal of integrating mechatronic controls into emerging ubiquitous computing systems.

Computer-aided design (CAD) and 3D visualization techniques are at the heart of many engineering processes such as aircraft, ship, and automobile design. These visualization tasks require users to navigate or wayfind through complex 3D geometric models consisting of millions of parts. Despite numerous studies, it remains unclear whether large-screen displays improve user performance for such activities. We present a user study comparing standard desktop, immersive room (i.e., CAVE), and wall displays with 3D stereo/head-tracking, and mono/no head-tracking. We observed individual differences between users and found that the presence of contextual structure greatly impacted performance, suggesting that providing structure and developing interaction techniques accommodating a wide range of users yields better performance than focusing on display characteristics alone.

Kinesthetic feedback is a key mechanism by which people perceive object properties during their daily tasks - particularly inertial properties. For example, transporting a glass of water without spilling, or dynamically positioning a handheld tool such as a hammer, both require inertial kinesthetic feedback. We describe a prototype for a novel ungrounded haptic feedback device, the TorqueBAR, that exploits a kinesthetic awareness of dynamic inertia to simulate complex coupled motion as both a display and input device. As a user tilts the TorqueBAR to sense and control computer programmed stimuli, the TorqueBAR's centre-of-mass changes in real-time according to the user's actions. We evaluate the TorqueBAR using both quantitative and qualitative techniques, and we describe possible applications for the device such as video games and real-time robot navigation.

Computing devices within current work and play environments are relatively static. As the number of 'networked' devices grows, and as people and their devices become more dynamic, situations will commonly arise where users will wish to use 'that device there' instead of navigating through traditional user interface widgets such as lists. This paper describes a process for identifying devices through a pointing gesture using custom tags and a custom stylus called the gesturePen. Implementation details for this system are provided along with qualitative and quantitative results from a formal user study. As ubiquitous computing environments become more pervasive, people will rapidly switch their focus between many computing devices. The results of our work demonstrate that our gesturePen method can improve the user experience in ubiquitous environments by facilitating significantly faster interactions between computing devices.

We describe a simple technique for accurately calibrating the temporal lag in augmented and virtual environments within the Enhanced Virtual Hand Lab (EVHL), a collection of hardware and software to support research on goal-directed human hand motion. Lag is the sum of various delays in the data pipeline associated with sensing, processing, and displaying information from the physical world to produce an augmented or virtual world. Our main calibration technique uses a modified phonograph turntable to provide easily tracked periodic motion, reminiscent of the pendulum-based calibration technique of Liang, Shaw and Green. Measurements show a three-frame (50 ms) lag for the EVHL. A second technique, which uses a specialized analog sensor that is part of the EVHL, provides a "closed loop" calibration capable of sub-frame accuracy. Knowing the lag to sub-frame accuracy enables a predictive tracking scheme to compensate for the end-toend lag in the data pipeline. We describe both techniques and the EVHL environment in which they are used.