The needs and capabilities of a person with severe disabilities are often so specific that designing for the person is like designing for a "universe of one." This project addresses this problem for women with Rett syndrome, a disorder accompanied by severe cognitive, communication, and motor impairment. The research team adapted participatory design techniques to work with five such women, and their families, to design and evaluate new assistive technology for these women. The process suggests a class of media-playing devices that would be generally useful to women with Rett syndrome: systems that can load multiple audio or video segments; be activated by many different switches; and respond instantly to switch-hits. As well, the systems should permit a caregiver to set the start and end time of each segment, and how the system advances through a sequence of segments. The paper also discusses patterns that were observed when collaborating with the families. For example, parents shared longstanding but untried ideas for new assistive technology; and expressed a strong interest in any device that would help their daughters do things for themselves.

Being able to predict the performance of interface designs using models of human cognition and performance is a long-standing goal of HCI research. This paper presents recent advances in cognitive modeling which permit increasingly realistic and accurate predictions for visual human-computer interaction tasks such as icon search by incorporating an "active vision" approach which emphasizes eye movements to visual features based on the availability of features in relationship to the point of gaze. A high fidelity model of a classic visual search task demonstrates the value of incorporating visual acuity functions into models of visual performance. The features captured by the high-fidelity model are then used to formulate a model simple enough for practical use, which is then implemented in an easy-to-use GLEAN modeling tool. Easy-to-use predictive models for complex visual search are thus feasible and should be further developed.

Human multitasking often involves complex task interactions and subtle tradeoffs which might be best understood through detailed computational cognitive modeling, yet traditional cognitive modeling approaches may not explore a sufficient range of task strategies to reveal the true complexity of multitasking behavior. This study proposes a systematic approach for exploring a large number of strategies using a computer-cluster-based parallelized modeling system. The paper demonstrates the efficacy of the approach for investigating and revealing the effects of different microstrategies on human performance, both within and across individuals, for a time-pressured multimodal dual task. The modeling results suggest that multitasking performance is not simply a matter of interleaving cognitive and sensorimotor processing but is instead heavily influenced by the selection of subtask microstrategies.

The gaze locations reported by eye trackers often contain error resulting from a variety of sources. Such error is of increasing concern to eye tracking researchers, and several techniques have been introduced to clean up the error. These methods, however, either compensate only for error caused by a particular source (such as pupil dilation) or require the error to be somewhat constant across space and time. This paper introduces a method that is applicable to error generated from a variety of sources and that is resilient to the change in error across the display. A study shows that, at least in some cases, although the change in error across the display appears to be random it in fact follows a consistent pattern which can be modeled using quadratic equations. The parameters of these equations can be estimated using linear regression on the error vectors between recorded fixations and possible target locations. The resulting equations can then be used to clean up the error. This regression-based approach is much easier to apply than some of the previously published methods. The method is applied to the data of a visual search experiment, and the results show that the regression-based error correction works very well.

Human visual search plays an important role in many human–computer interaction (HCI) tasks. Better models of visual search are needed not just to predict overall performance outcomes, such as whether people will be able to find the information needed to complete an HCI task, but to understand the many human processes that interact in visual search, which will in turn inform the detailed design of better user interfaces. This article describes a detailed instantiation, in the form of a computational cognitive model, of a comprehensive theory of human visual processing known as “active vision” (Findlay & Gilchrist, 2003). The computational model is built using the Executive Process-Interactive Control cognitive architecture. Eye-tracking data from three experiments inform the development and validation of the model. The modeling asks—and at least partially answers—the four questions of active vision: (a) What can be perceived in a fixation? (b) When do the eyes move? (c) Where do the eyes move? (d) What information is integrated between eye movements? Answers include: (a) Items nearer the point of gaze are more likely to be perceived, and the visual features of objects are sometimes misidentified. (b) The eyes move after the fixated visual stimulus has been processed (i.e., has entered working memory). (c) The eyes tend to go to nearby objects. (d) Only the coarse spatial information of what has been fixated is likely maintained between fixations. The model developed to answer these questions has both scientific and practical value in that the model gives HCI researchers and practitioners a better understanding of how people visually interact with computers, and provides a theoretical foundation for predictive analysis tools that can predict aspects of that interaction.

Human-computer systems intended for time-critical multitasking need to be designed with an understanding of how humans can coordinate and interleave perceptual, memory, and motor processes. This paper presents human performance data for a highly-practiced time-critical dual task. In the first of the two interleaved tasks, participants tracked a target with a joystick. In the second, participants keyed-in responses to objects moving across a radar display. Task manipulations include the peripheral visibility of the secondary display (visible or not) and the presence or absence of auditory cues to assist with the radar task. Eye movement analyses reveal extensive coordination and overlapping of human information processes and the extent to which task manipulations helped or hindered dual task performance. For example, auditory cues helped only a little when the secondary display was peripherally visible, but they helped a lot when it was not peripherally visible.

Children with severe motor impairments such as with disabilities resulting from severe cerebral palsy benefit greatly from assistive technology, but very little guidance is available on how to collaborate with this population as partners in the design of such technology. To explore how to facilitate such collaborations, a field-based participant observation study, as well as structured and unstructured interviews, were conducted at a home for children with severe disabilities. Team-building collaborative design activities were pursued. Guidelines are proposed for how to collaborate with children with severe motor impairments.

This paper discusses attempts that have been made to team with children with severe motor impairments in the design of technology to help those children express themselves. The project is still new, and the endeavor extremely challenging, but small successes as well as enormous challenges can be reported and discussed. Much can be learned from the literature and practice of alternative and augmentative communication, in which children are asked to assist in the design and implementation of a communication scheme for that child. The challenge is to integrate these approaches with what has been learned when collaborating with typically-developing children in the design of new technology.

Visual search is an important part of human-computer interaction. It is critical that we build theory about how people visually search displays in order to better support the users' visual capabilities and limitations in everyday tasks. One way of building such theory is through computational cognitive modeling. The ultimate promise for cognitive modeling in HCI it to provide the science base needed for predictive interface analysis tools. This paper discusses computational cognitive modeling of the perceptual, strategic, and oculomotor processes people used in a visual search task. This work refines and rounds out previously reported cognitive modeling and eye tracking analysis. A revised "minimal model" of visual search is presented that explains a variety of eye movement data better than the original model. The revised model uses a parsimonious strategy that is not tied to a particular visual structure or feature beyond the location of objects. Three characteristics of the minimal strategy are discussed in detail.

EyeDraw is a software program that, when run on a computer with an eye tracking device, enables children with severe motor disabilities to draw pictures by just moving their eyes. This paper discusses the motivation for building the software, how the program works, the iterative development of two versions of the software, user testing of the two versions by people with and without disabilities, and modifications to the software based on user testing. Feedback from both children and adults with disabilities, and from their caregivers, was especially helpful in the design process. The project identifies challenges that are unique to controlling a computer with the eyes, and unique to writing software for children with severe motor impairments.

A predictive tool to simulate human visual search behavior would help interface designers inform and validate their design. Such a tool would benefit from a semantic component that would help predict search behavior even in the absence of exact textual matches between goal and target. This paper discusses a comparison of three semantic systems-LSA, WordNet and PMI-IR-to evaluate their performance in predicting the link that people would select given an information goal and a webpage. PMI-IR best predicted human performance as observed in a user study.

The seeming contradiction between “banner blindness” and Web users' complaints about distracting advertisements motivates a pair of experiments into the effect of banner ads on visual search. Experiment 1 measures perceived cognitive workload and search times for short words with two banners on the screen. Four kinds of banners were examined: (1) animated commercial, (2) static commercial, (3) cyan with flashing text, and (4) blank. Using NASA's Task Load Index, participants report increased workload under flashing text banners. Experiment 2 investigates search through news headlines at two levels of difficulty: exact matches and matches requiring semantic interpretation. Results show both animated and static commercial banners decrease visual search speeds. Eye tracking data reveal people rarely look directly at banners. A post hoc memory test confirms low banner recall and, surprisingly, that animated banners are more difficult to remember than static look-alikes. Results have implications for cognitive modeling and Web design.

This paper describes the design and development of EyeDraw, a software program that will enable children with severe mobility impairments to use an eye tracker to draw pictures with their eyes so that they can have the same creative developmental experiences as nondisabled children. EyeDraw incorporates computer-control and software application advances that address the special needs of people with motor impairments, with emphasis on the needs of children. The contributions of the project include (a) a new technique for using the eyes to control the computer when accomplishing a spatial task, (b) the crafting of task-relevant functionality to support this new technique in its application to drawing pictures, and (c) a user-tested implementation of the idea within a working computer program. User testing with nondisabled users suggests that we have designed and built an eye-cursor and eye drawing control system that can be used by almost anyone with normal control of their eyes. The core technique will be generally useful for a range of computer control tasks such as selecting a group of icons on the desktop by drawing a box around them.

This article investigates the cognitive strategies that people use to search computer displays. Several different visual layouts are examined: unlabeled layouts that contain multiple groups of items but no group headings, labeled layouts in which items are grouped and each group has a useful heading, and a target-only layout that contains just one item. A number of plausible strategies were proposed for each layout. Each strategy was programmed into the EPIC cognitive architecture, producing models that simulate the human visual-perceptual, oculomotor, and cognitive processing required for the task. The models generate search time predictions. For unlabeled layouts, the mean layout search times are predicted by a purely random search strategy, and the more detailed positional search times are predicted by a noisy systematic strategy. The labeled layout search times are predicted by a hierarchical strategy in which first the group labels are systematically searched, and then the contents of the target group. The target-only layout search times are predicted by a strategy in which the eyes move directly to the sudden appearance of the target. The models demonstrate that human visual search performance can be explained largely in terms of the cognitive strategy that is used to coordinate the relevant perceptual and motor processes, a clear and useful visual hierarchy triggers a fundamentally different visual search strategy and effectively gives the user greater control over the visual navigation, and cognitive strategies will be an important component of a predictive visual search tool. The models provide insights pertaining to the visual-perceptual and oculomotor processes involved in visual search and contribute to the science base needed for predictive interface analysis.

This research investigates the cognitive strategies and eye movements that people use to search for a known item in a hierarchical computer display. Computational cognitive models were built to simulate the visual-perceptual and oculomotor processing required to search hierarchical and nonhierarchical displays. Eye movement data were collected and compared on over a dozen measures with the a priori predictions of the models. Though it is well accepted that hierarchical layouts are easier to search than nonhierarchical layouts, the underlying cognitive basis for this design heuristic has not yet been established. This work combines cognitive modeling and eye tracking to explain this and numerous other visual design guidelines. This research also demonstrates the power of cognitive modeling for predicting, explaining, and interpreting eye movement data, and how to use eye tracking data to confirm and disconfirm modeling details. Categories and subject descriptors H.5.2 [Information Interfaces and Presentation]: User Interfaces-- Evaluation/methodology, eye tracking,

An experiment investigates (1) how the physical structure of a computer screen layout affects visual search and (2) how people select a found target object with a mouse. Two structures are examined---labeled visual hierarchies (groups of objects with one label per group) and unlabeled visual hierarchies (groups without labels). Search and selection times were separated by imposing a point-completion deadline that discouraged participants from moving the mouse until they found the target. The observed search times indicate that labeled visual hierarchies can be searched much more efficiently than unlabeled visual hierarchies, and suggest that people use a fundamentally different strategy for each of the two structures. The results have implications for screen layout design and cognitive modeling of visual search. The observed mouse-pointing times suggest that people use a slower and more accurate speed-accuracy operating characteristic to select a target with a mouse when visual distractors are present, which suggests that Fitts' law coefficients derived from standard mouse-pointing experiments may under-predict mouse-pointing times for typical human-computer interactions. The observed mouse-pointing times also demonstrate that mouse movement times for a two-dimensional pointing task can be most-accurately predicted by setting the w in Fitts' law to the width of the target along the line of approach.

This research presents cognitive models of a person selecting an item from a familiar, ordered, pull-down menu. Two different models provide a good fit with human data and thus two different possible explanations for the lowlevel cognitive processes involved in the task. Both models assert that people make an initial eye and hand movement to an anticipated target location without waiting for the menu to appear. The first model asserts that a person knows the exact location of the target item before the menu appears, but the model uses nonstandard Fitts' law coefficients to predict mouse pointing time. The second model asserts that a person would only know the approximate location of the target item, and the model uses Fitts' law coefficients better supported by the literature. This research demonstrates that people can develop considerable knowledge of locations in a visual task environment, and that more work regarding Fitts' law is needed. KEYWORDS Cognitive models, Fitts' law, men...

Article Free Access Share on Cognitive modeling reveals menu search in both random and systematic Authors: Anthony J. Hornof Artificial Intelligence Laboratory, Electrical Engineering & Computer Science Department, University of Michigan, 1101 Beal Avenue, Ann Arbor, MI Artificial Intelligence Laboratory, Electrical Engineering & Computer Science Department, University of Michigan, 1101 Beal Avenue, Ann Arbor, MIView Profile , David E. Kieras Artificial Intelligence Laboratory, Electrical Engineering & Computer Science Department, University of Michigan, 1101 Beal Avenue, Ann Arbor, MI Artificial Intelligence Laboratory, Electrical Engineering & Computer Science Department, University of Michigan, 1101 Beal Avenue, Ann Arbor, MIView Profile Authors Info & Claims CHI '97: Proceedings of the ACM SIGCHI Conference on Human factors in computing systemsMarch 1997 Pages 107–114https://doi.org/10.1145/258549.258621Published:27 March 1997Publication History 50citation946DownloadsMetricsTotal Citations50Total Downloads946Last 12 Months35Last 6 weeks2 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my Alerts New Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

Engineering models of human performance permit some aspects of usability of interface designs to be predicted from an analysis of the task, and thus can replace to some extent expensive user testing data.The best developed such tools are GOMS models, which have been shown to be accurate and effective in predicting usability of the procedural aspects of interface designs.This paper describes a computer-based tool, GLEAN, that generates quantitative predictions from a supplied GOMS model and a set of benchmark tasks.GLEAN is demonstrated to reproduce the results of a case study of GOMS model application with considerable time savings over both manual modeling as well as empirical testing.