Data scientists are responsible for the analysis decisions they make, but it is hard for them to track the process by which they achieved a result. Even when data scientists keep logs, it is onerous to make sense of the resulting large number of history records full of overlapping variants of code, output, plots, etc. We developed algorithmic and visualization techniques for notebook code environments to help data scientists forage for information in their history. To test these interventions, we conducted a think-aloud evaluation with 15 data scientists, where participants were asked to find specific information from the history of another person's data science project. The participants succeed on a median of 80% of the tasks they performed. The quantitative results suggest promising aspects of our design, while qualitative results motivated a number of design improvements. The resulting system, called Verdant, is released as an open-source extension for JupyterLab.

Literate programming tools are used by millions of programmers today, and are intended to facilitate presenting data analyses in the form of a narrative. We interviewed 21 data scientists to study coding behaviors in a literate programming environment and how data scientists kept track of variants they explored. For participants who tried to keep a detailed history of their experimentation, both informal and formal versioning attempts led to problems, such as reduced notebook readability. During iteration, participants actively curated their notebooks into narratives, although primarily through cell structure rather than markdown explanations. Next, we surveyed 45 data scientists and asked them to envision how they might use their past history in an future version control system. Based on these results, we give design guidance for future literate programming tools, such as providing history search based on how programmers recall their explorations, through contextual details including images and parameters.

As software systems evolve, new interface features such as keyboard shortcuts and toolbars are introduced. While it is common to regression test the new features for functional correctness, there has been less focus on systematic regression testing for usability, due to the effort and time involved in human studies. Cognitive modeling tools such as CogTool provide some help by computing predictions of user performance, but they still require manual effort to describe the user interface and tasks, limiting regression testing efforts. In recent work, we developed CogTool-Helper to reduce the effort required to generate human performance models of existing systems. We build on this work by providing task specific test case generation and present our vision for human performance regression testing (HPRT) that generates large numbers of test cases and evaluates a range of human performance predictions for the same task. We examine the feasibility of HPRT on four tasks in LibreOffice, find several regressions, and then discuss how a project team could use this information. We also illustrate that we can increase efficiency with sampling by leveraging an inference algorithm. Samples that take approximately 50% of the runtime lose at most 10% of the performance predictions.

Pinching and spreading gestures are prevalent in mobile applications today, but these gestures have not yet been studied extensively. We conducted an exploratory study of pinch and spread gestures with seated participants on a phone and a tablet device. We found device orientation did not have a significant effect on gesture performance, most pinch and spread tasks were completed in a single action, and they were executed in 0.9-1.2 seconds. We also report how participants chose to sit with the mobile device, variations in gesture execution method, and the effect of varying target width and gesture size. Our task execution times for different gesture distances and precision levels display a surprisingly good fit to a simple Fitts's Law model. We conclude with recommendations for future studies.

In this study, we explore the role of age and fluid intelligence on the behavior of people looking for information in a real-world search space. Analyses of mouse moves, clicks, and eye movements provide a window into possible differences in both task strategy and performance, and allow us to begin to separate the influence of age from the correlated but isolable influence of cognitive ability. We found little evidence of differences in strategy between younger and older participants matched on fluid intelligence. Both performance and strategy differences were found between older participants having higher versus lower fluid intelligence, however, suggesting that cognitive factors, rather than age per se, exert the dominant influence. This underscores the importance of measuring and controlling for cognitive abilities in studies involving older adults.

Information Foraging Theory (IFT) has established itself as an important theory to explain how people seek information, but most work has focused more on the theory itself than on how best to apply it. In this paper, we investigate how to apply a reactive variant of IFT (Reactive IFT) to design IFT-based tools, with a special focus on such tools for ill-structured problems. Toward this end, we designed and implemented a variety of recommender algorithms to empirically investigate how to help people with the ill-structured problem of finding where to look for information while debugging source code. We varied the algorithms based on scent type supported (words alone vs. words + code structure), and based on use of foraging momentum to estimate rapidity of foragers' goal changes. Our empirical results showed that (1) using both words and code structure significantly improved the ability of the algorithms to recommend where software developers should look for information; (2) participants used recommendations to discover new places in the code and also as shortcuts to navigate to known places; and (3) low-momentum recommendations were significantly more useful than high-momentum recommendations, suggesting rapid and numerous goal changes in this type of setting. Overall, our contributions include two new recommendation algorithms, empirical evidence about when and why participants found IFT-based recommendations useful, and implications for the design of tools based on Reactive IFT.

With the rise of tools for predictive human performance modeling in HCI comes a need to model legacy applications. Models of legacy systems are used to compare products to competitors, or new proposed design ideas to the existing version of an application. We present CogTool-Helper, an exemplar of a tool that results from joining this HCI need to research in automatic GUI testing from the Software Engineering testing community. CogTool-Helper uses automatic UI-model extraction and test case generation to automatically create CogTool storyboards and models and infer methods to accomplish tasks beyond what the UI designer has specified. A design walkthrough with experienced CogTool users reveal that CogTool-Helper resonates with a "pain point" of real-world modeling and provide suggestions for future work.

CogTool-Explorer 1.2 (CTE1.2) predicts novice exploration behavior and how it varies with different user-interface (UI) layouts. CTE1.2 improves upon previous models of information foraging by adding a model of hierarchical visual search to guide foraging behavior. Built within CogTool so it is easy to represent UI layouts, run the model, and present results, CTE1.2's vision is to assess many design ideas at the storyboard stage before implementation and without the cost of running human participants. This paper evaluates CTE1.2 predictions against observed human behavior on 108 tasks (36 tasks on 3 distinct website layouts). CTE1.2's predictions accounted for 63-82% of the variance in the percentage of participants succeeding on each task, the number of clicks to success, and the percentage of participants succeeding without error. We demonstrate how these predictions can be used to identify areas of the UI in need of redesign.

Predictive human performance modeling has traditionally been used to make quantitative comparisons between alternative designs (e.g., task execution time for skilled users) instead of identifying UI problems or making design recommendations. This note investigates how reliably novice modelers can extract design recommendations from their models. Many HCI evaluation methods have been plagued by the "evaluator effect" [3], i.e., different people using the same method find different UI problems. Our data and analyses show that predictive human performance modeling is no exception. Novice modelers using CogTool [5] display a 34% Any-Two Agreement in their design recommendations, a result in the upper quartile of evaluator effect studies. However, because these recommendations are grounded in models, they may have more reliable impact on measurable performance than recommendations arising from less formal methods.

We studied the extent to which time-on-task is correlated with perception of usability for people who are familiar with a phone model and for those who are not. Our controlled experiment, conducted in Japan, correlated subjective usability assessments with time-on-task for expert and novice users on three different mobile phone models. We found that the correlation between perceived usability and time-on-task is stronger when participants are more familiar with the phone model. While not significant when initially inspecting a new phone model, a negative correlation between time-on-task and perceived usability becomes significant with as little as an hour's time doing tasks on the unfamiliar phone. This suggests that designing the UI to make time-on-task as short as possible may not have much effect on the purchase decision, but as experience increases, it may increase the loyalty of existing users.

Usability is recognized as a critically important quality of software, as evidenced by its inclusion in SE 2004: Curriculum Guidelines for Undergraduate Degree Programs in Software Engineering. However, many techniques for designing useful, usable and desirable software depend on knowledge and skills in the behavioral sciences, making them difficult for SE students to learn and for SE educators to teach with confidence. This tutorial will introduce SE educators to a quantitative usability assessment method that has had over 100 papers validating its predictions in the Human-Computer Interaction literature and a tool embodying that method suitable for inclusion in SE courses.

Usability concerns are often difficult to integrate into real-world software development processes. To remedy this situation, IBM research and development, partnering with Carnegie Mellon University, has begun to employ a repeatable and quantifiable usability analysis method, embodied in CogTool, in its development practice. CogTool analyzes tasks performed on an interactive system from a storyboard and a demonstration of tasks on that storyboard, and predicts the time a skilled user will take to perform those tasks. We discuss how IBM designers and UX professionals used CogTool in their existing practice for contract compliance, communication within a product team and between a product team and its customer, assigning appropriate personnel to fix customer complaints, and quantitatively assessing design ideas before a line of code is written. We then reflect on the lessons learned by both the development organizations and the researchers attempting this technology transfer from academic research to integration into real-world practice, and we point to future research to even better serve the needs of practice.

Software developers frequently need to perform code maintenance tasks, but doing so requires time-consuming navigation through code. A variety of tools are aimed at easing this navigation by using models to identify places in the code that a developer might want to visit, and then providing shortcuts so that the developer can quickly navigate to those locations. To date, however, only a few of these models have been compared head-to-head to assess their predictive accuracy. In particular, we do not know which models are most accurate overall, which are accurate only in certain circumstances, and whether combining models could enhance accuracy. Therefore, we have conducted an empirical study to evaluate the accuracy of a broad range of models for predicting many different kinds of code navigations in sample maintenance tasks. Overall, we found that models tended to perform best if they took into account how recently a developer has viewed pieces of the code, and if models took into account the spatial proximity of methods within the code. We also found that the accuracy of single-factor models can be improved by combining factors, using a spreading-activation based approach, to produce multi-factor models. Based on these results, we offer concrete guidance about how these models could be used to provide enhanced software development tools that ease the difficulty of navigating through code.

Designers often have no access to individuals who use screen reading software, and may have little understanding of how their design choices impact these users. We explore here whether cog-nitive models of auditory interaction could provide insight into screen reader usability. By comparing human data with a tool-generated model of a practiced task performed using a screen reader, we identify several requirements for such models and tools. Most important is the need to represent parallel execution of hearing with thinking and acting. Rules for placement of cogni-tive operators that were developed for visual user interfaces may not be applicable in the auditory domain. Other mismatches be-tween the data and the model were attributed to the extremely fast listening rate and differences between the typing patterns of screen reader usage and the model's assumptions. This work in-forms the development of more accurate models of auditory inter-action. Tools incorporating such models could help designers create user interfaces that are well tuned for screen reader users, without the need for modeling expertise.

The long-term goal of Project Halo is to build an application called Digital Aristotle that can answer questions on a wide variety of science topics and provide user- and domain-appropriate explanations. As a near-term goal, we are focusing on enabling subject matter experts (SMEs) to construct declarative knowledge bases (KBs) from 50 pages of a science textbook in the domains of Physics, Chemistry, and Biology in a way that the system can answer questions similar to those in an Advanced Placement (AP) exam in the respective discipline. The textbook knowledge is a mixture of textual information, mathematical equations, tables, diagrams, and domain-specific representations such as chemical reactions. In this paper, we explore the following question: Can we build a knowledge capture system to enable SMEs to construct KBs from the knowledge found in science textbooks and use the resulting KB for deductive question answering? We answer this question in the context of a system called AURA that supports knowledge capture from science textbooks.

As part of the ongoing project, Project Halo, our goal is to build a system capable of answering questions posed by novice users to a formal knowledge base. In our current context, the knowledge base covers selected topics in physics, chemistry, and biology, and our question set consists of AP (advanced high-school) level examination questions. The task is challenging because the questions are linguistically complex and are often incomplete (assume unstated knowledge), and because the users do not have prior knowledge of the system's contents. Our solution involves two parts: a controlled language interface, in which users reformulate the original natural language questions in a simplified version of English, and a novel problem solver that can elaborate initially inadequate logical interpretations of a question by selecting relevant pieces of knowledge in the knowledge base. An evaluation of the work in 2006 showed that this approach is feasible and that complex, multisentence questions can be posed and answered, thus illustrating novel ways of dealing with the knowledge capture impedance between users and a formal knowledge base, while also revealing challenges that still remain.

Design patterns have been claimed to facilitate modification and improve understanding in software design. A controlled experiment was performed to assess the usefulness of portions of a usability-supporting architectural pattern (USAP) in modifying the design of software architectures to support a specific usability concern. Software engineering and information technology graduate students received different subsets of a USAP supporting cancellation functionality. They then studied a software architecture design and made modifications to add the ability to cancel commands. Results showed that participants who received a usability scenario, a list of general responsibilities, and a sample solution thought of significantly more key issues than participants who saw only the scenario. Implications for software development are that usability concerns can be included at architecture design time, and that USAPs can significantly help software architects to consider responsibilities inherent from usability concerns.

A priori prediction of skilled human performance has the potential to be of great practical value but is difficult to carry out. This article reports on an approach that facilitates modeling of human behavior at the level of cognitive, perceptual, and motor operations, following the CPM-GOMS method (John, 1990). CPM-GOMS is a powerful modeling method that has remained underused because of the expertise and labor required. We describe a process for automatically generating CPM-GOMS models from a hierarchical task decomposition expressed in a computational modeling tool, taking advantage of reusable behavior templates and their efficacy for generating zero-parameter a priori predictions of complex human behavior. To demonstrate the process, we present a model of automated teller machine interaction. The model shows that it is possible to string together existing behavioral templates that compose basic HCI tasks, (e.g., mousing to a button and clicking on it) to generate powerful human performance predictions. Because interleaving of templates is now automated, it becomes possible to construct arbitrarily long sequences of behavior. In addition, the manipulation and adaptation of complete models has the potential of becoming dramatically easier. Thus, the tool described here provides an engine for CPM-GOMS that may facilitate computational modeling of human performance at the millisecond level.

Although engineering models of user behavior have enjoyed a rich history in HCI, they have yet to have a widespread impact due to the complexities of the modeling process. In this paper we describe a development system in which designers generate predictive cognitive models of user behavior simply by demonstrating tasks on HTML mock-ups of new interfaces. Keystroke-Level Models are produced automatically using new rules for placing mental operators, then implemented in the ACT-R cognitive architecture. They interact with the mock-up through integrated perceptual and motor modules, generating behavior that is automatically quantified and easily examined. Using a query-entry user interface as an example [19], we demonstrate that this new system enables more rapid development of predictive models, with more accurate results, than previously published models of these tasks.

Software architects have techniques to deal with many quality attributes such as performance, reliability, and maintainability. Usability, however, has traditionally been concerned primarily with presentation and not been a concern of software architects beyond separating the user interface from the remainder of the application. In this paper, we present usability-supporting architectural patterns. Each pattern describes a usability concern that is not supported by separation alone. For each concern, a usability-supporting architectural pattern provides the forces from the characteristics of the task and environment, the human, and the state of the software to motivate an implementation independent solution cast in terms of the responsibilities that must be fulfilled to satisfy the forces. Furthermore, each pattern includes a sample solution implemented in the context of an overriding separation based pattern such as J2EE Model View Controller.

CPM-GOMS is a modeling method that combines the task decomposition of a GOMS analysis with a model of human resource usage at the level of cognitive, perceptual, and motor operations. CPM-GOMS models have made accurate predictions about skilled user behavior in routine tasks, but developing such models is tedious and error-prone. We describe a process for automatically generating CPM-GOMS models from a hierarchical task decomposition expressed in a cognitive modeling tool called Apex. Resource scheduling in Apex automates the difficult task of interleaving the cognitive, perceptual, and motor resources underlying common task operators (e.g. mouse move-and-click). Apex’s UI automatically generates PERT charts, which allow modelers to visualize a model’s complex parallel behavior. Because interleaving and visualization is now automated, it is feasible to construct arbitrarily long sequences of behavior. To demonstrate the process, we present a model of automated teller interactions in Apex and discuss implications for user modeling.

Mobile computing has been an active area of research for the past decade, but its importance will increase substantially in the decade to come. One problem faced by designers of mobile systems is that of maintaining the illusion of connectivity even when network performance is poor or non-existent. The Coda file system uses its cache to maintain this illusion. Extensive experience with the system suggests that, although users find the functionality provided by the system extremely valuable, new users face an arduous learning curve and even experienced users are sometimes confused by the system's behavior. The fundamental problem is that the lack of a strong network connection causes the system to violate a key property of caching: transparency. To overcome this problem, we have built an interface, called the CodaConsole, that makes caching translucent to users through controlled exposure of cache management internals. The interface exposes critical aspects of caching to support the mobile user while hiding noncritical details to preserve usability. This article presents the design, implementation, and usability evaluation of this interface. The CodaConsole successfully makes caching translucent in the presence of disconnected or weakly connected operation. The most surprising result was that novice Coda users performed almost as well as experienced Coda users.

Despite experience, many users do not make efficient use of complex computer applications. We argue that this is caused by a lack of strategic knowledge that is difficult to acquire just by knowing how to use commands. To address this problem, we present efficient and general strategies for using computer applications, and identify the components of strategic knowledge required to use them. We propose a framework for teaching strategic knowledge, and show how we implemented it in a course for freshman students. In a controlled study, we compared our approach to the traditional approach of just teaching commands. The results show that efficient and general strategies can in fact be taught to students of diverse backgrounds in a limited time without harming command knowledge. The experiment also pinpointed those strategies that can be automatically learned just from learning commands, and those that require more practice than we provided. These results are important to universities and companies that wish to foster more efficient use of complex computer applications.

The role of software architecture with respect to usability has evolved over the past 20 years. The architectures of the 1980s and early 1990s assumed that usability was primarily a property of the presentation of information. Therefore, simply separating the presentation from the dialogue and application made it easy to modify that presentation after user testing. A more popular belief in the 1990s was that usability concerns greatly affected system functionality as well as the presentation. This emphasis took attention away from architectural support (beyond separation). Achieving the correct functionality for a given system became paramount. It is our observation that even if presentation and functionality of a system are well designed, the usability of a system can be greatly compromised if the underlying architecture does not support human concerns beyond modifiability. This paper will present a new role for software architecture in usability, preliminary research and practice stemming from this role and a research agenda for the future.

We compare three tools for creating GOMS models, QGOMS [2), CATHCI (17) and GLEAN3 [12], along several dimensions. We examine the representation and available constructs in each tool, the qualitative and quantitative design information provided, the support for building cognitively plausible models, and pragmatics about using each tool (e.g., how easy it is to modify a model). While each tool has its strengths, they all leave something to be desired as a practical UI design tool.

Several studies show that despite experience, many users with basic command knowledge do not progress to an efficient use of complex computer applications. These studies suggest that knowledge of tasks and knowledge of tools are insufficient to lead users to become efficient. To address this problem, we argue that users also need to learn strategies in the intermediate layers of knowledge lying between tasks and tools. These strategies are (a) efficient because they exploit specific powers of computers, (b) difficult to acquire because they are suggested by neither tasks nor tools, and (c) general in nature having wide applicability. The above characteristics are first demonstrated in the context of aggregation strategies that exploit the iterative power of computers. A cognitive analysis of a real-world task reveals that even though such aggregation strategies can have large effects on task time, errors, and on the quality of the final product, they are not often used by even experienced users. We identify other strategies beyond aggregation that can be efficient and useful across computer applications and show how they were used to develop a new approach to training with promising results. We conclude by suggesting that a systematic analysis of strategies in the intermediate layers of knowledge can lead not only to more effective ways to design training but also to more principled approaches to design systems. These advances should lead users to make more efficient use of complex computer systems.

The inefficient use of complex computer systems has been widely reported. These studies show the persistence of inefficient methods despite many years of experience and formal training. To counteract this phenomenon, we present the design of a new course, called the Strategic Use of CAD. The course aims at teaching students efficient strategies to use a computer-aided drafting system through a two-pronged approach. Learning to See teaches students to recognize opportunities to use efficient strategies by studying the nature of the task, and Learning to Do teaches students to implement the strategies. Results from a pilot experiment show that this approach had a positive effect on the strategic behavior of students who did not exhibit knowledge of efficient strategies before the class, and had no effect on the strategic behavior of those who did. Strategic training can thus assist users in recognizing opportunities to use efficient strategies. We present the ramifications of these results on the design of training and future experiments.

A prerequisite to the effective design of user interfaces is an understanding of the tasks for which that interface will actually be used. Surprisingly little task analysis has appeared for one of the most discussed and fastest-growing computer applications, browsing the World-Wide Web (WWW). Based on naturally-collected verbal protocol data, we present a taxonomy of tasks undertaken on the WWW. The data reveal that several previous claims about browsing behavior are questionable, and suggests that that widget-centered approaches to interface design and evaluation may be incomplete with respect to good user interfaces for the Web.

A central goal of many user interface development tools has been to make the construction of high quality interfaces easy enough that iterative design approaches could be a practical reality. In the last 15 years significant advances in this regard have been achieved. However, the evaluation portion of the iterative design process has received relatively little support from tools. Even though advances have also been made in usability evaluation methods, nearly all evaluation is still done “by hand,” making it more expensive and difficult than it might be. This paper considers a partial implementation of the CRITIQUE usability evaluation tool that is being developed to help remedy this situation by automating a number of evaluation tasks. This paper will consider techniques used by the system to produce predictive models (keystroke level models and simplified GOMS models) from demonstrations of sample tasks in a fraction of the time needed by conventional handcrafting methods. A preliminary comparison of automatically generated models with models created by an expert modeler show them to produce very similar predictions (within 2%). Further, because they are automated, these models promise to be less subject to human error and less affected by the skill of the modeler.

Throughout history, inefficient methods to use devices have been replaced by more efficient ones.This shift typically occurs when users discover how to &legate work to the powers of a tool, and to circumvent its liiitations.Strategies of delegation and circumvention, therefore, appear to be the core of efficient use.To show how this approach can expiain the relationship between tools and strategies in complex computer systems, we describe five ways to perform a real-world drawing task with current as well as 5.rture tools.We then present five corresponding GOMS models that demonstrate the value of efficient strategies when compared to the observed behavior of a professional CAD user.We conclude by presenting a generalized framework to characterize efficient strategies and discuss its relevance to design and training.

Can good design guarantee the eflicient use of computer tools?Can experience guarantee it?We raise these questions to explore why empirical studies of real-world usage show even experienced users under-utilizing the capabilities of computer applications.By analyzing the use of everyday devices and computer applications, as well as reviewing empirical studies, we conclude that neither good design nor experience may be able to guarantee efficient usage.Efficient use requires task decomposition strategies that exploit capabilities offered by computer applications such as the ability to aggregute objects, and to manipulate the aggregates with powerful operators.To understand the effects that strategies can have on performance, we present results from a GOMS analysis of a CAD task.Furthermore, we identify some key aggregation strategies that appear to generalize across applications.Such strategies may provide a framework to enable users to move from a sufficient to a more ef)icient use of computer tools.

We present a case study that tracks usability problems predicted with six usability evaluation methods (claims analysis, cognitive walkthrough, GOMS, heuristic evaluation, user action notation, and simply reading the specification) through a development process. We assess the method's predictive power by comparing the predictions to the results of user tests. We assess the method's persuasive power by seeing how many problems led to changes in the implemented code. We assess design-change effectiveness by user testing the resulting new versions of the system. We concludethatpredictivemethodsarenot as effective as the HCI field would like and discuss directions for future research.

We present a detailed case study of a computer scientist learning and using the Cognitive Walkthrough (CW) technique to assess a multimedia authoring tool. We then compare the predictions produced by the analysis to the usability problems actually found in empirical usability tests. In this case, the concepts of Cognitive Walkthrough were learnable by the analyst who had little prior training in psychology or Human-Computer Interaction, flexible enough for the analyst to make reasonable modifications to the technique to fit the design situation, but disappointingly ineffective in predicting actual user problems. We present several hypotheses about the cause of low effectiveness, which suggest that additional knowledge, currently tacit in CW-experts, could be used to improve the technique. In addition, the emergent picture of the process this evaluator went through to produce his analysis sets realistic expectations for other novice analysts who contemplate learning and using Cognitive Walkthroughs. © 1997 John Wiley & Sons, Inc.

Despite huge investments by vendors and users, CAD productivity remains disappointing. Our analysis of real-world CAD usage shows that even after many years of experience, users tend to use suboptimal strategies to perform complex CAD tasks. Additionally, some of these strategies have a marked resemblance to manual drafting techniques. Although this phenomenon has been previously reported, this paper explores explanations for its causes and persistence. We argue that the strategic knowledge to use CAD effectively is neither defined nor explicitly taught. In the absence of a well-formed strategy, users often develop a synthetic mental model of CAD containing a mixture of manual and CAD methods. As these suboptimal strategies do not necessarily prevent users from producing clean, accurate drawings, the inefficiencies tend to remain unrecognized and users have little motivation to develop better strategies. To reverse this situation we recommend that the strategic knowledge to use CAD effectively should be made explicit and provided early in training. We use our analysis to begin the process of making this strategic knowledge explicit. We conclude by discussing the ramifications of this research in training as well as in the development of future computer aids for drawing and design.

We present an evaluation of a multimedia educational software system that includes text, graphics, animations, and simulations.When compared with an informationally equivalent control environment that used text and carefully selected still images, we found little evidence that the dynamic presentations enhanced student understanding of the declarative information in this lesson.Furthermore, students cannot be relied on to take fllll advantage of exploratory opportunities in computer-based instruction.These results prescribe further investigation of whether and how computer-based multimedia can be used effectively in education and training.

TYPIST is a Theory of Performance In Skilled Typing built within the framework of the Model Human Processor (MHP; Card, Moran, & Newell, 1983). As such, it can be used to make quantitative predictions of performance on typing tasks and can be integrated with other MHP-based models of performance. In this article, I present the theory and explain the source of each theoretical assumption (MHP, typing task analysis, or empirical typing data). I then demonstrate different ways to use TYPIST by applying it to six transcription typing tasks. Finally, I summarize its application to many more typing tasks that display robust behavioral phenomena identified by Salthouse (1986).

Since the seminal book, The Psychology of Human-Computer Interaction , the GOMS model has been one of the few widely known theoretical concepts in human-computer interaction. This concept has spawned much research to verify and extend the original work and has been used in real-world design and evaluation situations. This article synthesizes the previous work on GOMS to provide an integrated view of GOMS models and how they can be used in design. We briefly describe the major variants of GOMS that have matured sufficiently to be used in actual design. We then provide guidance to practitioners about which GOMS variant to use for different design situations. Finally, we present examples of the application of GOMS to practical design problems and then summarize the lessons learned.

Sine the publication of The Psychology of Human-Computer Interaction , the GOMS model has been one of the most widely known theoretical concepts in HCI. This concept has produced severval GOMS analysis techniques that differ in appearance and form, underlying architectural assumptions, and predictive power. This article compares and contrasts four popular variantsof the GOMS family (the Keystroke-Level Model, the original GOMS formulation, NGOMSL, and CPM-GOMS) by applying them to a single task example.

Article Display navigation by an expert programmer: a preliminary model of memory Share on Authors: Erik M. Altmann Computer Science, Carnegie Mellon University Computer Science, Carnegie Mellon UniversityView Profile , Jill H. Larkin Psychology, Carnegie Mellon University Psychology, Carnegie Mellon UniversityView Profile , Bonnie E. John Computer Science, Psychology and HCI Institute, Carnegie Mellon University Computer Science, Psychology and HCI Institute, Carnegie Mellon UniversityView Profile Authors Info & Claims CHI '95: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsMay 1995 Pages 3–10https://doi.org/10.1145/223904.223905Online:01 May 1995Publication History 4citation364DownloadsMetricsTotal Citations4Total Downloads364Last 12 Months7Last 6 weeks1 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

Article Modeling time-constrained learning in a highly interactive task Share on Authors: Malcolm I. Bauer Department of Computer Science and HCI Institute, Carnegie Mellon University, Pittsburgh, PA Department of Computer Science and HCI Institute, Carnegie Mellon University, Pittsburgh, PAView Profile , Bonnie E. John Departments of Computer Science and Psychology and HCI Institute, Carnegie Mellon University, Pittsburgh, PA Departments of Computer Science and Psychology and HCI Institute, Carnegie Mellon University, Pittsburgh, PAView Profile Authors Info & Claims CHI '95: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsMay 1995 Pages 19–26https://doi.org/10.1145/223904.223907Online:01 May 1995Publication History 7citation272DownloadsMetricsTotal Citations7Total Downloads272Last 12 Months9Last 6 weeks1 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

We present a detailed case study, drawn from many information sources, of a computer scientist learning and using Cognitive Walkthroughs to assess a multi-media authoring tool.This study results in several clear messages to both system designers and to developers of evaluation techniques: this technique is currenlty learnable and usable, but there are several areas where further methoddevelopment would greatly contribute to a designer's use of the technique.In addition, the emergent picture of the process this evaluator went through to produce his analysis sets realistic expectations for other novice evaluators who contemplate learning and using Cognitive Walkthroughs.

We present an analysis of an expert performing a highly interactive computer task. The analysis uses GOMS models, specifying the Goals, Operators, Methods, and Selection rules used by the expert. Two models are presented, one with function-level operators which perform high-level functions in the domain, and one with keystroke-level operators which describe hand movements. For a segment of behaviour in which the expert accomplished about 30 functions in about 30 s, the function-level model predicted the observed behaviour well, while the keystroke-level model predicted only about half of the observed hand movements. These results, including the discrepancy between the models, are discussed.

Project Ernestine served a pragmatic as well as a scientific goal: to compare the worktimes of telephone company toll and assistance operators on two different workstations and to validate a GOMS analysis for predicting and explaining real-world performance. Contrary to expectations, GOMS predicted and the data confirmed that performance with the proposed workstation was slower than with the current one. Pragmatically, this increase in performance time translates into a cost of almost $2 million a year to NYNEX. Scientifically, the GOMS models predicted performance with exceptional accuracy. The empirical data provided us with three interesting results: proof that the new workstation was slower than the old one, evidence that this difference was not constant but varied with call category, and (in a trial that spanned 4 months and collected data on 72,450 phone calls) proof that performance on the new workstation stabilized after the first month. The GOMS models predicted the first two results and explained all three. In this article, we discuss the process and results of model building as well as the design and outcome of the field trial. We assess the accuracy of GOMS predictions and use the mechanisms of the models to explain the empirical results. Last, we demonstrate how the GOMS models can be used to guide the design of a new workstation and evaluate design decisions before they are implemented.

INTRODUCTION Project Ernestine served a pragmatic as well as a scientific goal: to compare the worktimes of telephone company toll and assistance operators on two different workstations, and to test the validity of GOMSl models for predicting and explaining real-world performance. Contrary to expectations, GOMS predicted and the data confirmed, that performance with the proposed workstation was slower than with the current one, Pragmaticly, this increase in performance time translates into a cost of $2.4 million dollars a year to NYNEX. Scientificly, the GOMS models predicted performance with exceptional accuracy.

A GOMS analysis was used to predict the behavior of an expert in a graphic, machine-paced, highly interactive task. The analysis was implemented in a computational model using the Soar cognitive architecture. Using only the information available in an instruction booklet and some simple heuristics for selecting between operators, the functional-level behavior of the expert proved to be virtually dictated by the objects visible on the display. At the keystroke-level, the analysis predicted about 60% of the behavior, in keeping with similar results in previous GOMS research. We conclude that GOMS is capable of predicting expert behavior in a broader range of tasks than previously demonstrated.

Browser-Soar models the perceptual, cognitive, and motor operators of a user searching for information in an on-line help browser. The model accounts for 90% of the browsing behavior observed in ten episodes. This result suggests that much of browsing behavior is a routine cognitive task, describably by GOMS, and extends the boundary of tasks to which GOMS applies to include highly interactive tasks. Further, it also suggests that GOMS analyses can be used to evaluate text-editors and other computer applications, and to help focus design effort.

GOMS models of telephone toll and assistance operators (TAOS) are being constructed in an effort to provide theoretical predictions of expert performance on several dedicated workstations. This applied effort has pushed the development of GOMS modeling techniques into the area of speech input and output, and into a task where information is not always available when it is required by the TAO. This paper describes the task, heuristics for constructing the GOMS models, and parameters for making quantitative predictions of performance time.

In keeping with our claim that an applied psychology of HCI must be based on cumulative work within a unified framework, we present two extensions of the Model Human Processor. A model of immediate response behavior and stimulus-response (S-R) compatibility is presented and extended to a new domain: transcription typing. Parameters are estimated using one S-R compatibility experiment, used to make a priori predictions in four other S-R compatibility tasks, and then carried over into the area of typing. A model of expert transcription typing is described and its prediction of typing phenomena is demonstrated and summarized.

article Free Access Share on The role of laboratory experiments in HCI: help, hindrance, or ho-hum? Authors: C. G. Wolf IBM Watson Research Center, Yorktown Heights, NY IBM Watson Research Center, Yorktown Heights, NYView Profile , J. M. Carroll IBM Watson Research Center, Yorktown Heights, NY IBM Watson Research Center, Yorktown Heights, NYView Profile , T. K. Landauer Bellcore, Morristown, NJ Bellcore, Morristown, NJView Profile , B. E. John Carnegie Mellon Univ., Pittsburgh, PA Carnegie Mellon Univ., Pittsburgh, PAView Profile , J. Whiteside Digital Equipment Corporation Digital Equipment CorporationView Profile Authors Info & Claims ACM SIGCHI BulletinVolume 20Issue SIMarch 1989 pp 265–268https://doi.org/10.1145/67450.67500Online:01 March 1989Publication History 8citation0DownloadsMetricsTotal Citations8Total Downloads0Last 12 Months0Last 6 weeks0 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

A GOMS theory of stimulus-response compatibility is shown to predict response-time performance on a command/abbreviation encoding task. Working with parameters that were set by an earlier study and which have rational, task-meaningful interpretations as mapping, motor, perception and retrieval operators, zero-parameter predictions were made that fit the observed performance with r2 = 0.776 (p<0.05). The reasonableness of the parameters, the algorithms used to generate the predictions, and the weighting assumption used to combine algorithms into a single prediction are discussed.