The Cognitive Walkthrough for the Web (CWW) is a partially automated usability evaluation method for identifying and repairing website navigation problems. Building on five earlier experiments [3,4], we first conducted two new experiments to create a sufficiently large dataset for multiple regression analysis. Then we devised automatable problem-identification rules and used multiple regression analysis on that large dataset to develop a new CWW formula for accurately predicting problem severity. We then conducted a third experiment to test the prediction formula and refined CWW against an independent dataset, resulting in full cross-validation of the formula. We conclude that CWW has high psychological validity, because CWW gives us (a) accurate measures of problem severity, (b) high success rates for repairs of identified problems (c) high hit rates and low false alarms for identifying problems, and (d) high rates of correct rejections and low rates of misses for identifying non-problems.

Methods for identifying usability problems in web page designs should ideally also provide practical methods for repairing the problems found. Blackmon et al. [2] proved the usefulness of the Cognitive Walkthrough for the Web (CWW) for identifying three types of problems that interfere with users' navigation and information search tasks. Extending that work, this paper reports a series of two experiments that develop and prove the effectiveness of both full-scale and quick-fix CWW repair methods. CWW repairs, like CWW problem identification, use Latent Semantic Analysis (LSA) to objectively estimate the degree of semantic similarity (information scent) between representative user goal statements (100-200 words) and heading/link texts on each web page. In addition to proving the effectiveness of CWW repairs, the experiments reported here replicate CWW predictions that users will face serious difficulties if web developers fail to repair the usability problems that CWW identifies in web page designs [2].

This paper proposes a transformation of the Cognitive Walkthrough (CW), a theory-based usability inspection method that has proven useful in designing applications that support use by exploration. The new Cognitive Walkthrough for the Web (CWW) is superior for evaluating how well websites support users' navigation and information search tasks. The CWW uses Latent Semantic Analysis to objectively estimate the degree of semantic similarity (information scent) between representative user goal statements (100-200 words) and heading/link texts on each web page. Using an actual website, the paper shows how the CWW identifies three types of problems in web page designs. Three experiments test CWW predictions of users' success rates in accomplishing goals, verifying the value of CWW for identifying these usability problems

(2000). Introduction to This Special Issue on New Agendas for Human-Computer Interaction. Human–Computer Interaction: Vol. 15, New Agendas for Human-Computer Interaction, pp. 69-74.

Article Free Access Share on Relationships between users' and interfaces' task representations Authors: Robert B. Terwilliger Institute of Cognitive Science, University of Colorado, Boulder, CO Institute of Cognitive Science, University of Colorado, Boulder, COView Profile , Peter G. Polson Institute of Cognitive Science, University of Colorado, Boulder, CO Institute of Cognitive Science, University of Colorado, Boulder, COView Profile Authors Info & Claims CHI '97: Proceedings of the ACM SIGCHI Conference on Human factors in computing systemsMarch 1997 Pages 99–106https://doi.org/10.1145/258549.258617Published:27 March 1997Publication History 5citation440DownloadsMetricsTotal Citations5Total Downloads440Last 12 Months25Last 6 weeks7 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

The linked model of comprehension-based action planning and instruction taking (LICAI) simulates performing by exploration tasks using applications hosted on systems with graphical user interfaces. The tasks are given to the user as written exercises containing no information about the correct action sequences. LICAI's comprehension and action-planning processes are based on Kintsch's construction-integration (C-I) theory for text comprehension. The model assumes that comprehending instructions is a strategic process; instruction texts must be elaborated using specialized strategres that guide goal generation. LICAI comprehends the instructions and generates goals that are then stored in memory. The action-planning processes are controlled by goals retrieved from memory. Representations of goals that can guide exploration are restricted by the C-I architecture. The model predicts that successful exploration requires linking of the goal representation with the label on the correct object. The model is evaluated by comparing its predictions with results from an experimental study of learning by exploration by Franzke (1994, 1995). We discuss the implications of LICAI for designing instruction materials and interfaces that facilitate exploration.

Article A comprehension-based model of exploration Share on Authors: Muneo Kitajima National Institute of Bioscience and Human-Technology, 1-1 Higashi Tsukuba Ibaraki 305, Japan National Institute of Bioscience and Human-Technology, 1-1 Higashi Tsukuba Ibaraki 305, JapanView Profile , Peter G. Polson Institute of Cognitive Science, University of Colorado, Boulder, CO Institute of Cognitive Science, University of Colorado, Boulder, COView Profile Authors Info & Claims CHI '96: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsApril 1996 Pages 324–331https://doi.org/10.1145/238386.238536Online:13 April 1996Publication History 11citation611DownloadsMetricsTotal Citations11Total Downloads611Last 12 Months5Last 6 weeks0 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

Using models developed to analyze office automation (e.g.[3]), we identified skilIs needed to perform tasks using the flight management computer on advanced commercial aircraft, Our GOMS analysis showed that all tasks carried out on the device can be described in terms of three methods.Novices who received instruction in a part-task computer based training guided by the GOMS analysis were tested in a flight simulator, Their performance was compared with pilots who received an integrated type of training representing three to ten times the amount of time on task and with pilots who had been using this equipment on the line for at least one year ("experts").Inconsistencies revealed by the analysis were reflected in the performance of novices (experimentally and professionally trained) as well as the experts.

Users who have worked with just one or two pieces of application software on a computer system are often faced with the need to use a new piece of software on the same system.Consistency between program interfaces is intended to make the new program easier to learn in this situation, but how "consistency" should be defined is not always clear.We present a model of analogical reasoning that describes how users rely on interface consistency to induce correct actions in a new situation.Versions of the model are implemented in ACT-R and Soar.The model yields a clearer and more principled understanding of design guidelines that recommend interface consistency.

This paper describes a computational model of skilled use of a graphical user interface based on Kintsch's construction-integration theory [4, 8]. The model uses knowledge of a detailed representation of information on the display, a user's goals and expectations, knowledge about the interface, and knowledge about the application domain to compute actions necessary to accomplish the user's current goal. The model provides a well-motivated account of one kind of errors, action slips [14], made by skilled users. We show how information about the intermediate state of a task on the display plays a critical role in skilled performance, i.e., display-based problem solving.

This paper explores the behavior of experts choosing among various methods to accomplish tasks. Given the results showing that methods are not chosen solely on the basis of keystroke efficiency, we recommend a technique to help designers assess whether they should offer multiple methods for some tasks, and if they should, how to make them so that they are chosen appropriately.

No abstract available.

The value of theoretical analyses in user interface design has been hotly debated. All sides agree that it is difficult to apply current theoretical models within the constraints of real-world development projects. We attack this problem in the context of bringing the theoretical ideas within a model of exploratory learning [19] to bear on the evaluation of alternative interfaces for walk-up-and-use systems. We derived a “cognitive walkthrough” procedure for systematically evaluating features of an interface in the context of the theory. Four people independently applied this procedure to four alternative interfaces for which we have empirical usability data. Consideration of the walkthrough sheds light on the consistency with which such a procedure can be applied as well as the accuracy of the results.

Kieras and Polson (1985) proposed an approach for making quantitative predictions on ease of learning and ease of use of a system, based on a production system version of the goals, operators, methods, and selection rules (GOMS) model of Card, Moran, and Newel1 (1983). This article describes the principles for constructing such models and obtaining predictions of learning and execution time. A production rule model for a simulated text editor is described in detail and is compared to experimental data on learning and performance. The model accounted well for both learning and execution time and for the details of the increase in speed with practice. The relationship between the performance model and the Keystroke-Level Model of Card et al. (1983) is discussed. The results provide strong support for the original proposal that production rule models can make quantitative predictions for both ease of learning and ease of use.

Many important computer applications require that users be able to use them effectively with little or no formal training. Current examples include bank teller machines and airport information kiosks. Today successful systems of this kind can only be developed by iteration using costly empirical testing. This article aims to provide a theoretical foundation for the design of such systems, a model of learning by exploration called CE+. The theory incorporates assumptions from (a) the GOMS model and cognitive complexity theory (CCT) on the representation of procedural knowledge as productions, (b) the EXPL model on learning from examples, and (c) research on problem-solving processes for simple puzzlelike problems. Design guidelines for systems that can be learned by exploration, "design for successful guessing," are derived from the theory. These principles are compared with those developed by Norman (1988).

This study presents a theoretically motivated analysis of learning and performance on a micro-processor based oscilloscope. An analysis of the knowledge required to make basic measurements was done using the GOMS model and Cognitive Complexity Theory (CCT). From these analyses and the criterion used in Polson, Muncher, and Engelbeck (1986), tasks were selected for an experiment evaluating training order manipulations using naive users of oscilloscopes. Production system models for each training task were derived from CCT. The models successfully predicted transfer between tasks and training order effects. Implications for the design of systems with embedded micro-processors are discussed.

This paper investigates whether changes in the user/computer dialogue structure will affect the performance of users who are familiar with an earlier version of the product. Quantitative predictions using the Kieras and Polson (1985) production system model were derived to test whether changing the lexical attributes and structure of a popular menu-driven word-processor would permit transfer of existing knowledge of the word-processor to a new version. The results show that changes to the dialogue structure of the menu-system are not detrimental, while changes to the lexical attributes of the menus will hinder user performance.

This paper describes a successful test of a quantitative model that accounts for large positive transfer effects between similar screen editors, between different line editors and from line editors to a screen editor, and between text and graphic editors. The model is tested in an experiment using two very similar full-screen text-editors differing only in the structure of their editing commands, verb-noun vs noun-verb. Quantitative predictions for training time were derived from a production system model based on the Polson and Kieras (1985) model of text editing.

All discussions of interface design criteria emphasize the importance of consistent operating procedures both within and across applications. This paper presents a model for positive transfer and thus a theoretical definition of consistency. An experiment manipulating training orders for utility tasks was designed to evaluate the transfer model. The experimental manipulations produced large transfer effects. Quantitative predictions were derived from the Kieras and Polson (1985) theory of human-computer interaction and the transfer model and fit using regression techniques. The transfer model accounted for 88% of the variance of the 31 cell means.

A model of manuscript editing, implemented as a simulation program, is described in this paper. The model provides an excellent, quantitative description of learning, transfer, and performance data from two experiments on text editing methods. Implications of the underlying theory for the design process are briefly discussed.

A general method for describing the behavior of an interactive system is presented which is based on transition networks generalized enough to describe even very complex systems easily, as shown by an example description of a word processor. The key feature is the ability to easily describe hierarchies of modes or states of the system. The representation system is especially valuable as a design tool when used in a simulation of a proposed user interface.