select a product from among alternative products

This article reports on the results of the second meeting and workshop of the International Scientific Forum, "Towards an Information Society for All," that took place in Crete, Greece, June 15-16, 1998. In particular, it elaborates on the international research and development agenda (Stephanidis et al., 1998), which resulted from the first meeting and workshop of the Forum in San Francisco, California, on August 29, 1997, in the context of the HCI International '97 Conference. This article elaborates on the proposed research and development agenda by identifying human-computer interaction challenges and clusters of concrete recommendations for international collaborative research and technological development (RTD) activities. Four clusters of recommendations are proposed. The first three facilitate reaching technological targets, and the fourth comprises accompanying measures. The three technological clusters concern the corresponding transitions from (a) productivity tools to environments of use, (b) individual users to communities of users, and (c) computer-assisted business tasks to computer-mediated human activities. The fourth cluster covers support (horizontal) actions needed to establish a favorable environment for the creation of an information society acceptable to all citizens. Each cluster is elaborated by means of specific recommendations, plausible RTD objectives, and likely or expected outcomes.

This paper reports a method for measuring usability in terms of task performance-achievement of frequent and critical task goals by particular users in a context simulating the work environment. The terms usability and quality in use are defined in international standards as the effectiveness, efficiency and satisfaction with which goals are achieved in a specific context of use. The performance measurement method gives measures which, in combination with measures of satisfaction, operationalize these definitions. User performance is specified and assessed by measures including task effectiveness (the quantity and quality of task performance) and User efficiency (effectiveness divided by tasktime). Measures are obtained with users performing tasks in a context of evaluation which matches the intended context of use. This can also reveal usability problems which may not become evident if the evaluator interacts with the user. The method is supported by tools which make it practical in commercial timescales. The method has been widely applied in industry, and can be adapted for use early in design, and to evaluate non-computer products and the performance of small work groups.

Different approaches to the measurement of usability are reviewed and related to definitions of usability in international standards. It is concluded that reliable measures of overall usability can only be obtained by assessing the effectiveness, efficiency and satisfaction with which representative users carry out representative tasks in representative environments.This requires a detailed understanding of the context of use of a product. The ESPRIT MUSiC project has developed tools which can be used to measure usability in the laboratory and the field. An overview is given of the methods and tools for measuring user performance, cognitive workload, and user perceived quality.