Mobile technology requires new methods for studying its use under realistic conditions "in the field." Reflexively, mobile technology also creates new opportunities for data collection while participants are remotely located. We report on our experiences with a variation on the paper-based diary study technique, which we extend by using voice-mail paired with mobile and landline telephony to more easily collect data in natural situations. We discuss lessons learned from experiences with voice-mail diary studies in two investigations of different scope. We also present suggestions for tailoring the technique to different research objectives, garnering high subject participation, and configuring the voice-mail system for data collection.

Service-based wireless devices like wireless telephones require users to interact with aspects of the technology beyond the hardware and software of the handset. By entering into contractual relationships with service-providers, and by using network-based services, users interact with a larger system ---one that has social and technological components. The operation of the wireless telephone requires the assimilation of heterogeneous sources of information from the device manufacturer, sales people, customer service representatives, marketing people, and members of the popular media, among others, which can easily confound users' understanding of this new class of technology. Opportunities for usability problems therefore scale beyond the handset, as do opportunities for better design. We report the results of a study of 19 novice wireless phone users who were closely tracked for the first 6 weeks after service acquisition. Taking a technology-as-system analytical approach, we describe the wireless telephony system as four socio-technical components: hardware, software, "netware," and "bizware." This particular organization of the system is intended for the practical application of designing for usability.

We report on the results of a study in which 19 new mobile phone users were closely tracked for the first six weeks after service acquisition. Results show that new users tend to rapidly modify their perceptions of social appropriateness around mobile phone use, that actual nature of use frequently differs from what users initially predict, and that comprehension of service-oriented technologies can be problematic. We describe instances and features of mobile telephony practice. When in use, mobile phones occupy multiple social spaces simultaneously, spaces with norms that sometimes conflict: the physical space of the mobile phone user and the virtual space of the conversation.

This paper describes a research study that investigated how designers can use frames of reference (egocentric, exocentric, and a combination of the two) to support the mastery of abstract multidimensional information. The primary focus of this study was the relationship between FORs and mastery; the secondary focus was on other factors (individual characteristics and interaction experience) that were likely to influence the relationship between FORs and mastery. This studys outcomes (1) clarify how FORs work in conjunction with other factors in shaping mastery, (2) highlight strengths and weaknesses of different FORs, (3) demonstrate the benefits of providing multiple FORs, and (4) provide the basis for our recommendations to HCI researchers and designers.

An interest in the design of interfaces has been a core topic for researchers and practitioners in the field of human-computer interaction (HCI); an interest in the design of experiments has not. To the extent that reliable and valid guidance for the former depends on the results of the latter, it is necessary that researchers and practitioners understand how small features of an experimental design can cast large shadows over the results and conclusions that can be drawn. In this review we examine the design of 5 experiments that compared usability evaluation methods (UEMs). Each has had an important influence on HCI thought and practice. Unfortunately, our examination shows that small problems in the way these experiments were designed and conducted call into serious question what we thought we knew regarding the efficacy of various UEMs. If the influence of these experiments were trivial, then such small problems could be safely ignored. Unfortunately, the outcomes of these experiments have been used to justify advice to practitioners regarding their choice of UEMs. Making such choices based on misleading or erroneous claims can be detrimental-compromising the quality and integrity of the evaluation, incurring unnecessary costs, or undermining the practitioner's credibility within the design team. The experimental method is a potent vehicle that can help inform the choice of a UEM as well as help to address other HCI issues. However, to obtain the desired outcomes, close attention must be paid to experimental design.

published version is slightly different Gray, W. D., & Salzman, M. C. (1998). Repairing damaged merchandise: A

Setting the stage, or testing atmosphere, is an important step in preparing for a usability test. This article addresses how to create a good testing atmosphere. We liken this process to that of preparing the stage for a theatre or movie production. In a usability test production, usability professionals serve as directors and set designers, camouflaging the stage (lab equipment), creating a set (an appropriate workspace), recruiting performers (participants represenative of end-users), and execuitng the script (running the test). Usability professionals must attend to each of these issues because they can impact participants' performance, the flow of events, and, ultimately, data quality.