In this work, we examine the value that immersive technologies can bring to retailing through the retail practices they facilitate. To that end, a literature review is conducted resulting in the documentation of 28 augmented reality (AR) and virtual reality (VR) applications from 38 publications. After analyzing the applications’ functionality and use in retail, the following AR/VR-enabled retail practices emerged: branding and marketing; sales channel; after-sale customer service; virtual try-on; customer-as-designer; virtual training; and workflow management. A principal observation from the analysis is that current AR/VR applications are used mainly for customer-related innovation, with “branding and marketing” being a dominant practice. Simultaneously, some practices are available to serve organization-related and support-related innovation. Finally, it was observed that AR is a popular technology in the retail environment and of high practical value, being an ideal fit for the pu rchase journey and workflow management. However, VR is more difficult to implement in retail, as it can be more expensive and complicated to integrate with the sales channel. However, it can create strong emotional engagement due to high immersion and act as a useful tool for branding and training. Therefore, these two technologies in retail and their strengths can supplement each other, thereby creating promising innovation strategies when combined.

Online social platforms, such as blogs, discussion forums and social networking sites, are increasingly explored as venues for user-centred evaluations; in particular, for design feedback from users. We present a multi-case study providing needed knowledge on such evaluations. Our findings are based on analyses of the design feedback and post-factum data collections with development team representatives and users. The development team representatives reported as key benefits that the evaluations provided insight into users’ needs and competencies, input into ongoing design discussions and support for idea generation in the development team, but found the lack of direct contact and control with the users to be an important limitation. The users appreciated the opportunity to contribute to the design process, but the majority reported not to build on each other's contributions. Involving a relatively large number of users was found to be beneficial for generating constructive design suggestions. Practical implications and future research challenges are suggested.

The skills of software developers are important to the success of software projects. Also, when studying the general effect of a tool or method, it is important to control for individual differences in skill. However, the way skill is assessed is often ad hoc, or based on unvalidated methods. According to established test theory, validated tests of skill should infer skill levels from well-defined performance measures on multiple, small, representative tasks. In this respect, we show how time and quality, which are often analyzed separately, can be combined as task performance and subsequently be aggregated as an approximation of skill. Our results show significant positive correlations between our proposed measures of skill and other variables, such as seniority, lines of code written, and self-evaluated expertise. The method for combining time and quality is a promising first step to measuring programming skill in both industry and research settings.

This paper reports an explorative experimental comparison of (i) an experience-sampling method called feedback collection and (ii) the think-aloud methods with respect to their usefulness in studies on program comprehension. Think-aloud methods are widely used in studies of cognitive processes, including program comprehension. Alternatively, as in the feedback-collection method (FCM), cognitive processes can be traced by collecting written feedback from the subjects at regular intervals. We compare FCM with concurrent think-aloud (CTA) and retrospective think-aloud (RTA) regarding type and usefulness of the collected information, costs related to analysis of the collected information and effects of the data collection methods on the subjects' performance. FCM allowed us to identify a greater number of comprehension problems that prevented progress or caused significant delay (FCM: 30 problems; CTA: 5; RTA: 15). It was less precise in identifying strategies for comprehension than CTA (92% correctness for FCM; 100% for CTA). FCM was less expensive in analysis (transcription and coding) than the other two methods (FCM: 0.7 h of analysis per protocol; CTA: 31 h; RTA: 7.9 h). The results indicate that all three methods of data collection were intrusive and affected the performance of the subjects with respect to time and correctness (small to medium effect size). This research confirms that FCM can be used beneficially in studies that trace the cognitive processes involved in, and identify problems related to, the comprehension of software applications. On the basis of our experience, we recommend that FCM be used in studies that have a large number of subjects and as a complement to other methods for tracing cognitive processes, such as user log files. We recommend a design with two groups (verbalisation and silent control) and a pretest task to be used in studies with FCM or CTA that focus on performances.

Abstract Analysis of low‐level usage data collected in empirical studies of user interaction is well known as a demanding task. Existing techniques for data collection and analysis are either application specific or data‐driven. This paper presents a workspace for data cleaning, transformation and analysis of low‐level usage data that we have developed and reports our experience with it. By its five‐level architecture, the workspace makes a distinction between more general data that typically can be used in initial data analysis and the data answering a specific research question. The workspace was used in four studies and in total 6.5M user actions were collected from 238 participants. The collected data have been proven to be useful for: (i) validating solution times, (ii) validating process conformances, (iii) exploratory studies on program comprehension for understanding use of classes and documents and (iv) testing hypotheses on keystroke latencies. We have found workspace creation to be demanding in time. Particularly demanding were determining the context of actions and dealing with deficiencies. However, once these processes were understood, it was easy to reuse the workspace for different experiments and to extend it to answer new research questions. Based on our experience, we give a set of guidelines that might help in setting up studies, collecting and preparing data. We recommend that designers of data collection instruments add context to each action. Furthermore, we recommend rapid iterations starting early in the process of data preparation and analysis, and covering both general and specific data. Copyright © 2009 John Wiley & Sons, Ltd.

The classical method for identifying cause-effect relationships is to conduct controlled experiments. This paper reports upon the present state of how controlled experiments in software engineering are conducted and the extent to which relevant information is reported. Among the 5,453 scientific articles published in 12 leading software engineering journals and conferences in the decade from 1993 to 2002, 103 articles (1.9 percent) reported controlled experiments in which individuals or teams performed one or more software engineering tasks. This survey quantitatively characterizes the topics of the experiments and their subjects (number of subjects, students versus professionals, recruitment, and rewards for participation), tasks (type of task, duration, and type and size of application) and environments (location, development tools). Furthermore, the survey reports on how internal and external validity is addressed and the extent to which experiments are replicated. The gathered data reflects the relevance of software engineering experiments to industrial practice and the scientific maturity of software engineering research.