Agile software teams' frequent releases and fast iterations present a growing need for rigorous user experience research methods that are faster, lighter-weight, and more flexible. To this end, we developed the Informal Cognitive Walkthrough (ICW). This agile research methodology grew organically, over the course of three years, while working with a very large agile software development team. ICWs involve conducting one or more Simplified 'Streamlined Cognitive Walkthroughs' (SSCW), followed by one or more Simplified 'Pluralistic Walkthroughs' (SPW). In this paper, we present the ICW and provide a real-world example of its application. Preliminary experiences with the method revealed potential advantages over traditional lab studies, ranging from more quickly uncovering and fixing usability issues, to a stronger collaboration between the disciplines, and to acting as a forcing function in aligning diverse engineers to deliver on a common user goal.

Despite decades of research into how professional programmers debug, only recently has work emerged about how end-user programmers attempt to debug programs. Without this knowledge, we cannot build tools to adequately support their needs. This article reports the results of a detailed qualitative empirical study of end-user programmers' sensemaking about a spreadsheet's correctness. Using our study's data, we derived a sensemaking model for end-user debugging and categorized participants' activities and verbalizations according to this model, allowing us to investigate how participants went about debugging. Among the results are identification of the prevalence of information foraging during end-user debugging, two successful strategies for traversing the sensemaking model, potential ties to gender differences in the literature, sensemaking sequences leading to debugging progress, and sequences tied with troublesome points in the debugging process. The results also reveal new implications for the design of spreadsheet tools to support end-user programmers' sensemaking during debugging.

Although there has been significant research into gender regarding educational and workplace practices, there has been little awareness of gender differences as they pertain to software tools, such as spreadsheet applications, that try to support end users in problem-solving tasks. Although such software tools are intended to be gender agnostic, we believe that closer examination of this premise is warranted. Therefore, in this paper, we report an end-to-end investigation into gender differences with spreadsheet software. Our results showed gender differences in feature usage, feature-related confidence, and tinkering (playful exploration) with features. Then, drawing implications from these results, we designed and implemented features for our spreadsheet prototype that took the gender differences into account. The results of an evaluation on this prototype showed improvements for both males and females, and also decreased gender differences in some outcome measures, such as confidence. These results are encouraging, but also open new questions for investigation. We also discuss how our results compare to generalization studies performed with a variety of other software platforms and populations.

Programming has recently become more common among ordinary end users of computer systems. We believe that these end-user programmers are not just coders but also designers, in that they interlace making design decisions with coding rather than treating them as two separate phases. To better understand and provide support for the programming and design needs of end users, we propose a design theory-based approach to look at end-user programming. Toward this end, we conducted a think-aloud study with ten end users creating a web mashup. By analyzing users' verbal and behavioral data using Schon's reflection-in-action design model and the notion of ideations from creativity literature, we discovered insights into end-user programmers' problem-solving attempts, successes, and obstacles, with accompanying implications for the design of end-user programming environments for mashups. The contribution of our work is three-fold: 1) the methodology of using a design lens to view programming, 2) evidence, through insights gained, of the usefulness of this approach, and 3) the implications themselves.

End-user programmers' code is notoriously buggy. This problem is amplified by the increasing complexity of end users' programs. To help end users catch errors early and reliably, we employ a novel approach for the design of end-user debugging tools: a focus on supporting end users' effective debugging strategies. This paper makes two contributions. We first demonstrate the potential of a strategy-centric approach to tool design by presenting StratCel, an add-in for Excel. Second, we show the benefits of this design approach: participants using StratCel found twice as many bugs as participants using standard Excel, they fixed four times as many bugs, and all this in only a small fraction of the time. Other contributions included: a boost in novices' debugging performance near experienced participants' improved levels, validated design guidelines, a discussion of the generalizability of this approach, and several opportunities for future research.

Although there has been significant research into gender regarding educational and workplace practices, there has been little investigation of gender differences pertaining to problem solving with programming tools and environments. As a result, there is little evidence as to what role gender plays in programming tools---and what little evidence there is has involved mainly novice and end-user programmers in academic studies. This paper therefore investigates how widespread such phenomena are in industrial programming situations, considering three disparate programming populations involving almost 3000 people and three different programming platforms in industry. To accomplish this, we analyzed four industry "legacy" studies from a gender perspective, triangulating results against each other and against a new fifth study, also in industry. We investigated gender differences in software feature usage and in tinkering/exploring software features. Furthermore, we examined how such differences tied to confidence. Our results showed significant gender differences in all three factors---across all populations and platforms.

Can we learn about users' problem-solving strategies by observing their actions? This article introduces a data mining system that extracts complex behavioral patterns from logged user actions to discover users' high-level strategies. Our application domain is an HCI study aimed at revealing users' strategies in an end-user debugging task and understanding how the strategies relate to gender and to success. We cast this problem as a sequential pattern discovery problem, where user strategies are manifested as sequential behavior patterns. Problematically, we found that the patterns discovered by standard data mining algorithms were difficult to interpret and provided limited information about high-level strategies. To help interpret the patterns as strategies, we examined multiple ways of clustering the patterns into meaningful groups. This collectively led to interesting findings about users' behavior in terms of both gender differences and debugging success. These common behavioral patterns were novel HCI findings about differences in males' and females' behavior with software, and were verified by a parallel study with an independent data set on strategies. As a research endeavor into the interpretability issues faced by data mining techniques, our work also highlights important research directions for making data mining more accessible to non-data-mining experts.

Designers' extensive software needs have not been adequately documented in the research literature, and are poorly supported by software. Without appropriate tools to support their needs, designers have difficulty knowing the best way to evolve the look and feel of interactive applications they are designing. In order to inform the design of new tools for interactive application design, we used a grounded theory approach to find out what designers' needs are when designing such applications. This paper reports our findings (20 designer needs) from content analysis of five types of artifacts: surveys, blend discussion list emails, dreamweaver forum entries, flash forum entries, and interviews with ten designers. These 20 needs were then validated in follow-up interviews and focus group sessions. The results of this work revealed trends regarding the importance of each need and show that flow is one of the most important needs.

Little is known about the strategies end-user programmers use in debugging their programs, and even less is known about gender differences that may exist in these strategies. Without this type of information, designers of end-user programming systems cannot know the "target" at which to aim, if they are to support male and female end-user programmers. We present a study investigating this issue. We asked end-user programmers to debug spreadsheets and to describe their debugging strategies. Using mixed methods, we analyzed their strategies and looked for relationships among participants' strategy choices, gender, and debugging success. Our results indicate that males and females debug in quite different ways, that opportunities for improving support for end-user debugging strategies for both genders are abundant, and that tools currently available to end-user debuggers may be especially deficient in supporting debugging strategies used by females.

Recent research has begun to report that female end-user programmers are often more reluctant than males to employ features that are useful for testing and debugging. These earlier findings suggest that, unless such features can be changed in some appropriate way, there are likely to be important gender differences in end-user programmerspsila benefits from these features. In this paper, we compare end-user programmerspsila feature usage in an environment that supports end-user debugging, against an extension of the same environment with two features designed to help ameliorate the effects of low self-efficacy. Our results show ways in which these features affect female versus male enduser programmerspsila self-efficacy, attitudes, usage of testing and debugging features, and performance.

We are currently investigating what types of end user personas (or homogeneous groups in the population) exist and what works for or hinders each in end-user debugging. These personas will be determined using data mining methods such as cluster analysis to see how static (background and self-efficacy), behavioral, and success variables interact for each cluster of users. This research will help provide a better understanding of the needs of end users and the tools that are necessary for supporting both male and females in debugging tasks.

The problem of dependability in end-user programming is an emerging area of interest. Pair collaboration in end-user software development may offer a way for end users to debug their programs more effectively. While pair programming studies - primarily of computer science students and professionals - report positive outcomes in terms of overall program quality, little is known about specific activities that pairs engage in that lead to those outcomes, or of how the previous results may pertain to end-user programmers. In this paper we analyze protocols of end-user pairs debugging spreadsheets. The results suggest that end-user pairs can achieve rich reasoning, effective planning, and systematic evaluation. Furthermore, end-user pairs provide specific types of mutual support that facilitate the accomplishment of their goals

We have been working to uncover gender differences in the ways males and females problem solve in end-user programming situations, and have discovered differences in males' versus females' use of several debugging features. Still, because this line of investigation is new, knowing exactly what to look for is difficult and important information could escape our notice. We therefore decided to bring data mining techniques to bear on our data, with two aims: primarily, to expand what is known about how males versus females make use of end-user debugging features, and secondarily, to find out whether data mining could bring new understanding to this research, given that we had already studied the data manually using qualitative and quantitative methods. The results suggested several new hypotheses in how males versus females go about end-user debugging tasks, the factors that play into their choices, and how their choices are associated with success