Can we learn about users' problem-solving strategies by observing their actions? This article introduces a data mining system that extracts complex behavioral patterns from logged user actions to discover users' high-level strategies. Our application domain is an HCI study aimed at revealing users' strategies in an end-user debugging task and understanding how the strategies relate to gender and to success. We cast this problem as a sequential pattern discovery problem, where user strategies are manifested as sequential behavior patterns. Problematically, we found that the patterns discovered by standard data mining algorithms were difficult to interpret and provided limited information about high-level strategies. To help interpret the patterns as strategies, we examined multiple ways of clustering the patterns into meaningful groups. This collectively led to interesting findings about users' behavior in terms of both gender differences and debugging success. These common behavioral patterns were novel HCI findings about differences in males' and females' behavior with software, and were verified by a parallel study with an independent data set on strategies. As a research endeavor into the interpretability issues faced by data mining techniques, our work also highlights important research directions for making data mining more accessible to non-data-mining experts.

This paper focuses on mining human strategies by observing their actions. Our application domain is an HCI study aimed at discovering general strategies used by software users and understanding how such strategies relate to gender and success. We cast this as a sequential pattern discovery problem, where user strategies are manifested as sequential patterns. Problematically, we found that the patterns discovered by standard algorithms were difficult to interpret and provided limited information about high-level strategies. To help interpret the patterns and extract general strategies, we examined multiple ways of clustering the patterns into meaningful groups, which collectively led to interesting findings about user behavior both in terms of gender differences and problem-solving success. As a real-world application of data mining techniques, our work led to the discovery of new strategic patterns that are linked to user success and had not been revealed in more than nine years of manual empirical work. As a case study, our work highlights important research directions for making data mining more accessible to non-experts.

We have been working to uncover gender differences in the ways males and females problem solve in end-user programming situations, and have discovered differences in males' versus females' use of several debugging features. Still, because this line of investigation is new, knowing exactly what to look for is difficult and important information could escape our notice. We therefore decided to bring data mining techniques to bear on our data, with two aims: primarily, to expand what is known about how males versus females make use of end-user debugging features, and secondarily, to find out whether data mining could bring new understanding to this research, given that we had already studied the data manually using qualitative and quantitative methods. The results suggested several new hypotheses in how males versus females go about end-user debugging tasks, the factors that play into their choices, and how their choices are associated with success