In recent years, research into gender differences has established that individual differences in how people problem-solve often cluster by gender. Research also shows that these differences have direct implications for software that aims to support users' problem-solving activities, and that much of this software is more supportive of problem-solving processes favored (statistically) more by males than by females. However, there is almost no work considering how software practitioners—such as User Experience (UX) professionals or software developers—can find gender-inclusiveness issues like these in their software. To address this gap, we devised the GenderMag method for evaluating problem-solving software from a gender-inclusiveness perspective. The method includes a set of faceted personas that bring five facets of gender difference research to life, and embeds use of the personas into a concrete process through a gender-specialized Cognitive Walkthrough. Our empirical results show that a variety of practitioners who design software—without needing any background in gender research—were able to use the GenderMag method to find gender-inclusiveness issues in problem-solving software. Our results also show that the issues the practitioners found were real and fixable. This work is the first systematic method to find gender-inclusiveness issues in software, so that practitioners can design and produce problem-solving software that is more usable by everyone.

Developers performing maintenance activities must balance their efforts to learn the code vs. their efforts to actually change it. This balancing act is consistent with the “production bias” that, according to Carroll's minimalist learning theory, generally affects software users during everyday tasks. This suggests that developers' focus on efficiency should have marked effects on how they forage for the information they think they need to fix bugs. To investigate how developers balance fixing versus learning during debugging, we conducted the first empirical investigation of the interplay between production bias and information foraging. Our theory-based study involved 11 participants: half tasked with fixing a bug, and half tasked with learning enough to help someone else fix it. Despite the subtlety of difference between their tasks, participants foraged remarkably differently-making foraging decisions from different types of “patches,” with different types of information, and succeeding with different foraging tactics.

Many systems are designed to help novices who want to learn programming, but few support those who are not interested in learning (more) programming. This paper targets the subset of end-user programmers (EUPs) in this category. We present a set of principles on how to help EUPs like this learn just a little when they need to overcome a barrier. We then instantiate the principles in a prototype and empirically investigate the principles in two studies: a formative think-aloud study and a pair of summer camps attended by 42 teens. Among the surprising results were the complementary roles of implicitly actionable hints versus explicitly actionable hints, and the importance of both context-free and context-sensitive availability. Under these principles, the camp participants required significantly less in-person help than in a previous camp to learn the same amount of material in the same amount of time.

Although there are many systems designed to engage people in programming, few explicitly teach the subject, expecting learners to acquire the necessary skills on their own as they create programs from scratch. We present a principled approach to teach programming using a debugging game called Gidget, which was created using a unique set of seven design principles. A total of 44 teens played it via a lab study and two summer camps. Principle by principle, the results revealed strengths, problems, and open questions for the seven principles. Taken together, the results were very encouraging: learners were able to program with conditionals, loops, and other programming concepts after using the game for just 5 hours.

One of the least studied areas of Information Foraging Theory is diet: the information foragers choose to seek. For example, do foragers choose solely based on cost, or do they stubbornly pursue certain diets regardless of cost? Do their debugging strategies vary with their diets? To investigate "what" and "how" questions like these for the domain of software debugging, we qualitatively analyzed 9 professional developers' foraging goals, goal patterns, and strategies. Participants spent 50% of their time foraging. Of their foraging, 58% fell into distinct dietary patterns - mostly in patterns not previously discussed in the literature. In general, programmers' foraging strategies leaned more heavily toward enrichment than we expected, but different strategies aligned with different goal types. These and our other findings help fill the gap as to what programmers' dietary goals are and how their strategies relate to those goals.

Assessments have been shown to have positive effects on learning in compulsory educational settings. However, much less is known about their effects in discretionary learning settings, especially in computing education and educational games. We hypothesized that adding assessments to an educational computing game would provide extra opportunities for players to practice and correct misconceptions, thereby affecting their performance on subsequent levels and their motivation to continue playing. To test this, we designed a game called Gidget, in which players help a robot find and fix defects in programs that follow a mastery learning paradigm. Across two studies, we manipulated the inclusion of multiple choice and self-explanation assessment levels in the game, measuring their impact on engagement and level completion speed. In our first study, we found that including assessments caused learners to voluntarily play longer and complete more levels, suggesting increased engagement; in our second study, we found that including assessments caused learners to complete levels faster, suggesting increased understanding. These findings suggest that including assessments in a discretionary computing education game may be a key design strategy for improving informal learning of computing concepts.

Software projects involve diverse roles and artifacts that have dependencies to requirements. Project team members in different roles need to coordinate but their coordination is affected by the availability of domain knowledge, which is distributed among different project members, and organizational structures that control cross-functional communication. Our study examines how information flowed between different roles in two software projects that had contrasting distributions of domain knowledge and different communication structures. Using observations, interviews, and surveys, we examined how diverse roles working on requirements and their related artifacts coordinated along task dependencies. We found that communication only partially matched task dependencies and that team members that are boundary spanners have extensive domain knowledge and hold key positions in the control structure. These findings have implications on how organizational structures interfere with task assignments and influence communication in the project, suggesting how practitioners can adjust team configuration and communication structures.

End-user programmers often get stuck because they do not know how to overcome their barriers. We have previously presented an approach called the Idea Garden, which makes minimalist, on-demand problem-solving support available to end-user programmers in trouble. Its goal is to encourage end users to help themselves learn how to overcome programming difficulties as they encounter them. In this paper, we investigate whether the Idea Garden approach helps end-user programmers problem-solve their programs on their own. We ran a statistical experiment with 123 end-user programmers. The experiment's results showed that, even when the Idea Garden was no longer available, participants with little knowledge of programming who previously used the Idea Garden were able to produce higher-quality programs than those who had not used the Idea Garden.

Research is emerging on how end users can correct mistakes their intelligent agents make, but before users can correctly “debug” an intelligent agent, they need some degree of understanding of how it works. In this paper we consider ways intelligent agents should explain themselves to end users, especially focusing on how the soundness and completeness of the explanations impacts the fidelity of end users' mental models. Our findings suggest that completeness is more important than soundness: increasing completeness via certain information types helped participants' mental models and, surprisingly, their perception of the cost/benefit tradeoff of attending to the explanations. We also found that oversimplification, as per many commercial agents, can be a problem: when soundness was very low, participants experienced more mental demand and lost trust in the explanations, thereby reducing the likelihood that users will pay attention to such explanations at all.

Theories of human behavior are an important but largely untapped resource for software engineering research. They facilitate understanding of human developers’ needs and activities, and thus can serve as a valuable resource to researchers designing software engineering tools. Furthermore, theories abstract beyond specific methods and tools to fundamental principles that can be applied to new situations. Toward filling this gap, we investigate the applicability and utility of Information Foraging Theory (IFT) for understanding information-intensive software engineering tasks, drawing upon literature in three areas: debugging, refactoring, and reuse. In particular, we focus on software engineering tools that aim to support information-intensive activities, that is, activities in which developers spend time seeking information. Regarding applicability, we consider whether and how the mathematical equations within IFT can be used to explain why certain existing tools have proven empirically successful at helping software engineers. Regarding utility, we applied an IFT perspective to identify recurring design patterns in these successful tools, and consider what opportunities for future research are revealed by our IFT perspective.

What does a user need to know to productively work with an intelligent agent? Intelligent agents and recommender systems are gaining widespread use, potentially creating a need for end users to understand how these systems operate in order to fix their agent's personalized behavior. This paper explores the effects of mental model soundness on such personalization by providing structural knowledge of a music recommender system in an empirical study. Our findings show that participants were able to quickly build sound mental models of the recommender system's reasoning, and that participants who most improved their mental models during the study were significantly more likely to make the recommender operate to their satisfaction. These results suggest that by helping end users understand a system's reasoning, intelligent agents may elicit more and better feedback, thus more closely aligning their output with each user's intentions.

Building tools to help software developers communicate effectively requires a deep understanding of their communication dynamics. To date we do not have good comprehension of why developers talk to each other as a result of some events in the life of their projects, and not of others. This lack of knowledge makes it difficult to design useful communication models and support systems.

How can end-user programming environments better help their users overcome programming barriers? We have been investigating an approach called Idea Gardening, which addresses this problem by helping end users to help themselves overcome barriers in the context of “doing”. In this paper, we report on a qualitative empirical study of how effectively an Idea Garden prototype helped end users overcome programming barriers in the CoScripter environment, and the extent to which participants learned after interacting with our features. Our results showed that 9 out of 10 participants who encountered barriers and then used the Idea Garden, overcame their barriers. Further, all 9 went on to demonstrate evidence of having learned the programming concepts, patterns, and strategies relevant to overcoming these barriers.

Despite decades of research into how professional programmers debug, only recently has work emerged about how end-user programmers attempt to debug programs. Without this knowledge, we cannot build tools to adequately support their needs. This article reports the results of a detailed qualitative empirical study of end-user programmers' sensemaking about a spreadsheet's correctness. Using our study's data, we derived a sensemaking model for end-user debugging and categorized participants' activities and verbalizations according to this model, allowing us to investigate how participants went about debugging. Among the results are identification of the prevalence of information foraging during end-user debugging, two successful strategies for traversing the sensemaking model, potential ties to gender differences in the literature, sensemaking sequences leading to debugging progress, and sequences tied with troublesome points in the debugging process. The results also reveal new implications for the design of spreadsheet tools to support end-user programmers' sensemaking during debugging.

Sharing knowledge in a timely fashion is important in distributed software development. However, because experts are difficult to locate, developers tend to broadcast information to find the right people, which leads to overload and to communication breakdowns. We study the context in which experts are included in an email discussion so that team members can identify experts sooner. In this paper, we conduct a case study examining why people emerge in discussions by examining email within a distributed team. We find that people emerge in the following four situations: when a crisis occurs, when they respond to explicit requests, when they are forwarded in announcements, and when discussants follow up on a previous event such as a meeting. We observe that emergent people respond not only to situations where developers are seeking expertise, but also to execute routine tasks. Our findings have implications for expertise seeking and knowledge management processes.

Distributed Software Development (DSD) is an emerging research area in software engineering. Several conducted research studies identified similar communication problems among DSD teams and tried to solve them. In this paper we present patterns that we have identified while surveying state of the art research studies. The patterns can help to organize DSD teams better in order to enhance their communication. We also highlight some potential future research challenges.

Socio-technical congruence is an approach that measures coordination by examining the alignment between the technical dependencies and the social coordination in the project. We conduct a case study of coordination in the IBM Rational Team Concert project, which consists of 151 developers over seven geographically distributed sites, and expect that high congruence leads to a high probability of successful builds. We examine this relationship by applying two congruence measurements: an unweighted congruence measure from previous literature, and a weighted measure that overcomes limitations of the existing measure. We discover that there is a relationship between socio-technical congruence and build success probability, but only for certain build types, and observe that in some situations, higher congruence actually leads to lower build success rates. We also observe that a large proportion of zero-congruence builds are successful, and that socio-technical gaps in successful builds are larger than gaps in failed builds. Analysis of the social and technical aspects in IBM Rational Team Concert allows us to discuss the effects of congruence on build success. Our findings provide implications with respect to the limits of applicability of socio-technical congruence and suggest further improvements of socio-technical congruence to study coordination.

Global software teams face challenges when collaborating over long distances, such as communicating changes in the project. During a four-month case study at IBM Ottawa Software Lab we observed the collaboration patterns of a multi-site development project team. In this period, we inspected project documentation, interviewed team leaders, attended project meetings, and spoke with developers to identify problems originated by the lack of awareness of changes related to the implementation of work items. Our observations show (1) that organizational culture has an effect on how developers are made aware; (2) that communication-based social networks revolving around particular work items are dynamic throughout development, and therefore awareness needs to be maintained in infrastructures of work; and (3) that information overload and communication breakdowns contributed to the generation of a broken integration build. We discuss these breakdowns in communication and implications for the design of collaboration tools that could mitigate these problems.

Because of intense collaborative needs, requirements engineering is a challenge in global software development. How do distributed teams manage the development of requirements in environments that require significant cross-site collaboration and coordination? In this paper, we report research that used social network analysis to explore collaboration and awareness among team members during requirements management in an industrial distributed software team. Using the lens of a requirements-centred social network to group team members who work on a particular requirement, we collected data to characterize requirements-centric collaborations in a project, and to examine aspects of awareness of requirements changes within these networks. Our findings indicate organic patterns of collaboration involving considerable cross-site interaction, in which communication of changes was the most predominant reason for interaction. Although we did not find evidence that distance affects developers' awareness of remote team members who work on the same requirements, distance affected how accessible the remote colleagues were. We discuss implications for knowledge sharing and coordination of work on a requirement in distributed teams, and propose directions for the design of collaboration tools that support awareness in distributed requirements management.

Project collaborators in a software development project need to stay aware not only of changes to requirements and other artifacts, but also of each other's current work. The set of team members working on a requirement is dynamic, and team members who were not assigned to the requirement in the plan may be involved. If this requirement changes, those team members who are dependent on that requirement must be notified quickly before they do outdated work. However, project plans often do not provide an easy method of listing all of the emergent team members who should be notified of changes to a requirement. We propose a requirements-dependency diagram that displays interdependent requirements and team members who are assigned to these interdependent requirements. The visualization highlights prominent collaborators, lists each collaborator and each requirement only once, marks emergent collaborators, and is simple and clutter-free. By viewing this diagram, collaborators will know who to contact to notify others of changes to requirements, and can contact experts working on interrelated requirements.