Hybrid hackathons, which combine in-person and online participation, present unique challenges for organizers and participants. Although now widely practiced, research on them remains fragmented, with limited integration between hackathon studies and hybrid collaboration theories. Existing strategies for in-person or online-only events often fail to address hybrid-specific issues, such as managing communication across physical and virtual spaces and ensuring balanced participation. Our work examines hybrid hackathons through the lens of hybrid collaboration theories, focusing on how organizers structure these events and how participants navigate associated challenges. We frame our analysis using established theories of hybrid collaboration, focusing on four key dimensions - synchronicity, physical distribution, dynamic transitions, and technological infrastructure - that shape collaboration in hybrid events. Using an exploratory case study of three hybrid hackathons, involving observations and interviews with organizers and participants, we investigate how these dimensions are implemented and how they shape participant experiences. Our findings show differing organizer approaches to the hybrid dimensions, leading to varied participant experiences in access to resources, communication, and coordination. Implementation styles, favoring in-person, online, or balanced participation, produced disparities in engagement. Organizers often relied on technology to bridge physical and virtual spaces, but overlooked critical aspects such as time-zone management, dynamic transitions, and targeted hybrid team support. Participants, in turn, adapted collaboration tactics in response to gaps in event scaffolding, sometimes enabling collaboration, but at other times creating new challenges, underscoring that considerations for the hybrid format are still not fully integrated into hackathon planning. Learning from our findings, we offer practical recommendations for both organizers and participants to improve planning, participation, and collaboration in hybrid hackathon events.

Accountability promotes responsible behavior, particularly in fulfilling obligations, achieving planned outcomes, and serving the collective good. While systems of accountability are well-structured in corporate environments, their role in open-source communities remains less understood. Existing research has explored requirements-gathering processes and motivations for contributing to open source, focusing on individual benefits. However, there is limited research on how these motivations and processes translate into accountability that guides collective actions to meet the needs of diverse stakeholders. This study examines accountability within the Rust community, particularly during the transition from roadmap-based planning to a goal-oriented process with designated owners within various teams. We provide a rich description, based on meeting notes, project documents and blog posts, as well as 21 interviews with a variety of project participants, to draw lessons about accountability in open source. We found that the Rust roadmapping process facilitated various forms of accountability but lost its effectiveness because (a) unfinished work items on the yearly roadmaps were unavoidable and made the process feel disingenuous; (b) the community culture shifted and facilitated a gradual decline of the roadmapping process; (c) external factors such as the pandemic and layoffs at Mozilla led to changes in the community priorities. Our findings suggest that accountability in open source is an emergent phenomenon that arises from participatory work processes around community engagement and that moral accountability, rather than transactional accountability, is crucial to fostering these processes.

In this retrospective, we reflect on the motivation, findings, and impact of our 2003 paper. We see evidence of influence in increasing the visibility of the topic of global software development (GSD), the use of version data for software engineering research, and using a mixed methods empirical approach. These topics have all continued to grow in importance in software engineering, as revealed by citation counts, but radical changes in business and technical environments have substantially changed how the challenges of GSD manifest. We see evidence of this in the paper’s “life cycle,” as citation counts grew for its first decade, and declined for its second. We consider this a marker of progress, and a very good thing for the field.

Who creates the most innovative open-source software projects? And what fate do these projects tend to have? Building on a long history of research to understand innovation in business and other domains, as well as recent advances towards modeling innovation in scientific research from the science of science field, in this paper we adopt the analogy of innovation as emerging from the novel recombination of existing bits of knowledge. As such, we consider as innovative the software projects that recombine existing software libraries in novel ways, i.e., those built on top of atypical combinations of packages as extracted from import statements. We then report on a large-scale quantitative study of innovation in the Python open-source software ecosystem. Our results show that higher levels of innovativeness are statistically associated with higher GitHub star counts, i.e., novelty begets popularity. At the same time, we find that controlling for project size, the more innovative projects tend to involve smaller teams of contributors, as well as be at higher risk of becoming abandoned in the long term. We conclude that innovation and open source sustainability are closely related and, to some extent, antagonistic.

Past work has sought to design AI ethics interventions--such as checklists or toolkits--to help practitioners design more ethical AI systems. However, other work demonstrates how these interventions may instead serve to limit critique to that addressed within the intervention, while rendering broader concerns illegitimate. In this paper, drawing on work examining how standards enact discursive closure and how power relations affect whether and how people raise critique, we recruit three corporate teams, and one activist team, each with prior context working with one another, to play a game designed to trigger broad discussion around AI ethics. We use this as a point of contrast to trigger reflection on their teams' past discussions, examining factors which may affect their ''license to critique'' in AI ethics discussions. We then report on how particular affordances of this game may influence discussion, and find that the hypothetical context created in the game is unlikely to be a viable mechanism for real world change. We discuss how power dynamics within a group and notions of ''scope'' affect whether people may be willing to raise critique in AI ethics discussions, and discuss our finding that games are unlikely to enable direct changes to products or practice, but may be more likely to allow members to find critically-aligned allies for future collective action.

The diffusion of information about open-source projects is a key factor influencing the adoption of projects and the allocation of developer efforts. Developers learn about new projects, and evaluate their quality and importance by accessing the related information. Social media is an important channel for information diffusion about open-source projects, with previous research suggesting the existence of a social media ecosystem that consists of multiple platforms and collectively supports information diffusion in open source.With different features supporting information diffusion, the same piece of information likely reaches different developer communities on different platforms, which attracts the attention and contribution of different developers and thus influences the success of open-source projects. Despite its importance, few works looked at the identity of the developer community that project-related information reaches on social media platforms and its associated impact on the discussed project.In this work, we track social media discussions on open-source projects on three different platforms: Twitter, HackerNews, and Reddit. We first describe the dynamics of project-related information diffusion across platforms, and we analyze the association between the number of posts on each platform, and the number of developers attracted to the discussed project from different communities. We find that posts about open-source projects first appear on Twitter and HackerNews, then move more towards Reddit. The number of project-related posts on Twitter mostly associate with the attracted developers from communities that are close to the project’s main contributor, while posts on other platforms associate more with the attention from remote communities.

Attracting and retaining new developers is often at the heart of open-source project sustainability and success. Previous research found many intrinsic (or endogenous) project characteristics associated with the attractiveness of projects to new developers, but the impact of factors external to the project itself have largely been overlooked. In this work, we focus on one such external factor, a project's labor pool, which is defined as the set of contributors active in the overall open-source ecosystem that the project could plausibly attempt to recruit from at a given time. How are the size and characteristics of the labor pool associated with a project's attractiveness to new contributors? Through an empirical study of over 516,893 Python projects, we found that the size of the project's labor pool, the technical skill match, and the social connection between the project's labor pool and members of the focal project all significantly influence the number of new developers that the focal project attracts, with the competition between projects with overlapping labor pools also playing a role. Overall, the labor pool factors add considerable explanatory power compared to models with only project-level characteristics.

Abstract Much has changed since the inaugural ICGSE conference in 2006. Tools have improved, awareness of cultural differences is widespread, and developments such as the foregrounding of open source have all enhanced our ability to work across geographic divides. But the pervasive and profound impact of software in the world—especially for societal scale systems such as social media—forces new and deeply challenging responsibilities on both developers and academics. We must find better ways of incorporating ethics into our development practices and pay far more attention to harmful unintended consequences as deployed systems interact with and often disrupt crucial social systems.

Twitter is widely used by software developers. But how effective are tweets at promoting open source projects? How could one use Twitter to increase a project's popularity or attract new contributors? In this paper we report on a mixed-methods empirical study of 44,544 tweets containing links to 2,370 open-source GitHub repositories, looking for evidence of causal effects of these tweets on the projects attracting new GitHub stars and contributors, as well as characterizing the high-impact tweets, the people likely being attracted by them, and how they differ from contributors attracted otherwise. Among others, we find that tweets have a statistically significant and practically sizable effect on obtaining new stars and a small average effect on attracting new contributors. The popularity, content of the tweet, as well as the identity of tweet authors all affect the scale of the attraction effect. In addition, our qualitative analysis suggests that forming an active Twitter community for an open source project plays an important role in attracting new committers via tweets. We also report that developers who are new to GitHub or have a long history of Twitter usage but few tweets posted are most likely to be attracted as contributors to the repositories mentioned by tweets. Our work contributes to the literature on open source sustainability.

Context: Hackathons have become popular events for teams to collaborate on projects and develop software prototypes. Most existing research focuses on activities during an event with limited attention to the evolution of the hackathon code. Objective: We aim to understand the evolution of code used in and created during hackathon events, with a particular focus on the code blobs, specifically, how frequently hackathon teams reuse pre-existing code, how much new code they develop, if that code gets reused afterwards, and what factors affect reuse. Method: We collected information about 22,183 hackathon projects from Devpost and obtained related code blobs, authors, project characteristics, original author, code creation time, language, and size information from World of Code. We tracked the reuse of code blobs by identifying all commits containing blobs created during hackathons and identifying all projects that contain those commits. We also conducted a series of surveys in order to gain a deeper understanding of hackathon code evolution that we sent out to hackathon participants whose code was reused, whose code was not reused, and developers who reused some hackathon code. Result: 9.14% of the code blobs in hackathon repositories and 8% of the lines of code (LOC) are created during hackathons and around a third of the hackathon code gets reused in other projects by both blob count and LOC. The number of associated technologies and the number of participants in hackathons increase reuse probability. Conclusion: The results of our study demonstrates hackathons are not always "one-off" events as the common knowledge dictates and it can serve as a starting point for further studies in this area.

Time-bounded events such as hackathons, data dives, codefests, hack-days, sprints or edit-a-thons have increasingly gained attention from practitioners and researchers. Yet there is a paucity of research on corporate hackathons, which are nearly ubiquitous and present significant organizational, cultural, and managerial challenges. To provide a comprehensive understanding of team processes and broad array of outcomes of corporate hackathons, we conducted a mixed-methods, multiple case study of five teams that participated in a large scale corporate hackathon. Two teams were “pre-existing” teams (PETs) and three were newly-formed “flash” teams (FTs). Our analysis revealed that PETs coordinated almost as if it was just another day at the office while creating innovations within the boundary of their regular work, whereas FTs adopted role-based coordination adapted to the hackathon context while creating innovations beyond the boundary of their regular work. Project sustainability depended on how much effort the team put into finding a home for their projects and whether their project was a good fit with existing products in the organization’s product portfolio. Moreover, hackathon participation had perceived positive effects on participants’ skills, careers, and social networks.

Open source software represents an important form of digital infrastructure as well as a pathway to technical careers for many developers, but women are drastically underrepresented in this setting. Although there is a good body of literature on open source participation, there is very little understanding of the participation trajectories and contribution experiences of women developers, and how they compare to those of men developers, in open source software projects. In order to understand their joining and participation trajectories, we conducted interviews with 23 developers (11 men and 12 women) who became core in an open source project. We identify differences in women and men's motivations for initial contributions and joining processes (e.g. women participating in projects that they have been invited to) and sustained involvement in a project. We also describe unique negative experiences faced by women contributors in this setting in each stage of participation. Our results have implications for diversifying participation in open source software and understanding open source as a pathway to technical careers.

The amount of autonomy in software engineering tools is increasing as developers build increasingly complex systems. We study factors influencing software engineers’ trust in an autonomous tool situated in a high stakes workplace, because research in other contexts shows that too much or too little trust in autonomous tools can have negative consequences. We present the results of a ten week ethnographic case study of engineers collaborating with an autonomous tool to write control software at the National Aeronautics and Space Administration to support high stakes missions. We find that trust in an autonomous software engineering tool in this setting was influenced by four main factors: the tool’s transparency, usability, its social context, and the organization’s associated processes. Our observations lead us to frame trust as a quality the operator places in their collaboration with the automated system, and we outline implications of this framing and other results for researchers studying trust in autonomous systems, designers of software engineering tools, and organizations conducting high stakes work with these tools.

Background: Hackathons have become popular events for teams to collaborate on projects and develop software prototypes. Most existing research focuses on activities during an event with limited attention to the evolution of the code brought to or created during a hackathon. Aim: We aim to understand the evolution of hackathon-related code, specifically, how much hackathon teams rely on pre-existing code or how much new code they develop during a hackathon. Moreover, we aim to understand if and where that code gets reused, and what factors affect reuse. Method: We collected information about 22,183 hackathon projects from Devpost- a hackathon database - and obtained related code (blobs), authors, and project characteristics from the World of Code. We investigated if code blobs in hackathon projects were created before, during, or after an event by identifying the original blob creation date and author, and also checked if the original author was a hackathon project member. We tracked code reuse by first identifying all commits containing blobs created during an event before determining all projects that contain those commits. Result: While only approximately 9.14% of the code blobs are created during hackathons, this amount is still significant considering time and member constraints of such events. Approximately a third of these code blobs get reused in other projects. The number of associated technologies and the number of participants in a project increase reuse probability. Conclusion: Our study demonstrates to what extent pre-existing code is used and new code is created during a hackathon and how much of it is reused elsewhere afterwards. Our findings help to better understand code reuse as a phenomenon and the role of hackathons in this context and can serve as a starting point for further studies in this area.

Unlike in commercial software development, open source software (OSS) projects do not generally have managers with direct control over how developers spend their time, yet for projects with large, diverse sets of contributors, the need exists to focus and steer development in a particular direction in a coordinated way. This is especially important for "infrastructure" projects, such as critical libraries and programming languages that many other people depend on. Some projects have taken the approach of borrowing planning tools that originated in commercial development, despite the fact that these techniques were designed for very different contexts, e.g. strong top-down control and profit motives. Little research has been done to understand how these practices are adapted to a new context. In this paper, we examine the Rust project's use of roadmaps: how has an important OSS infrastructure project adapted an inherently top-down tool to the freewheeling world of OSS? We find that because Rust's roadmaps are built in part by summarizing what motivated developers most prefer to work on, they are in some ways more a description of the motivated labor available than they are a directive that the community move in a particular direction. They allow the community to avoid wasting time on unpopular proposals by revealing that there will be little help in building them, and encouraging work on popular features by making visible the amount of consensus in those features. Roadmaps generate a collective focus without limiting the full scope of what developers work on: roadmap issues consume proportionally more effort than other issues, but constitute a minority of the work done (i.e issues and pull requests made) by both central and peripheral participants. They also create transparency among and beyond the community into what central contributors' plans are, and allow more rational decision-making by providing a way for evidence about community needs to be linked to decision-making.

Work of all kinds increasingly takes place via networked digital environments that support diverse modes of collaboration. Previous work has investigated how collaborative writing takes place in shared and open workspaces. This study investigates how the "fork-and-pull'' model of distributed version management, now widely utilized in software development, supports coordination and public contribution in collaborative writing. We employ a case study approach to understand coordination around written artifacts in an open pull-based environment. Through interviews and archival analysis of two open text projects: a mathematics textbook on homotopy type theory (HoTT) and open source policies by 18F, a branch of federal agency, we reconstruct how text artifacts were created. We find that in both cases an intensive multi-channel communication among a small core preceded an explicit release action in the form of public communication to solicit broader contribution. Our analysis reveals a dichotomy between two modes of collaboration: a perfecting mode of crafting a public written document versus an evolution mode. In perfecting, authors used the fork-and-pull model to manage contributions refining the text to a more correct version. In evolution, authors used the fork-and-pull model to use a written artifact as a template and starting point, customizing it to their relevant scope. Based on our results, we consider how we might design systems and policies to support pull-based coordination around written artifacts and the forms it takes.

Open source software projects often rely on package management systems that help projects discover, incorporate, and maintain dependencies on other packages, maintained by other people. Such systems save a great deal of effort over ad hoc ways of advertising, packaging, and transmitting useful libraries, but coordination among project teams is still needed when one package makes a breaking change affecting other packages. Ecosystems differ in their approaches to breaking changes, and there is no general theory to explain the relationships between features, behavioral norms, ecosystem outcomes, and motivating values. We address this through two empirical studies. In an interview case study, we contrast Eclipse, NPM, and CRAN, demonstrating that these different norms for coordination of breaking changes shift the costs of using and maintaining the software among stakeholders, appropriate to each ecosystem’s mission. In a second study, we combine a survey, repository mining, and document analysis to broaden and systematize these observations across 18 ecosystems. We find that all ecosystems share values such as stability and compatibility, but differ in other values. Ecosystems’ practices often support their espoused values, but in surprisingly diverse ways. The data provides counterevidence against easy generalizations about why ecosystem communities do what they do.

Social media, especially Twitter, has always been a part of the professional lives of software developers, with prior work reporting on a diversity of usage scenarios, including sharing information, staying current, and promoting one's work. However, previous studies of Twitter use by software developers typically lack information about activities of the study subjects (and their outcomes) on other platforms. To enable such future research, in this paper we propose a computational approach to cross-link users across Twitter and GitHub, revealing (at least) 70,427 users active on both. As a preliminary analysis of this dataset, we report on a case study of 786 tweets by open-source developers about GitHub work, combining automatic characterization of tweet authors in terms of their relationship to the GitHub items linked in their tweets with qualitative analysis of the tweet contents. We find that different developer roles tend to have different tweeting behaviors, with repository owners being perhaps the most distinctive group compared to other project contributors and followers. We also note a sizeable group of people who follow others on GitHub and tweet about these people's work, but do not otherwise contribute to those open-source projects. Our results and public dataset open up multiple future research directions.

Developers of open-source software projects tend to collaborate in bursts of activity over a few days at a time, rather than at an even pace. A project might find its productivity suffering if bursts of activity occur when a key person with the right role or right expertise is not available to participate. Open-source projects could benefit from monitoring the way they orchestrate attention among key developers, finding ways to make themselves available to one another when needed. In commercial software development, Sociotechnical Congruence (STC) has been used as a measure to assess whether coordination among developers is sufficient for a given task. However, STC has not previously been successfully applied to open-source projects, in which some industrial assumptions do not apply: management-chosen targets, mandated steady work hours, and top-down task allocation of inputs and targets. In this work we propose an operationalization of STC for open-source software development. We use temporal bursts of activity as a unit of analysis more suited to the natural rhythms of open-source work, as well as open source analogues of other component measures needed for calculating STC. As an illustration, we demonstrate that open-source development on PyPI projects in GitHub is indeed bursty, that activities in the bursts have topical coherence, and we apply our operationalization of STC. We argue that a measure of socio-technical congruence adapted to open source could provide projects with a better way of tracking how effectively they are collaborating when they come together to collaborate.

Time-based events, such as hackathons and codefests, have become a global phenomenon attracting thousands of participants to hundreds of events every year. While research on hackathons has grown considerably, there is still limited insight into what happens to hackathon projects after the event itself has ended. While case studies have provided rich descriptions of hackathons and their aftermath, we add to this literature a large-scale quantitative study of continuation across hackathons in a variety of domains. Our findings indicate that a considerable number of projects get continued after a hackathon has ended. Our results also suggest that short- and long-term continuation are different phenomena. While short-term continuation is associated with technical preparation, number of technologies used in a project and winning a hackathon, long-term continuation is predicated on skill diversity among team members, their technical capabilities in relationship to the technologies and their intention to expand the reach of a project. Moreover, we found intensive short-term activity to be associated with a lower likelihood of long-term project continuation.

Time-bounded events such as hackathons have become a global phenomenon. Scientific communities in particular show growing interest in organizing them to attract newcomers and develop technical artifacts to expand their code base. Current hackathon approaches presume that participants have sufficient expertise to work on projects on their own. They only provide occasional support by domain experts serving as mentors which might not be sufficient for newcomers. Drawing from work on workplace and educational mentoring, we developed and evaluated an approach where each hackathon team is supported by a community member who serves in a mentor role that goes beyond providing occasional support. Evaluating this approach, we found that teams who took ownership of their projects, set achievable goals early while building social ties with their mentor and receiving learning-oriented support reported positive perceptions related to their project and an increased interest in the scientific community that organized the hackathon. Our work thus contributes to our understanding of mentoring in hackathons, an area which has not been extensively studied. It also proposes a feasible approach for scientific communities to attract and integrate newcomers which is crucial for their long-term survival.

Open-source projects do not exist in a vacuum. They benefit from reusing other projects and themselves are being reused by others, creating complex networks of interdependencies, i.e., software ecosystems. Therefore, the sustainability of projects comprising ecosystems may no longer by determined solely by factors internal to the project, but rather by the ecosystem context as well.

Time bounded events such as hackathons, data dives, codefests, hack-days, sprints or edit-a-thons have increasingly gained attention from practitioners and researchers. Existing research, however, has mainly focused on the event itself, while potential outcomes of hackathons have received limited attention. Furthermore, most research around hackathons focuses on collegiate or civic events. Research around hackathons internal to tech companies, which are nearly ubiquitous, and present significant organizational, cultural, and managerial challenges, remains scarce. In this paper we address this gap by presenting findings from a case study of five teams which participated in a large scale corporate hackathon. Most team members voiced their intentions to continue the projects their worked on during the hackathon, but those whose projects did get continued were characterized by meticulous preparation, a focus on executing a shared vision during the hackathon, extended dissemination activities afterwards and a fit to existing product lines. Such teams were led by individuals who perceived the hackathon as an opportunity to bring their idea to life and advance their careers, and who recruited teams who had a strong interest in the idea and in learning the skills necessary to contribute efficiently. Our analysis also revealed that individual team members perceived hackathon participation to have positive effects on their career parts, networks and skill development.

Abstract Transparent environments and social‐coding platforms as GitHub help developers to stay abreast of changes during the development and maintenance phase of a project. Especially, notification feeds can help developers to learn about relevant changes in other projects. Unfortunately, transparent environments can quickly overwhelm developers with too many notifications, such that they lose the important ones in a sea of noise. Complementing existing prioritization and filtering strategies based on binary compatibility and code ownership, we develop an anomaly detection mechanism to identify unusual commits in a repository, which stand out with respect to other changes in the same repository or by the same developer. Among others, we detect exceptionally large commits, commits at unusual times, and commits touching rarely changed file types given the characteristics of a particular repository or developer. We automatically flag unusual commits on GitHub through a browser plug‐in. In an interactive survey with 173 active GitHub users, rating commits in a project of their interest, we found that, although our unusual score is only a weak predictor of whether developers want to be notified about a commit, information about unusual characteristics of a commit changes how developers regard commits. Our anomaly detection mechanism is a building block for scaling transparent environments.

Prior work and perception theory suggests that when exposed to discussion related to a particular piece of crowdsourced text content, readers generally perceive that content to be of lower quality than readers who do not see those comments, and that the effect is stronger if the comments display conflict. This paper presents a controlled experiment with over 1000 participants testing to see if this effect carries over to other documents from the same platform, including those with similar content or by the same author. Although we do generally find that perceived quality of the commented-on document is affected, effects do not carry over to the second item and readers are able to judge the second in isolation from the comment on the first. We confirm a prior finding about the negative effects conflict can have on perceived quality but note that readers report learning more from constructive conflict comments.

Negative experiences in diverse software development teams have the potential to turn off minority participants from future team-based software development activity. We examine the use of brainstorming as one concrete team processes that may be used to improve the satisfaction of minority developers when working in a group. Situating our study in time-intensive hackathon-like environments where engagement of all team members is particularly crucial, we use a combination of survey and interview data to test our propositions. We find that brainstorming strategies are particularly effective for team members who identify as minorities, and support satisfaction with both the process and outcomes of teamwork through different mechanisms.

Team-based learning is a structured, small-group learning method that has been associated with many positive outcomes in traditional classroom settings. However, relatively little research has focused on how to form and support teams within online learning platforms, such as Massive Open Online Courses (MOOCs). A number of challenges arise for team formation in voluntary online classes: students may drop out and leave their team, and even if they do persist with the course, the team may not work together effectively. In this paper, we introduce a team-formation strategy that incorporates a deliberation process, where participants hold discussions in preparation for the collaboration task. First, we present a crowdsourced experiment that compares teams that are formed before or after a community deliberation process. Results demonstrate that teams engaging in a larger community deliberative process prior to team formation exhibit better team performance--as measured by team collaboration product quality--than pre-discussion teams. In a second crowdsourced experiment, we further explore the benefits of community-wide processes by automatically assigning teams based on participants' transactive interaction during deliberation. The results demonstrate advantages in terms of team performance for teams formed based on observed interactions during the community-level deliberation, compared to randomly formed teams. Finally, in a case study, we demonstrate how we successfully adapted the team formation strategy for use in a small MOOC.

Hackathons are events where people who are not normally collocated converge for a few days to write code together. Hackathons, it seems, are everywhere. We know that long- term collocation helps advance technical work and facilitate enduring interpersonal relationships, but can similar benefits come from brief, hackathon-style collocation? How do participants spend their time preparing, working face-to- face, and following through these brief encounters? Do the activities participants select suggest a tradeoff between the social and technical benefits of collocation? We present results from a multiple-case study that suggest the way that hackathon-style collocation advances technical work varies across technical domain, community structure, and expertise of participants. Building social ties, in contrast, seems relatively constant across hackathons. Results from different hackathon team formation strategies suggest a tradeoff between advancing technical work and building social ties. Our findings have implications for technology support that needs to be in place for hackathons and for understanding the role of brief interludes of collocation in loosely-coupled, geographically distributed work.

Change introduces conflict into software ecosystems: breaking changes may ripple through the ecosystem and trigger rework for users of a package, but often developers can invest additional effort or accept opportunity costs to alleviate or delay downstream costs. We performed a multiple case study of three software ecosystems with different tooling and philosophies toward change, Eclipse, R/CRAN, and Node.js/npm, to understand how developers make decisions about change and change-related costs and what practices, tooling, and policies are used. We found that all three ecosystems differ substantially in their practices and expectations toward change and that those differences can be explained largely by different community values in each ecosystem. Our results illustrate that there is a large design space in how to build an ecosystem, its policies and its supporting infrastructure; and there is value in making community values and accepted tradeoffs explicit and transparent in order to resolve conflicts and negotiate change-related costs.

Research aimed at understanding and addressing coordination breakdowns experienced in global software development (GSD) projects at Lucent Technologies took a path from open-ended qualitative exploratory studies to quantitative studies with a tight focus on a key problem – delay – and its causes. Rather than being directly associated with delay, multi-site work items involved more people than comparable same-site work items, and the number of people was a powerful predictor of delay. To counteract this, we developed and deployed tools and practices to support more effective communication and expertise location. After conducting two case studies of open source development, an extreme form of GSD, we realized that many tools and practices could be effective for multi-site work, but none seemed to work under all conditions. To achieve deeper insight, we developed and tested our Socio-Technical Theory of Coordination (STTC) in which the dependencies among engineering decisions are seen as defining a constraint satisfaction problem that the organization can solve in a variety of ways. I conclude by explaining how we applied these ideas to transparent development environments, then sketch important open research questions.

Several intelligent technologies designed to improve navigability in and digestibility of text corpora use topic modeling such as the state-of-the-art Latent Dirichlet Allocation (LDA). This model and variants on it provide lower-dimensional document representations used in visualizations and in computing similarity between documents. This article contributes a method for validating such algorithms against human perceptions of similarity, especially applicable to contexts in which the algorithm is intended to support navigability between similar documents via dynamically generated hyperlinks. Such validation enables researchers to ground their methods in context of intended use instead of relying on assumptions of fit. In addition to the methodology, this article presents the results of an evaluation using a corpus of short documents and the LDA algorithm. We also present some analysis of potential causes of differences between cases in which this model matches human perceptions of similarity more or less well.

Sharing scientific data, software, and instruments is becoming increasingly common as science moves toward large-scale, distributed collaborations. Sharing these resources requires extra work to make them generally useful. Although we know much about the extra work associated with sharing data, we know little about the work associated with sharing contributions to software, even though software is of vital importance to nearly every scientific result. This paper presents a qualitative, interview-based study of the extra work that developers and end users of scientific software undertake. Our findings indicate that they conduct a rich set of extra work around community management, code maintenance, education and training, developer-user interaction, and foreseeing user needs. We identify several conditions under which they are likely to do this work, as well as design principles that can facilitate it. Our results have important implications for future empirical studies as well as funding policy.

The goal of this one-day workshop is to bring together researchers interested in techniques and tools that leverage context information that accumulates around development activities. Developers continuously make use of context to make decisions, coordinate their work, understand the purpose behind their tasks, and understand how their tasks fit with the rest of the project. However, there is little research on defining what context is, how we can model it, and how we can use those models to better support software development at large. This workshop brings together scholars interested in identifying, gathering and modelling context information in software development, as well as discussing its applications.

Application Programming Interfaces (APIs) often define protocols -- restrictions on the order of client calls to API methods. API protocols are common and difficult to use, which has generated tremendous research effort in alternative specification, implementation, and verification techniques. However, little is understood about the barriers programmers face when using these APIs, and therefore the research effort may be misdirected. To understand these barriers better, we perform a two-part qualitative study. First, we study developer forums to identify problems that developers have with protocols. Second, we perform a think-aloud observational study, in which we systematically observe professional programmers struggle with these same problems to get more detail on the nature of their struggles and how they use available resources. In our observations, programmer time was spent primarily on four types of searches of the protocol state space. These observations suggest protocol-targeted tools, languages, and verification techniques will be most effective if they enable programmers to efficiently perform state search.

Community code engagements -- short-term, intensive software development events -- are used by some scientific communities to create new software features and promote community building. But there is as yet little empirical support for their effectiveness. This paper presents a qualitative study of two types of community code engagements: Google Summer of Code (GSoC) and hackathons. We investigated the range of outcomes these engagements produce and the underlying practices that lead to these outcomes. In GSoC, the vision and experience of core members of the community influence project selection, and the intensive mentoring process facilitates creation of strong ties. Most GSoC projects result in stable features. The agenda setting phase of hackathons reveals high priority issues perceived by the community. Social events among the relatively large numbers of participants over brief engagements tend to create weak ties. Most hackathons result in prototypes rather than finished tools. We discuss themes and tradeoffs that suggest directions for future empirical work around designing community code engagements.

Open source software is commonly portrayed as a meritocracy, where decisions are based solely on their technical merit. However, literature on open source suggests a complex social structure underlying the meritocracy. Social work environments such as GitHub make the relationships between users and between users and work artifacts transparent. This transparency enables developers to better use information such as technical value and social connections when making work decisions. We present a study on open source software contribution in GitHub that focuses on the task of evaluating pull requests, which are one of the primary methods for contributing code in GitHub. We analyzed the association of various technical and social measures with the likelihood of contribution acceptance. We found that project managers made use of information signaling both good technical contribution practices for a pull request and the strength of the social connection between the submitter and project manager when evaluating pull requests. Pull requests with many comments were much less likely to be accepted, moderated by the submitter's prior interaction in the project. Well-established projects were more conservative in accepting pull requests. These findings provide evidence that developers use both technical and social information when evaluating potential contributions to open source software projects.

Open source software projects often rely on code contributions from a wide variety of developers to extend the capabilities of their software. Project members evaluate these contributions and often engage in extended discussions to decide whether to integrate changes. These discussions have important implications for project management regarding new contributors and evolution of project requirements and direction. We present a study of how developers in open work environments evaluate and discuss pull requests, a primary method of contribution in GitHub, analyzing a sample of extended discussions around pull requests and interviews with GitHub developers. We found that developers raised issues around contributions over both the appropriateness of the problem that the submitter attempted to solve and the correctness of the implemented solution. Both core project members and third-party stakeholders discussed and sometimes implemented alternative solutions to address these issues. Different stakeholders also influenced the outcome of the evaluation by eliciting support from different communities such as dependent projects or even companies. We also found that evaluation outcomes may be more complex than simply acceptance or rejection. In some cases, although a submitter's contribution was rejected, the core team fulfilled the submitter's technical goals by implementing an alternative solution. We found that the level of a submitter's prior interaction on a project changed how politely developers discussed the contribution and the nature of proposed alternative solutions.

Science policy makers are looking for approaches to increase the extent of collaboration in the production of scientific software, looking to open collaborations in open source software for inspiration. We examine the software ecosystem surrounding BLAST, a key bioinformatics tool, identifying outside improvements and interviewing their authors. We find that academic credit is a powerful motivator for the production and revealing of improvements. Yet surprisingly, we also find that improvements motivated by academic credit are less likely to be integrated than those with other motivations, including financial gain. We argue that this is because integration makes it harder to see who has contributed what and thereby undermines the ability of reputation to function as a reward for collaboration. We consider how open source avoids these issues and conclude with policy approaches to promoting wider collaboration by addressing incentives for integration.

In this paper we describe a qualitative investigation of impression formation in an online distributed software development community with social media functionality. We find that users in this setting seek out additional information about each other to explore the project space, inform future interactions, and understand the potential future value of a new person. They form impressions around other users' expertise based on history of activity across projects, and successful collaborations with key high status projects in the community. These impressions influence their receptivity to strangers' work contributions.

Large-scale collaboration systems often separate their content from the deliberation around how that content was produced. Surfacing this deliberation may engender trust in the content generation process if the deliberation process appears fair, well-reasoned, and thorough. Alternatively, it could encourage doubts about content quality, especially if the process appears messy or biased. In this paper we report the results of an experiment where we found that surfacing deliberation generally led to decreases in perceptions of quality for the article under consideration, especially - but not only - if the discussion revealed conflict. The effect size depends on the type of editors' interactions. Finally, this decrease in actual article quality rating was accompanied by self-reported improved perceptions of the article and Wikipedia overall.

The success of software development projects depends on carefully coordinating the effort of many individuals across the multiple stages of the development process. In software engineering, modularization is the traditional technique intended to reduce the interdependencies among modules that constitute a system. Reducing technical dependencies, the theory argues, results in a reduction of work dependencies between teams developing interdependent modules. Although that research stream has been quite influential, it considers a static view of the problem of coordination in engineering activities. Building on a dynamic view of coordination, we studied the relationship between socio-technical congruence and software quality and development productivity. In order to investigate the generality of our findings, our analyses were performed on two large-scale projects from two companies with distinct characteristics in terms of product and process maturity. Our results revealed that the gaps between coordination requirements and the actual coordination activities carried out by the developers significantly increased software failures. Our analyses also showed that higher levels of congruence are associated with improved development productivity. Finally, our results showed the congruence between dependencies and coordinative actions is critical both in mature development settings as well as in novel and dynamic development contexts.

Social applications on the web let users track and follow the activities of a large number of others regardless of location or affiliation. There is a potential for this transparency to radically improve collaboration and learning in complex knowledge-based activities. Based on a series of in-depth interviews with central and peripheral GitHub users, we examined the value of transparency for large-scale distributed collaborations and communities of practice. We find that people make a surprisingly rich set of social inferences from the networked activity information in GitHub, such as inferring someone else's technical goals and vision when they edit code, or guessing which of several similar projects has the best chance of thriving in the long term. Users combine these inferences into effective strategies for coordinating work, advancing technical skills and managing their reputation.

Software plays an increasingly critical role in science, including data analysis, simulations, and managing workflows. Unlike other technologies supporting science, software can be copied and distributed at essentially no cost, potentially opening the door to unprecedented levels of sharing and collaborative innovation. Yet we do not have a clear picture of how software development for science fits into the day-to-day practice of science, or how well the methods and incentives of its production facilitate realization of this potential. We report the results of a multiple-case study of software development in three fields: high energy physics, structural biology, and microbiology. In each case, we identify a typical publication, and use qualitative methods to explore the production of the software used in the science represented by the publication. We identify several different production systems, characterized primarily by differences in incentive structures. We identify ways in which incentives are matched and mismatched with the needs of the science fields, especially with respect to collaboration.

Feature-driven software development is a novel approach that has grown in popularity over the past decade. Researchers and practitioners alike have argued that numerous benefits could be garnered from adopting a feature-driven development approach. However, those persuasive arguments have not been matched with supporting empirical evidence. Moreover, developing software systems around features involves new technical and organizational elements that could have significant implications for outcomes such as software quality. This paper presents an empirical analysis of a large-scale project that implemented 1195 features in a software system. We examined the impact that technical attributes of product features, attributes of the feature teams and crossfeature interactions have on software integration failures. Our results show that technical factors such as the nature of component dependencies and organizational factors such as the geographic dispersion of the feature teams and the role of the feature owners had complementary impact suggesting their independent and important role in terms of software quality. Furthermore, our analyses revealed that cross-feature interactions, measured as the number of architectural dependencies between two product features, are a major driver of integration failures. The research and practical implications of our results are discussed.

In this paper, we examined the impact of project-level configurational choices of globally distributed software teams on project productivity, quality, and profits. Our analysis used data from 362 projects of four different firms. These projects spanned a wide range of programming languages, application domain, process choices, and development sites spread over 15 countries and 5 continents. Our analysis revealed fundamental tradeoffs in choosing configurational choices that are optimized for productivity, quality, and/or profits. In particular, achieving higher levels of productivity and quality require diametrically opposed configurational choices. In addition, creating imbalances in the expertise and personnel distribution of project teams significantly helps increase profit margins. However, a profit-oriented imbalance could also significantly affect productivity and/or quality outcomes. Analyzing these complex tradeoffs, we provide actionable managerial insights that can help software firms and their clients choose configurations that achieve desired project outcomes in globally distributed software development.

Coordinating activities across groups in systems engineering or product development projects is critical to project success, but substantially more difficult when the work is innovative and dynamic. It is not clear how technology should best support cross-group collaboration on these types of projects. Recent work on coordination in dynamic settings has identified cross-boundary knowledge exchange as a critical mechanism for aligning activities. In order to inform the design of collaboration technology for creative work settings, we examined the nature of cross-group knowledge exchange in an innovative engineering research project developing a lunar rover robot as part of the Google Lunar X-Prize competition. Our study extends the understanding of communication and coordination in creative design work, and contributes to theory on coordination. We introduce four types of cross-team knowledge exchange mechanisms we observed on this project and discuss challenges associated with each. We consider implications for the design of collaboration technology to support cross-team knowledge exchange in dynamic, creative work environments.

MSR has established an impressive presence in the intellectual landscape of software engineering in its seven short years. Insights accumulate as methods continue to mature. Results of practical significance attract increasing numbers of papers and attendees each year. Yet I will argue that MSR is insufficiently ambitious. The community should be seeking enduring scientific results as well as immediate impact. I will argue that progress in three directions will help move MSR toward this possible future. First, while ¿black box¿ prediction models can be quite useful, the community should be driving toward development of a body of theory that sheds light on the underlying phenomena. Second, the community should not be content just to analyze data that happens to exist, but should tackle the problem of defining the data that would be scientifically useful, and follow up by designing and deploying environments that automatically collect it. Finally, the community should push beyond software artifacts, recognizing that many forms of technical design and production work share fundamental characteristics. We should seek to join forces with other research communities that are analyzing behavioral traces in areas such as social networking, blogs, and online communities. As successful as MSR has been, it has only scratched the surface of its potential to forge a science of socio-technical behavior.

It has been widely recognized that coordination amongst the people involved in developing a software intensive system is required for successful completion. Many projects, particularly those with team members separated by geographic distance, have an impeded ability to coordinate amongst some project members, which may cause a mismatch between the need of the project to coordinate and the ability to coordinate resulting in coordination break downs. This paper presents a coordination risk analysis method for determining the significant coordination mismatches before they become an issue and experiences applying this method to geographically distributed projects.

The documentation of API functions typically conveys detailed specifications for the benefit of interested readers. In some cases, however, it also contains usage directives, such as rules or caveats, of which authors of invoking code must be made aware to prevent errors and inefficiencies. There is a risk that these directives may be ldquolostrdquo within the verbose text, or that the text would not be read because there are so many invoked functions. To address these concerns for Java, an Eclipse plug-in named eMoose decorates method invocations whose targets have associated directives. Our goal is to lead readers to investigate further, which we aid by highlighting the tagged directives in the JavaDoc hover. We present a lab study that demonstrates the directive awareness problem in traditional documentation use and the potential benefits of our approach.

Software developers have long known that project success requires a robust understanding of both technical and social linkages. However, research has largely considered these independently. Research on networks of technical artifacts focuses on techniques like code analysis or mining project archives. Social network analysis has been used to capture information about relations among people. Yet, each type of information is often far more useful when combined, as when the ldquogoodnessrdquo of social networks is judged by the patterns of dependencies in the technical artifacts. To bring such information together, we have developed Tesseract, an interactive exploratory environment that utilizes cross-linked displays to visualize the myriad relationships between artifacts, developers, bugs, and communications. We evaluated Tesseract by (1) demonstrating its feasibility with GNOME project data (2) assessing its usability via informal user evaluations, and (3) verifying its suitability for the open source community via semi-structured interviews.

Comprehending an unfamiliar code fragment requires an awareness of explicit usage directives that may be present in the documentation of some invoked functions. Since it is not practical for developers to thoroughly investigate every call, directives may be missed and errors may occur. We previously reported on a tool called eMoose, which highlights calls to methods with associated directives, and on a controlled comparative lab study in which eMoose users were more successful at fixing bugs in given code fragments. In this paper we attempt to shed light on the factors behind these differences with a detailed analysis of videos from the study. We argue that information foraging theory may explain the subjects' reading choices and the impact of our tool. We also suggest ways to structure documentation to increase the prospects of knowledge acquisition.

Prior research has shown that customer-reported software faults are often the result of violated dependencies that are not recognized by developers implementing software. Many types of dependencies and corresponding measures have been proposed to help address this problem. The objective of this research is to compare the relative performance of several of these dependency measures as they relate to customer-reported defects. Our analysis is based on data collected from two projects from two independent companies. Combined, our data set encompasses eight years of development activity involving 154 developers. The principal contribution of this study is the examination of the relative impact that syntactic, logical, and work dependencies have on the failure proneness of a software system. While all dependencies increase the fault proneness, the logical dependencies explained most of the variance in fault proneness, while workflow dependencies had more impact than syntactic dependencies. These results suggest that practices such as rearchitecting, guided by the network structure of logical dependencies, hold promise for reducing defects.

In this paper, we seek to shed light on how communication networks in geographically distributed projects evolve in order to address the limits of the modular design strategy. We collected data from a geographically distributed software development project covering 39 months of activity. Our analysis showed that over time a group of developers emerge as the liaisons between formal teams and geographical locations. In addition to handling the communication and coordination load across teams and locations, those engineers contributed the most to the development effort.

Recent techniques show the benefits of attaching community generated knowledge to artifacts in an information space and presenting it to subsequent readers. We argue that such knowledge may also be relevant to the readers of artifacts which link to this target. Such situations are particularly frequent in software development, where a lack of awareness of critical directives associated with an invoked function can lead to costly errors. We describe how eMoose, a group memory-aid for this domain, addresses these problems by visually "pushing" annotated knowledge from invocation targets into the invoking code. Similar techniques could potentially be applied to other development phases and to other domains.

The identification and management of work dependencies is a fundamental challenge in software development organizations. This paper argues that modularization, the traditional technique intended to reduce interdependencies among components of a system, has serious limitations in the context of software development. We build on the idea of congruence, proposed in our prior work, to examine the relationship between the structure of technical and work dependencies and the impact of dependencies on software development productivity. Our empirical evaluation of the congruence framework showed that when developers' coordination patterns are congruent with their coordination needs, the resolution time of modification requests was significantly reduced. Furthermore, our analysis highlights the importance of identifying the "right" set of technical dependencies that drive the coordination requirements among software developers. Call and data dependencies appear to have far less impact than logical dependencies.

As a globally operating company with about 30,000 software engineers worldwide, Siemens has accumulated a wide variety of experiences in global development. Many individuals and organizations have adjusted their practices to deal with the challenges related to the geographic distribution of the development effort. From a corporate perspective, Siemens has accumulated a rich base of knowledge about global development and how to approach it successfully. The Siemens software initiative - a company-wide improvement program for software development at Siemens - has worked on collecting this widely-distributed knowledge and synthesizing it in a form accessible to the wider software development community. In this paper, the approach as well as key learnings in people and communication-related aspects of collaboration are summarized.

The ability of an organization to successfully carry out its tasks depends on the appropriate combination of organizational structure, processes, and communication and coordination mechanisms. In this paper, we present four case studies that exemplify coordination breakdown problems in global software development. Our analysis showed those problems took place even in the presence of a collection of processes, organizational mechanisms and communication tools established to increases the ability of the teams to perform their tasks. Finally, we discuss possible solutions to overcome the identified problems.

Cyber foraging is the transient and opportunistic use of compute servers bymobile devices. The short market life of such devices makes rapid modification of applications for remote execution an important problem. We describe a solution that combines a "little language" for cyber foraging with an adaptive runtime system. We report results from a user study showing that even novice developers are able to successfully modify large, unfamiliar applications in just a few hours. We also show that the quality of novice-modified and expert-modified applications are comparable in most cases.

Software designers in the object-oriented paradigm can make use of modeling tools and standard notations such as UML. Nevertheless, casual observations from collocated design collaborations suggest that teams tend to use physical mediums to sketch a plethora of informal diagrams in varied representations that often diverge from UML. To better understand such collaborations and support them with tools, we need to understand the origins, roles, uses, and implications of these alternate representations. To this end we conducted observational studies of collaborative design exercises, in which we focused on representation use. Our primary finding is that teams intentionally improviserepresentations and organize design information in responseto ad-hoc needs, which arise from the evolution of the design, and which are difficult to meet with fixed standard notations. This behavior incurs orientation and grounding difficulties for which teams compensate by relying on memory, other communication mediums, and contextual cues. Without this additional information the artifacts are difficult to interpret and have limited documentation potential. Collaborative design tools and processes should therefore focus on preserving contextual information while permitting unconstrained mixing and improvising of notations.

Little is known about how developers think about design during code modification tasks or how experienced developers' design knowledge helps them work more effectively. We performed a lab study in which thirteen developers worked for 3 hours under-standing the design of a 54 KLOC open source application. Par-ticipants had from 0 to 10.5 years of industry experience and were grouped into three "experts" and ten "novices." We observed that participants spent their time seeking, learning, critiquing, explain-ing, proposing, and implementing facts about the code such as "getFoldLevel has effects". These facts served numerous roles, such as suggesting changes, constraining changes, and predicting the amount of additional investigation necessary to make a change. Differences between experts and novices included that the experts explained the root cause of the design problem and made changes to address it, while novice changes addressed only the symptoms. Experts did not read more methods but also did not visit some methods novices wasted time understanding. Experts talked about code in terms of abstractions such as "caching" while novices more often described code statement by statement. Ex-perts were able to implement a change faster than novices. Experts perceived problems novices did not and were able to explain facts novices could not. These findings have interesting implications for future tools.

It has been well documented that there is a correlation between the structure of an architecture and the organization that produces it. More concretely there is a correlation between task interdependencies and coordination among the people or teams realizing these tasks. The amount of coordination needed among teams is related to the nature of these task interdependencies. As the scale and complexity of organization and systems grow it is not uncommon to have factors such as geographic boundaries, organization boundaries, cultural differences, and so forth impede the ability of certain individuals or teams to coordinate effectively. While there is some understanding of the factors that impede the ability of teams to coordinate, the factors that cause task interdependence in software systems is less well understood. The current view is that it is the interactions across module boundaries (assuming a module is assigned as a task or work item to a single team) that cause task interdependence; we have found that this view is not sufficient. In this paper we present three cases where additional architectural mechanisms created task interdependencies that the organizations were unable to accommodate. We go on to discuss the implications of these findings and suggest future research activities.

Task dependencies drive the need to coordinate work activities. We describe a technique for using automatically generated archi-val data to compute coordination requirements, i.e., who must coordinate with whom to get the work done. Analysis of data from a large software development project revealed that coordina-tion requirements were highly volatile, and frequently extended beyond team boundaries. Congruence between coordination re-quirements and coordination activities shortened development time. Developers, particularly the most productive ones, changed their use of electronic communication media over time, achieving higher congruence. We discuss practical implications of our technique for the design of collaborative and awareness tools.

As software specifications for complex systems are practically never entirely complete and consistent, the recipient of the specification needs domain knowledge in order to decide which parts of the system are specified clearly and which parts are specified ambiguously and thus need inquiry to get a more detailed specification. By analyzing the evidence gained in multiple-case study, the necessary components for achieving a mature inquiry culture in distributed software development derived from the practices at Siemens Program and System Engineering (PSE) are identified. These components are presented in three categories-pillars: project communication, requirements communication and inquiry practices

Open source practices and tools have proven to be highly effective for overcoming the many problems of geographically distributed software development. We know relatively little, however, about the range of settings in which they work. In particular, can corporations use the open source development model effectively for software projects inside the corporate domain? Or are these tools and practices incompatible with development environments, management practices, and market-driven schedule and feature decisions typical of a commercial software house? We present a case study of open source software development methodology adopted by a significant commercial software project in the telecommunications domain. We extract a number of lessons learned from the experience, and identify open research questions.

Quantitatively-based risk management can reduce the risks associated with field defects for both software producers and software consumers. In this paper, we report experiences and results from initiating risk-management activities at a large systems development organization. The initiated activities aim to improve product testing (system/integration testing), to improve maintenance resource allocation, and to plan for future process improvements. The experiences we report address practical issues not commonly addressed in research studies: how to select an appropriate modeling method for product testing prioritization and process improvement planning, how to evaluate accuracy of predictions across multiple releases in time, and how to conduct analysis with incomplete information. In addition, we report initial empirical results for two systems with 13 and 15 releases. We present prioritization of configurations to guide product testing, field defect predictions within the first year of deployment to aid maintenance resource allocation, and important predictors across both systems to guide process improvement planning. Our results and experiences are steps towards quantitatively-based risk management.

The Web plays a critical role in hosting Web communities, their content and interactions. A prime example is the open source software (OSS) community, whose members, including software developers and users, interact almost exclusively over the Web, constantly generating, sharing and refining content in the form of software code through active interaction over the Web on code design and bug resolution processes. The Semantic Web is an envisaged extension of the current Web, in which content is given a well defined meaning, through the specification of metadata and ontologies, increasing the utility of the content and enabling information from heterogeneous sources to be integrated. We developed a prototype Semantic Web system for OSS communities, Dhruv. Dhruv provides an enhanced semantic interface to bug resolution messages and recommends related software objects and artifacts. Dhruv uses an integrated model of the OpenACS community, the software, and the Web interactions, which is semi-automatically populated from the existing artifacts of the community.

Computer science is necessary but not sufficient to understand and overcome the problems we face in software engineering. We need to understand not only the properties of the software itself, but also the limitations and competences humans bring to the engineering task. Rather than rely on commonsense notions, we need a deep and nuanced view of human capabilities in order to determine how to enhance them. I discuss what I regard as promising examples of cognitive and organizational theories and propose research directions to develop new ways of representing run-time behavior and ways of thinking about project coordination. I conclude with observations on creating an interdisciplinary culture.

We report on the experiences of Siemens Corporation in nine globally-distributed software development projects. These projects represent a range of collaboration models, from co-development to outsourcing of components to outsourcing the software for an entire project. We report experience and lessons in issues of project management, division of labor, ongoing coordination of technical work, and communication. We include lessons learned, and conclude the paper with suggestions about important open research issues in this area.

Software has jumped "out of the box" - it controls critical systems; it pervades business and commerce; it is embedded in myriad mechanisms; it infuses entertainment, communication, and other activities of everyday life. Designs for these applications are constrained not only by traditional considerations of capability and performance but also by economic, business, market, and policy issues and the context of intended use. The diversity of applications requires adaptability in responding to client needs, and the diversity of clients and contexts requires the ability to discriminate among criteria for success.As a result, software designers must also get out of their boxes: in addition to mastering traditional software development skills, they must understand the contextual issues that discriminate good solutions from merely competent ones. Current software engineering education, however, remains largely "in the box": it neglects the rich fabric of issues that lie between the client's problem and actual software development. At Carnegie Mellon we have addressed this major shortcoming with a course that teaches students to understand both the capabilities required by the client and the constraints imposed by the client's context.This paper presents our view of the engineering character of software engineering, describes the content and organization of our new course, reports on our experience from the first three offerings of our course, and suggests ways to adapt our course for other educational settings.

Open source software systems are critical infrastructure for many applications; however, little has been precisely measured about their quality. Forecasting the field defect-occurrence rate over the entire lifespan of a release before deployment for open source software systems may enable informed decision-making. In this paper, we present an empirical case study often releases of OpenBSD. We use the novel approach of predicting model parameters of software reliability growth models (SRGMs) using metrics-based modeling methods. We consider three SRGMs, seven metrics-based prediction methods, and two different sets of predictors. Our results show that accurate field defect-occurrence rate forecasts are possible for OpenBSD, as measured by the Theil forecasting statistic. We identify the SRGM that produces the most accurate forecasts and subjectively determine the preferred metrics-based prediction method and set of predictors. Our findings are steps towards managing the risks associated with field defects

Defect-occurrence projection is necessary for the development of methods to mitigate the risks of software defect occurrences. In this paper, we examine user-reported software defect-occurrence patterns across twenty-two releases of four widely-deployed, business-critical, production, software systems: a commercial operating system, a commercial middleware system, an open source operating system (OpenBSD), and an open source middleware system (Tomcat). We evaluate the suitability of common defect-occurrence models by first assessing the match between characteristics of widely-deployed production software systems and model structures. We then evaluate how well the models fit real world data. We find that the Weibull model is flexible enough to capture defectoccurrence behavior across a wide range of systems. It provides the best model fit in 16 out of the 22 releases. We then evaluate the ability of the moving averages and the exponential smoothing methods to extrapolate Weibull model parameters using fitted model parameters from historical releases. Our results show that in 50% of our forecasting experiments, these two naïve parameter-extrapolation methods produce projections that are worse than the projection from using the same model parameters as the most recent release. These findings establish the need for further research on parameter-extrapolation methods that take into account variations in characteristics of widely-deployed, production, software systems across multiple releases.

Motivated by evidence that coordination and dependencies among engineering decisions in a software project are key to better understanding and better methods of software creation, we set out to create empirically testable theory to characterize and make predictions about coordination of engineering decisions. We demonstrate that our theory is capable of expressing some of the main ideas about coordination in software engineering, such as Conway's law and the effects of information hiding in modular design. We then used software project data to create measures and test two hypotheses derived from our theory. Our results provide preliminary support for our formulations.

Global software development is rapidly becoming the norm for technology companies. Previous qualitative research suggests that distributed development may increase development cycle time for individual work items (modification requests). We use both data from the source code change management system and survey data to model the extent of delay in a distributed software development organization and explore several possible mechanisms for this delay. One key finding is that distributed work items appear to take about two and one-half times as long to complete as similar items where all the work is colocated. The data strongly suggest a mechanism for the delay, i.e., that distributed work items involve more people than comparable same-site work items, and the number of people involved is strongly related to the calendar time to complete a work item. We replicate the analysis of change data in a different organization with a different product and different sites and confirm our main findings. We also report survey results showing differences between same-site and distributed social networks, testing several hypotheses about characteristics of distributed social networks that may be related to delay. We discuss implications of our findings for practices and collaboration technology that have the potential for dramatically speeding distributed software development.

We report on our experiences of introducing an instant messaging and group chat application into geographically distributed workgroups. We describe a number of issues we encountered, including privacy concerns, individual versus group training, and focusing on teams or individuals. The perception of the tool's utility was a complex issue, depending both on users' views of the importance of informal communication, and their perceptions of the nature of cross-site communication issues. Finally, we conclude with a discussion of critical mass, which is related to the features each user actually uses. More generally, we encountered a dilemma that imposes serious challenges for user-centered design of groupware systems

We report an empirical study of a synchronous messaging application with group-oriented functionality designed to support teams in the workplace. In particular, the tool supports group chat windows that allow members of a group to communicate with text that persists for about a day. We describe the experience of 6 globally-distributed work groups who used the tool over a period of 17 months. An analysis of use shows that the group functionality was used primarily for bursts of synchronous conversations and occasional asynchronous exchanges. The content was primarily focused on work tasks, and negotiating availability, with a smattering of non-work topics and humor. Nearly all groups were remarkably similar in the content of their group chat, although the research group chatted far more frequently than the others. We conclude with suggestions for future research, and a discussion of the place of team-oriented synchronous messaging tools in the workplace.

Finding relevant expertise is a critical need in collaborative software engineering, particularly in geographically distributed developments. We introduce a tool that uses data from change management systems to locate people with desired expertise. It uses a quantification of experience, and presents evidence to validate this quantification as a measure of expertise. The tool enables developers, for example, easily to distinguish someone who has worked only briefly in a particular area of the code from someone who has more extensive experience, and to locate people with broad expertise throughout large parts of the product, such as module or even subsystems. In addition, it allows a user to discover expertise profiles for individuals or organizations. Data from a deployment of the tool in a large software development organization shows that newer, remote sites tend to use the tool for expertise location more frequently. Larger, more established sites used the tool to find expertise profiles for people or organizations. We conclude by describing extensions that provide continuous awareness of ongoing work and an interactive, quantitative resume.

According to its proponents, open source style software development has the capacity to compete successfully, and perhaps in many cases displace, traditional commercial development methods. In order to begin investigating such claims, we examine data from two major open source projects, the Apache web server and the Mozilla browser. By using email archives of source code change history and problem reports we quantify aspects of developer participation, core team size, code ownership, productivity, defect density, and problem resolution intervals for these OSS projects. We develop several hypotheses by comparing the Apache project with several commercial projects. We then test and refine several of these hypotheses, based on an analysis of Mozilla data. We conclude with thoughts about the prospects for high-performance commercial/open source process hybrids.

Global software development is rapidly becoming the norm for technology companies. Previous qualitative research suggests that multi-site development may increase the development cycle time. We use both survey data and data from the source code change management system to model the extent of delay in a multi-site software development organization, and explore several possible mechanisms for this delay. We also measure differences in same-site and cross-site communication patterns, and analyze the relationship of these variables to delay. Our results show that, compared to same-site work, cross-site work takes much longer and requires more people for work of equal size and complexity. We also report a strong relationship between delay in cross-site work and the degree to which remote colleagues are perceived to help out when workloads are heavy. We discuss the implications of our findings for collaboration technology for distributed software development.

Providing information about other users and their activites is a central function of many collaborative applications. The data that provide this "presence awareness" are usually automatically generated and highly dynamic. For example, services such as AOL Instant Messenger allow users to observe the status of one another and to initiate and participate in chat sessions. As such services become more powerful, privacy and security issues regarding access to sensitive user data become critical. Two key software engineering challenges arise in this context:

Collaborations over distance must contend with the loss of the rich, subtle interactions that co-located teams use to coordinate their work. Previous research has suggested that one consequence of this loss is that cross-site work will take longer than comparable single-site work. We use both survey data and data from the change management system to measure the extent of delay in a multi-site software development organization. We also measure site interdependence, differences in same-site and cross-site communication patterns, and analyze the relationship of these variables to delay. Our results show a significant relationship between delay in cross-site work and the degree to which remote colleagues are perceived to help out when workloads are heavy. This result is particularly troubling in light of the finding that workers generally believed they were as helpful to their remote colleagues as to their local colleagues. We discuss implications of our findings for collaboration technology for distributed organizations.

According to its proponents, open source style software development has the capacity to compete successfully, and perhaps in many cases displace, traditional commercial development methods. In order to begin investigating such claims, we examine the development process of a major open source application, the Apache web server. By using email archives of source code change history and problem reports we quantify aspects of developer participation, core team size, code ownership, productivity, defect density, and problem resolution interval for this OSS project. This analysis reveals a unique process, which performs well on important measures. We conclude that hybrid forms of development that borrow the most effective techniques from both the OSS and commercial worlds may lead to high performance software processes.

Geographically distributed development creates new questions about how to coordinate multi-site work. In this paper, we present four methods product development organizations used to coordinate their work: functional areas of expertise, product structure, process steps, and customization. We describe the benefits and difficulties with each model. Finally, we discuss two difficulties that occur irrespective of the model used: consequences of unequal distribution of project mass, and finding expertise.

It is widely acknowledged that coordination of large scale software development is an extremely difficult and persistent problem. Since the structure of the code mirrors the structure of the organization, one might expect that splitting the organization across time zones, cultures, and (natural) languages would make it difficult to assemble the components. This paper presents a case study of what indeed turned out to be the most difficult part of a geographically distributed software project, i.e., integration. Coordination problems were greatly exaggerated across sites, largely because of the breakdown of informal communication channels. The results imply that multi-site development can benefit to some extent from stable plans, processes, and specifications. The inherently unpredictable aspects of projects, however, require communication channels that can be invoked spontaneously, by developers, as needed. These results shed light on the problems and mechanisms underlying the coordination needs of development projects generally, be they co-located or distributed.

A corporate-wide metrics program faces enormous and poorly understood challenges as its implementation spreads out from the centralized planning body across many organizational boundaries into the sites where the data collection actually occurs. This paper presents a case study of the implementation of one corporate-wide program, focusing particularly on the unexpected difficulties of collecting a small number of straightforward metrics. Several mechanisms causing these difficulties are identified, including attenuated communication across organizational boundaries, inertia created by existing data collection systems, and the perceptions, expectations, and fears about how the data will be used. We describe how these factors influence the interpretation of the definitions of the measurements and influence the degree of conformance that is actually achieved. We conclude with lessons learned about both content and mechanisms to help in navigating the tricky waters of organizational dynamics in implementing a company-wide program.

The selection and structuring of requirements engineering data necessitates an empirical prediction about the future utility of the selected data and of the retrieval paths available in the structure. In order to investigate these predictions, the questions asked by software engineers at software requirements and preliminary design meetings are analyzed. It is assumed that the kinds of information elicited by these questions are good indicators of what data would be most useful. Results lend support to systems that focus on the developing artifact itself, and indicate the importance of user scenarios.>