Software is crucial to science, but sustaining projects for long term impact is challenging. Scientists and funders look to ''the open source way'' (known as peer production in the organizational literature) as a promising route to sustainability. We studied scientific software projects funded by research grants and which were encouraged by their funder to develop a sustainable community around their open source code. Using interviews and content analysis of online presences, we studied the projects over a seven year period, from receiving grants around 2014 through the typically three years of funding, and for up to four years following the funding. We make four contributions: First, we find that by far the most successful route to peer production was beginning as peer production, a result with clear policy implications. Second, through our taxonomies of organizational forms and of organizational change, we paint a landscape of the variety of forms that scientific software development can take on and transition between providing language for discussion in the literature and among communities of practice. Third, we use these taxonomies to describe multiple routes to sustainable software development. Finally, we discuss the challenges and strategies involved in sustaining scientific software, including choices not to pursue peer production at all.

In recent years, concerns have increased over the lack of contributor diversity in open source software (OSS), despite its status as a paragon of open collaboration. OSS is an important form of digital infrastructure and part of a career path for many developers. While there exists a growing body of literature on cisgender women's under-representation in OSS, the experiences of contributors from other marginalized groups are comparatively absent from the literature. Such is the case for trans contributors, a historically influential group in OSS. In this study, we interviewed 21 trans participants to understand and represent their experiences in the OSS literature. From their experiences, we theorize two related paradoxes of openness in OSS: the paradox of openness and display and the paradox of openness and governance. In an increasingly violent world for trans people, we draw on our theorizing to build recommendations for more inclusive and safer OSS projects for contributors.

We present a comprehensive information extraction system dedicated to software entities in scientific literature. This task combines the complexity of automatic reading of scientific documents (PDF processing, document structuring, styled/rich text, scaling) with challenges specific to mining software entities: high heterogeneity and extreme sparsity of mentions, document-level cross-references, disambiguation of noisy software mentions and poor portability of Machine Learning approaches between highly specialized domains. While NER is a key component to recognize new and unseen software, considering this task as a simple NER application fails to address most of these issues.

Abstract Software contributions to academic research are relatively invisible, especially to the formalized scholarly reputation system based on bibliometrics. In this article, we introduce a gold‐standard dataset of software mentions from the manual annotation of 4,971 academic PDFs in biomedicine and economics. The dataset is intended to be used for automatic extraction of software mentions from PDF format research publications by supervised learning at scale. We provide a description of the dataset and an extended discussion of its creation process, including improved text conversion of academic PDFs. Finally, we reflect on our challenges and lessons learned during the dataset creation, in hope of encouraging more discussion about creating datasets for machine learning use.

Software is increasingly crucial to scholarship, yet the visibility and usefulness of software in the scientific record are in question. Just as with data, the visibility of software in publications is related to incentives to share software in reusable ways, and so promote efficient science. In this article, we examine software in publications through content analysis of a random sample of 90 biology articles. We develop a coding scheme to identify software “mentions” and classify them according to their characteristics and ability to realize the functions of citations. Overall, we find diverse and problematic practices: Only between 31% and 43% of mentions involve formal citations; informal mentions are very common, even in high impact factor journals and across different kinds of software. Software is frequently inaccessible (15%–29% of packages in any form; between 90% and 98% of specific versions; only between 24%–40% provide source code). Cites to publications are particularly poor at providing version information, whereas informal mentions are particularly poor at providing crediting information. We provide recommendations to improve the practice of software citation, highlighting recent nascent efforts. Software plays an increasingly great role in scientific practice; it deserves a clear and useful place in scholarly communication.

Online‐community management is commonly presented as the facilitation of conversation and contributions, especially converting readers to contributors. However, the goal of many discussion communities is to produce a high‐quality knowledge resource, whether to improve external task performance or to increase reputation and site traffic. What do moderation practices look like when the community is focused on the creation of a useable knowledge resource rather than facilitating an inclusive conversation? Under what conditions is this style of moderation likely to be successful? We present a case study from online gaming—Elitist Jerks—in which aggressive moderation is used to transform a conversational medium into a high‐quality knowledge resource, using the strategy of open censorship. We present a content analysis of moderator comments regarding censored messages. Our analysis revealed differences in types of contributor mistakes and the severity of moderator actions: infractions that interfered with both conversation and resource quality were punished harshly, whereas a set of infractions that supported conversation but undermined resource quality were more respectfully removed. We describe a set of conditions under which moderators should intervene in the conversion of conversation to knowledge resource rather than the conversion of lurkers to contributors.

Science policy makers are looking for approaches to increase the extent of collaboration in the production of scientific software, looking to open collaborations in open source software for inspiration. We examine the software ecosystem surrounding BLAST, a key bioinformatics tool, identifying outside improvements and interviewing their authors. We find that academic credit is a powerful motivator for the production and revealing of improvements. Yet surprisingly, we also find that improvements motivated by academic credit are less likely to be integrated than those with other motivations, including financial gain. We argue that this is because integration makes it harder to see who has contributed what and thereby undermines the ability of reputation to function as a reward for collaboration. We consider how open source avoids these issues and conclude with policy approaches to promoting wider collaboration by addressing incentives for integration.

Software plays an increasingly critical role in science, including data analysis, simulations, and managing workflows. Unlike other technologies supporting science, software can be copied and distributed at essentially no cost, potentially opening the door to unprecedented levels of sharing and collaborative innovation. Yet we do not have a clear picture of how software development for science fits into the day-to-day practice of science, or how well the methods and incentives of its production facilitate realization of this potential. We report the results of a multiple-case study of software development in three fields: high energy physics, structural biology, and microbiology. In each case, we identify a typical publication, and use qualitative methods to explore the production of the software used in the science represented by the publication. We identify several different production systems, characterized primarily by differences in incentive structures. We identify ways in which incentives are matched and mismatched with the needs of the science fields, especially with respect to collaboration.