The COVID-19 pandemic necessitated urgent solutions to support governmental programs that were critical to mitigating unacceptable economic, social, and health risks. We conducted a multiple case study, confirmation workshop, and survey of Norwegian public institutions that had defied conventional wisdom about public sector inertia by successfully completing important, urgent, and unexpected development efforts (do-or-die projects). We found consistent patterns credited for their accomplishments, summarized as follows: (1) antecedent capabilities for collaboration, use of expertise, and interdisciplinary problem-solving; (2) characteristics of the project mandates including specificity of goals, importance, deadlines, and tie-in with societal mission; (3) emergent development practices that empowered a core team to make decisions, benefit from organizational support, and integrate effectively with diverse stakeholders; and (4) outcomes that reinforced the practices, such as success, pride in work, transparency, and continuous capability development. We examined our findings through the theoretical lenses of complexity leadership and goal-setting and found support for the relevance of do-or-die projects in both non-urgent and urgent contexts. The findings contribute to understanding how the public sector can perform under pressure by mobilizing latent capabilities and responding with flexibility and focus. Implications are outlined for diverse fields of agile practices and principles, organizational resilience, public sector interaction, complexity leadership, goal-setting, and project studies. Our study suggests that successful do-or-die projects result from a combination of contextual alignment, clear mandates, and empowered execution — offering insight into how such performance may be fostered in and beyond crises. Editor’s note: Open Science material was validated by the Journal of Systems and Software Open Science Board .

The uptake of practices labeled as “agile” is widely discussed in both research and practitioner communities, yet the scope, prevalence, and implementation of such practices remain unclear. A field-level hypothesis suggests that agility manifests in three distinct but related disciplines: software development, product orientation, and continuous delivery. This study tested that proposition through surveys of IT executives in the Norwegian public sector, which represents a substantial share of the country’s national digitalization efforts. Surveys were conducted in 2021 (136 invited, 78 responses) and 2024 (146 invited, 58 responses). The results suggest that agile practices are prevalent and perceived as growing, but systematic adoption patterns are not apparent. No stable factor structures emerged for the hypothesized disciplines, suggesting that agility manifests instead as a multifaceted, emergent phenomenon. Adoption appears to be enabled by team-level commitment and individual initiative, while constraints related to budgeting and governance inhibit it. These findings contribute to theory by questioning assumptions that agile practices are organized into stable domains, method by demonstrating the limitations of current indicators and survey approaches, and practice by underscoring that successful adoption requires enabling organizational conditions rather than prescriptive frameworks. Future research should focus on clarifying what constitutes an agile practice, developing robust indicators of adoption, and comparing implementation patterns across various sectors and countries.

One of the areas within project management that is increasingly resonating in the IT professionals’ minds is benefits management. Anecdotal evidence suggests that recent guidelines for the identification, monitoring/tracking, and realization of benefits (and disbenefits) have not always been applied as intended. Indeed, the intentions in such guidelines may not even be clear, leading practitioners to be confused as to how to implement benefits management in their organizations.

Context: To select the essential, non-negotiable product features is a key skill for stakeholders in software projects. Such selection relies on human judgment, possibly supported by structured prioritization techniques and tools. Goal: Our goal was to investigate whether certain attributes of prioritization techniques affect stakeholders' threshold for judging product features as essential. The four investigated techniques represent four combinations of granularity (low, high) and cognitive support (low, high). Method: To control for robustness and masking effects when investigating in the field, we conducted both an artificial experiment and a field experiment using the same prioritization techniques. In the artificial experiment, 94 subjects in four treatment groups indicated the features (from a list of 16) essential when buying a new cell phone. In the field experiment, 44 domain experts indicated the software product features that were essential for the fulfillment of the project's vision. The effects of granularity and cognitive support on the number of essential ratings were analyzed and compared between the experiments. Result: With lower granularity, significantly more features were rated as essential. The effect was large in the general experiment and extreme in the field experiment. Added cognitive support had medium effect, but worked in opposite directions in the two experiments, and was not statistically significant in the field experiment. Implications: Software projects should avoid taking stakeholders' judgments of essentiality at face value. Practices and tools should be designed to counteract biases and to support the conscious knowledge-based elements of prioritizing.

The skills of software developers are important to the success of software projects. Also, when studying the general effect of a tool or method, it is important to control for individual differences in skill. However, the way skill is assessed is often ad hoc, or based on unvalidated methods. According to established test theory, validated tests of skill should infer skill levels from well-defined performance measures on multiple, small, representative tasks. In this respect, we show how time and quality, which are often analyzed separately, can be combined as task performance and subsequently be aggregated as an approximation of skill. Our results show significant positive correlations between our proposed measures of skill and other variables, such as seniority, lines of code written, and self-evaluated expertise. The method for combining time and quality is a promising first step to measuring programming skill in both industry and research settings.

Numerous factors are involved when deciding when to implement which features in incremental software development. To facilitate a rational and efficient planning process, release planning models make such factors explicit and compute release plan alternatives according to optimization principles. However, experience suggests that industrial use of such models is limited. To investigate the feasibility of model and tool support, we compared input factors assumed by release planning models with factors considered by expert planners. The former factors were cataloged by systematically surveying release planning models, while the latter were elicited through repertory grid interviews in three software organizations. The findings indicate a substantial overlap between the two approaches. However, a detailed analysis reveals that models focus on only select parts of a possibly larger space of relevant planning factors. Three concrete areas of mismatch were identified: (1) continuously evolving requirements and specifications, (2) continuously changing prioritization criteria, and (3) authority-based decision processes. With these results in mind, models, tools and guidelines can be adjusted to address better real-life development processes.

Applying agile methodology in large software development projects introduces many challenges. For example, one may expect that the combination of autonomous teams and the necessity for an overall organizational control structure may lead to conflicts, and one may expect that Agile's informal means of knowledge sharing breaks down as the number of project participants increases. Such issues may in turn compromise the project's productivity. In order to better understand potential threats to productivity in large agile development projects, we conducted repertory grid interviews with 13 project members on their perceptions of threats to productivity. The project was a large software development project consisting of 11 Scrum teams from three different subcontractors. The repertory grid sessions produced 100 issues, which were content analyzed into 10 main problem areas: (1) Restraints on collaboration due to contracts, ownership, and culture, (2) Architectural and technical qualities are given low priority, (3) Conflicts between organizational control and flexibility, (4) Volatile and late requirements from external parties, (5) Lack of a shared vision for the end product, (6) Limited dissemination of functional knowledge, (7) Excessive dependencies within the system, (8) Overloading of key personnel, (9) Difficulties in maintaining well-functioning technical environments, (10) Difficulties in coordinating test and deployment with external parties. Using critical-case reasoning, we claim that projects deploying agile practices in projects with less favorable conditions than those enjoyed in the current project, and that are larger and more complex, are likely to face similar challenges.

Personality tests in various guises are commonly used in recruitment and career counseling industries. Such tests have also been considered as instruments for predicting the job performance of software professionals both individually and in teams. However, research suggests that other human-related factors such as motivation, general mental ability, expertise, and task complexity also affect the performance in general. This paper reports on a study of the impact of the Big Five personality traits on the performance of pair programmers together with the impact of expertise and task complexity. The study involved 196 software professionals in three countries forming 98 pairs. The analysis consisted of a confirmatory part and an exploratory part. The results show that: (1) Our data do not confirm a meta-analysis-based model of the impact of certain personality traits on performance and (2) personality traits, in general, have modest predictive value on pair programming performance compared with expertise, task complexity, and country. We conclude that more effort should be spent on investigating other performance-related predictors such as expertise, and task complexity, as well as other promising predictors, such as programming skill and learning. We also conclude that effort should be spent on elaborating on the effects of personality on various measures of collaboration, which, in turn, may be used to predict and influence performance. Insights into such malleable, rather than static, factors may then be used to improve pair programming performance.

To identify and systematize software practitioners' perceptions of a problem is an important first step toward analyzing and searching for a solution to the problem. This paper reports on an industrial study, in which the repertory grid technique was used to elicit practitioners' perceptions of key challenges in the company's software integration practices. The perceptions of a total of nine practitioners from three organizational groups (Developer, QA Manager, Project Manager) were elicited and analyzed. We found that perceptions differ markedly between groups, but that on some issues, there is a consensus across all groups. Three types of challenges were identified as critical, causes, and easy to handle, namely responsibility, requirements, and knowledge. The elicited information may be used to plan process improvement for integration projects in the company, and may also be used in building a general ontology for integration challenges and their solutions.

The benefits of synergistic collaboration are at the heart of arguments in favor of pair programming. However, empirical studies usually investigate direct effects of various factors on pair programming performance without looking into the details of collaboration. This paper reports from an empirical study that (1) investigated the nature of pair programming collaboration, and (2) subsequently investigated postulated effects of personality on pair programming collaboration. Audio recordings of 44 professional programmer pairs were categorized according to a taxonomy of collaboration. We then measured postulated relationships between the collaboration categories and the personality of the individuals in the pairs. We found evidence that personality generally affects the type of collaboration that occurs in pairs, and that different levels of a given personality trait between two pair members increases the amount of communication-intensive collaboration exhibited by a pair.

Increased realism in software engineering experiments is often promoted as an important means to increase generalizability and industrial relevance. In this context, artificiality, e.g., the use of constructed tasks in place of realistic tasks, is seen as a threat. In this article, we examine the opposite view, that deliberately introduced artificial design elements may increase knowledge gain and enhance both generalizability and relevance. In the first part of the article, we identify and evaluate arguments and examples in favor of, and against, deliberately introducing artificiality into software engineering experiments. In the second part of the article, we summarize a content analysis of articles reporting software engineering experiments published over the ten-year period 1993-2002. The analysis reveals a striving for realism and external validity, but little awareness of for what and when, various degrees of artificiality and realism are appropriate. We conclude that an increased awareness and deliberation in these respects is essential. However, arguments in favor of artificial design elements should not be used to justify studies that are badly designed or that have research questions of low relevance.

Empirically based theories are generally perceived as foundational to science. However, in many disciplines, the nature, role and even the necessity of theories remain matters for debate, particularly in young or practical disciplines such as software engineering. This article reports a systematic review of the explicit use of theory in a comprehensive set of 103 articles reporting experiments, from of a total of 5,453 articles published in major software engineering journals and conferences in the decade 1993-2002. Of the 103 articles, 24 use a total of 40 theories in various ways to explain the cause-effect relationship(s) under investigation. The majority of these use theory in the experimental design to justify research questions and hypotheses, some use theory to provide post hoc explanations of their results, and a few test or modify theory. A third of the theories are proposed by authors of the reviewed articles. The interdisciplinary nature of the theories used is greater than that of research in software engineering in general. We found that theory use and awareness of theoretical issues are present, but that theory-driven research is, as yet, not a major issue in empirical software engineering. Several articles comment explicitly on the lack of relevant theory. We call for an increased awareness of the potential benefits of involving theory, when feasible. To support software engineering researchers who wish to use theory, we show which of the reviewed articles on which topics use which theories for what purposes, as well as details of the theories' characteristics

The classical method for identifying cause-effect relationships is to conduct controlled experiments. This paper reports upon the present state of how controlled experiments in software engineering are conducted and the extent to which relevant information is reported. Among the 5,453 scientific articles published in 12 leading software engineering journals and conferences in the decade from 1993 to 2002, 103 articles (1.9 percent) reported controlled experiments in which individuals or teams performed one or more software engineering tasks. This survey quantitatively characterizes the topics of the experiments and their subjects (number of subjects, students versus professionals, recruitment, and rewards for participation), tasks (type of task, duration, and type and size of application) and environments (location, development tools). Furthermore, the survey reports on how internal and external validity is addressed and the extent to which experiments are replicated. The gathered data reflects the relevance of software engineering experiments to industrial practice and the scientific maturity of software engineering research.