Abstract Scrum is widely adopted in software project management due to its adaptability and collaborative nature. The recent emergence of Large Language Models (LLMs) has created new opportunities to support knowledge-intensive Scrum practices. However, existing research has largely focused on technical activities such as coding and testing, with limited evidence on the use of LLMs in management-related Scrum activities. In this study, we investigate the use of LLMs in Scrum management activities through a survey of 70 Brazilian professionals. Among them, 49 actively use Scrum, and 33 reported using LLM-based assistants in their Scrum practices. The results indicate a high level of proficiency and frequent use of LLMs, with 85% of respondents reporting intermediate or advanced proficiency and 52% using them daily. LLM use concentrates on exploring Scrum practices, with artifacts and events receiving targeted yet uneven support, whereas broader management tasks appear to be adopted more cautiously. The main benefits include increased productivity (78%) and reduced manual effort (75%). However, several critical risks remain, as respondents report ‘almost correct’ outputs (81%), confidentiality concerns (63%), and hallucinations during use (59%). This work provides one of the first empirical characterizations of LLM use in Scrum management, identifying current practices, quantifying benefits and risks, and outlining directions for responsible adoption and integration in Agile environments.

ABSTRACT The COVID-19 pandemic has permanently altered workplace structures, making remote work a widespread practice. While many employees advocate for flexibility, many employers reconsider their attitude toward remote work and opt for structured return-to-office mandates. Media headlines repeatedly emphasize that the corporate world returns to full-time office work. This study examines how companies in software-intensive industry regulate work location, whether corporate policies have evolved in the last five years, and, if so, how, and why. We collected data on remote work regulation from corporate HR and management representatives from 68 companies that vary in size, location, and preferred work modality. Our findings reveal that although many companies prioritize office-oriented work (50%), most companies in our sample permit hybrid work (84%) and only four companies are returning to full-time office work. Remote work regulation does not reveal any particular new “best practice” as policies differ greatly; however, the single most popular arrangement was the three in-office days per week. More than half of the companies (53%) encourage or mandate office attendance centrally, with additional 18% having decentralized mandates. Over a quarter (28%) have changed regulations gradually increasing the mandatory office presence or implementing differentiated conditions. Our key recommendation for office-oriented companies is to consider trust-based recommendations as an alternative to centralized office presence mandates, while for companies oriented toward remote working, we warn about the points of no (or hard) return. Finally, the current state of policies is clearly not final, as companies continue to experiment and adjust their work regulation

[Context] Technical debt (TD) in machine learning (ML) systems, much like its counterpart in software engineering (SE), holds the potential to lead to future rework, posing risks to productivity, quality, and team morale. Despite growing attention to TD in SE, the understanding of ML-specific code-related TD remains underexplored. [Objective] This paper aims to identify and discuss the relevance of code-related issues that lead to TD in ML code throughout the ML workflow. [Method] The study first compiled a list of 34 potential issues contributing to TD in ML code by examining the phases of the ML workflow, their typical associated activities, and problem types. This list was refined through two focus group sessions involving nine experienced ML professionals, where each issue was assessed based on its occurrence contributing to TD in ML code and its relevance. [Results] The list of issues contributing to TD in the source code of ML systems was refined from 34 to 30, with 24 of these issues considered highly relevant. The data pre-processing phase was the most critical, with 14 issues considered highly relevant. Shortcuts in code related to typical pre-processing tasks (e.g., handling missing values, outliers, inconsistencies, scaling, rebalancing, and feature selection) often result in “patch fixes” rather than sustainable solutions, leading to the accumulation of TD and increasing maintenance costs. Relevant issues were also found in the data collection, model creation and training, and model evaluation phases. [Conclusion] We have made the final list of issues available to the community and believe it will help raise awareness about issues that need to be addressed throughout the ML workflow to reduce TD and improve the maintainability of ML code.

In recent years, Software Engineering (SE) scholars and practitioners have emphasized the importance of integrating soft skills into SE education. However, teaching and learning soft skills are complex, as they cannot be acquired passively through raw knowledge acquisition. On the other hand, hackathons have attracted increasing attention due to their experiential, collaborative, and intensive nature, which certain tasks could be similar to real-world software development. This paper aims to discuss the idea of hackathons as an educational strategy for shaping SE students' soft skills in practice. Initially, we overview the existing literature on soft skills and hackathons in SE education. Then, we report preliminary empirical evidence from a seven-day hybrid hackathon involving 40 students. We assess how the hackathon experience promoted innovative and creative thinking, collaboration and teamwork, and knowledge application among participants through a structured questionnaire designed to evaluate students' self-awareness. Lastly, our findings and new directions are analyzed through the lens of SelfDetermination Theory (SDT), which offers a psychological lens to understand human behavior. This paper contributes to academia by advocating the potential of hackathons in SE education and proposing concrete plans for future research within SDT. For industry, our discussion has implications around developing soft skills in future SE professionals, thereby enhancing their employability and readiness in the software market.

[Context] The lack of practical relevance in many Software Engineering (SE) research contributions is often rooted in oversimplified views of industrial practice, weak industry connections, and poorly defined research problems. Clear criteria for evaluating SE research problems can help align their value, feasibility, and applicability with industrial needs. [Goal] In this paper, we introduce the Lean Research Inception (LRI) framework, designed to support the formulation and assessment of practically relevant research problems in SE. We describe its initial evaluation strategy conducted in a workshop with a network of SE researchers experienced in industry-academia collaboration and report the evaluation of its three assessment criteria (valuable, feasible, and applicable) regarding their importance and completeness in assessing practical relevance. [Method] We applied LRI retroactively to a published research paper, engaging workshop participants in discussing and assessing the research problem by applying the proposed criteria using a semantic differential scale. Participants provided feedback on the criteria’s importance and completeness, drawn from their own experiences in industry-academia collaboration. [Results] The findings reveal an overall agreement on the importance of the three criteria – valuable (83.3%), feasible (76.2%), and applicable (73.8%) – for aligning research problems with industrial needs. Qualitative feedback suggested adjustments in terminology with a clearer distinction between feasible and applicable, and refinements for valuable by more clearly considering business value, ROI, and originality. [Conclusion] While LRI still constitutes ongoing research and requires further evaluation, our emerging results strengthen our confidence that the three criteria applied using the semantic differential scale can already help the community assess the practical relevance of SE research problems.

Machine learning (ML)-enabled systems are being increasingly adopted by companies aiming to enhance their products and operational processes. This paper aims to deliver a comprehensive overview of the current status quo of engineering ML-enabled systems and lay the foundation to steer practically relevant and problem-driven academic research. We conducted an international survey to collect insights from practitioners on the current practices and problems in engineering ML-enabled systems. We received 188 complete responses from 25 countries. We conducted quantitative statistical analyses on contemporary practices using bootstrapping with confidence intervals and qualitative analyses on the reported problems using open and axial coding procedures. Our survey results reinforce and extend existing empirical evidence on engineering ML-enabled systems, providing additional insights into typical ML-enabled systems project contexts, the perceived relevance and complexity of ML life cycle phases, and current practices related to problem understanding, model deployment, and model monitoring. Furthermore, the qualitative analysis provides a detailed map of the problems practitioners face within each ML life cycle phase and the problems causing overall project failure. The results contribute to a better understanding of the status quo and problems in practical environments. We advocate for the further adaptation and dissemination of software engineering practices to enhance the engineering of ML-enabled systems. • International survey gathering insights from 188 practitioners across 25 countries. • Overview of current practices and challenges in engineering ML-enabled systems. • Inferential quantitative analysis reporting the status quo with confidence intervals. • Qualitative analysis mapping ML life cycle challenges and causes of project failure.

In the ever-expanding landscape of Artificial Intelligence (AI), where innovation thrives and new products and services are continuously being delivered, ensuring that AI systems are designed and developed responsibly throughout their entire lifecycle is crucial. To this end, several AI ethics principles and guidelines have been issued to which AI systems should conform. Nevertheless, relying solely on high-level AI ethics principles is far from sufficient to ensure the responsible engineering of AI systems. In this field, AI professionals often navigate by sight. Indeed, while recommendations promoting Trustworthy AI (TAI) exist, they are often high-level statements difficult to translate into concrete implementation strategies. Currently, there is a significant gap between high-level AI ethics principles and low-level concrete practices for AI professionals. To address this challenge, our work presents an experience report where we develop a novel holistic framework for Trustworthy AI --- designed to bridge the gap between theory and practice --- and report insights from its application in an industrial case study. The framework builds up from the results of a systematic review of the state of the practice as well as a survey and think-aloud interviews with 34 AI practitioners. The framework, unlike most of the ones in literature, is designed to provide actionable guidelines and tools to support different types of stakeholders throughout the entire Software Development Life Cycle (SDLC). Our goal is to empower AI professionals to confidently navigate the ethical dimensions of TAI through practical insights, ensuring that the vast potential of AI is exploited responsibly for the benefit of society as a whole.

[Context] Applying design principles has long been acknowledged as beneficial for understanding and maintainability in traditional software projects. These benefits may similarly hold for Machine Learning (ML) projects, which involve iterative experimentation with data, models, and algorithms. However, ML components are often developed by data scientists with diverse educational backgrounds, potentially resulting in code that doesn't adhere to software design best practices. [Goal] In order to better understand this phenomenon, we investigated the impact of the SOLID design principles on ML code understanding. [Method] We conducted a controlled experiment with three independent trials involving 100 data scientists. We restructured real industrial ML code that did not use SOLID principles. Within each trial, one group was presented with the original ML code, while the other was presented with ML code incorporating SOLID principles. Participants of both groups were asked to analyze the code and fill out a questionnaire that included both open-ended and closed-ended questions on their understanding. [Results] The study results provide statistically significant evidence that the adoption of the SOLID design principles can improve code understanding within the realm of ML projects. [Conclusion] We put forward that software engineering design principles should be spread within the data science community and considered for enhancing the maintainability of ML code.

Recently, there has been growing attention on behalf of both academic and practice communities towards the ability of Artificial Intelligence (AI) systems to operate responsibly and ethically. As a result, a plethora of frameworks and guidelines have appeared to support practitioners in implementing Trustworthy AI applications (TAI). However, little research has been done to investigate whether such frameworks are being used and how. In this work, we study the vision AI practitioners have on TAI principles, how they address them, and what they would like to have – in terms of tools, knowledge, or guidelines – when they attempt to incorporate such principles into the systems they develop. Through a survey and semi-structured interviews, we systematically investigated practitioners’ challenges and needs in developing TAI systems. Based on these practical findings, we highlight recommendations to help AI practitioners develop Trustworthy AI applications.

Engineering successful machine learning (ML)-enabled systems poses various challenges from both a theoretical and a practical side. Among those challenges are how to effectively address unrealistic expectations of ML capabilities from customers, managers and even other team members, and how to connect business value to engineering and data science activities composed by interdisciplinary teams. In this paper, we present PerSpecML , a perspective-based approach for specifying ML-enabled systems that helps practitioners identify which attributes, including ML and non-ML components, are important to contribute to the overall system’s quality. The approach involves analyzing 60 concerns related to 28 tasks that practitioners typically face in ML projects, grouping them into five perspectives: system objectives, user experience , infrastructure, model, and data. Together, these perspectives serve to mediate the communication between business owners, domain experts, designers, software and ML engineers, and data scientists. The creation of PerSpecML involved a series of formative evaluations conducted in different contexts: (i) in academia, (ii) with industry representatives, and (iii) in two real industrial case studies . As a result of the diverse validations and continuous improvements, PerSpecML stands as a promising approach, poised to positively impact the specification of ML-enabled systems, particularly helping to reveal key components that would have been otherwise missed without using PerSpecML . Editor’s note: Open Science material was validated by the Journal of Systems and Software Open Science Board .

The modernization of monolithic legacy systems with microservices has been a trend in recent years. As part of this modernization, identifying microservice candidates starting from legacy code is challenging, as maintainers may consider many criteria simultaneously. Multi-objective search-based approaches represent a promising state-of-the-art solution to support this decision-making process. However, the rationale to adopt each microservice candidate automatically identified by these approaches is poorly investigated in industrial cases. Furthermore, studies with these approaches have not carefully investigated how maintainers reason and make decisions when designing microservice architectures from legacy systems. To address this gap, we conducted an on-site case study with maintainers of an industrial legacy system to investigate the usefulness of automatically generated microservice architectures. We analyze design decisions pointed out by the maintainers when reasoning about microservice candidates using several criteria at the same time. Our study is the first to assess a search-based approach involving actual maintainers conceiving microservice architectures in an industrial setting. Therefore, firstly, we considered individual evaluation of microservice candidates to understand the rationale for identifying a service. Secondly, we conducted a focus group study with maintainers with the goal of investigating design decisions at an architectural level. The results show that:(i)the automated approach is able to identify useful microservices;(ii)the criteria observed by previous studies are, in fact, considered by maintainers; and(iii)the maintainer profiles, i.e., the preferred granularity for microservice, highly affect design decisions. Finally, we observed the maintainers needed little effort in adjusting the automatically identified microservices to make them adoptable. In addition to indicating a promising potential of search-based approaches to generate microservice architectures, our findings highlight the need for:(i)interactive and/or customizable approaches that enable maintainers to include their preferences during the search process, and(ii)flexible or automated selection of criteria that fits the scenario in which the modernization is taking place.

Background. Due to the widespread adoption of Artificial Intelligence (AI) and Machine Learning (ML) for building software applications, companies are struggling to recruit employees with a deep understanding of such technologies. In this scenario, AutoML is soaring as a promising solution to fill the AI/ML skills gap since it promises to automate the building of end-to-end AI/ML pipelines that would normally be engineered by specialized team members. Aims. Despite the growing interest and high expectations, there is a dearth of information about the extent to which AutoML is currently adopted by teams developing AI/ML-enabled systems and how it is perceived by practitioners and researchers. Method. To fill these gaps, in this paper, we present a mixed-method study comprising a benchmark of 12 end-to-end AutoML tools on two SE datasets and a user survey with follow-up interviews to further our understanding of AutoML adoption and perception. Results. We found that AutoML solutions can generate models that outperform those trained and optimized by researchers to perform classification tasks in the SE domain. Also, our findings show that the currently available AutoML solutions do not live up to their names as they do not equally support automation across the stages of the ML development workflow and for all the team members. Conclusions. We derive insights to inform the SE research community on how AutoML can facilitate their activities and tool builders on how to design the next generation of AutoML technologies.

Several refactorings performed while evolving software features aim to improve internal quality attributes like cohesion and complexity. Indeed, internal attributes can become critical if their measurements assume anomalous values. Yet, current knowledge is scarce on how developers perceive the relevance of critical internal attributes while evolving features. This qualitative study investigates the developers’ perception of the relevance of critical internal attributes when evolving features. We target six class-level critical attributes: low cohesion, high complexity, high coupling, large hierarchy depth, large hierarchy breadth, and large size. We performed two industrial case studies based on online focus group sessions. Developers discussed how much (and why) critical attributes are relevant when adding or enhancing features. We assessed the relevance of critical attributes individually and relatively, the reasons behind the relevance of each critical attribute, and the interrelations of critical attributes. Low cohesion and high complexity were perceived as very relevant because they often make evolving features hard while tracking failures and adding features. The other critical attributes were perceived as less relevant when reusing code or adopting design patterns. An example of perceived interrelation is high complexity leading to high coupling.

Context: New scientific evidence continuously arises with advances in Software Engineering (SE) research. Conventionally, Systematic Literature Reviews (SLRs) are not updated or updated intermittently, leaving gaps between updates, during which time the SLR may be missing crucial new evidence. Goal: We propose and evaluate a concept and process called Continuous Systematic Literature Review (CSLR) in SE. Method: To elaborate on the CSLR concept and process, we performed a synthesis of evidence by conducting a meta-ethnography, addressing knowledge from varied research areas. Furthermore, we conducted a case study to evaluate the CSLR process. Results: We describe the resulting CSLR process in BPMN format. The case study results provide indications on the importance and preliminary feasibility of applying CSLR in practice to continuously update SLR evidence in SE. Conclusion: The CSLR concept and process provide a feasible and systematic way to continuously incorporate new evidence into SLRs, supporting trustworthy and up-to-date evidence for SLRs in SE.

Machine learning (ML) teams often work on a project just to realize the performance of the model is not good enough. Indeed, the success of ML-enabled systems involves aligning data with business problems, translating them into ML tasks, experimenting with algorithms, evaluating models, capturing data from users, among others. Literature has shown that ML-enabled systems are rarely built based on precise specifications for such concerns, leading ML teams to become misaligned due to incorrect assumptions, which may affect the quality of such systems and overall project success. In order to help addressing this issue, this paper describes our work towards a perspective-based approach for specifying ML-enabled systems. The approach involves analyzing a set of 45 ML concerns grouped into five perspectives: objectives, user experience, infrastructure, model, and data. The main contribution of this paper is to provide two new artifacts that can be used to help specifying ML-enabled systems: (i) the perspective-based ML task and concern diagram and (ii) the perspective-based ML specification template.

A good search strategy is essential for a successful systematic literature study. Historically, database searches have been the norm, which was later complemented with snowball searches. Our conjecture is that we can perform even better searches if combining these two search approaches, referred to as a hybrid search strategy. Our main objective was to compare and evaluate a hybrid search strategy. Furthermore, we compared four alternative hybrid search strategies to assess whether we could identify more cost-efficient ways of searching for relevant primary studies. To compare and evaluate the hybrid search strategy, we replicated the search procedure in a systematic literature review (SLR) on industry–academia collaboration in software engineering. The SLR used a more “traditional” approach to searching for relevant articles for an SLR, while our replication was executed using a hybrid search strategy. In our evaluation, the hybrid search strategy was superior in identifying relevant primary studies. It identified 30% more primary studies and even more studies when focusing only on peer-reviewed articles. To embrace individual viewpoints when assessing research articles and minimise the risk of missing primary studies, we introduced two new concepts, wild cards and borderline articles, when performing systematic literature studies. The hybrid search strategy is a strong contender for being used when performing systematic literature studies. Furthermore, alternative hybrid search strategies may be viable if selected wisely in relation to the start set for snowballing. Finally, the two new concepts were judged as essential to cater for different individual judgements and to minimise the risk of excluding primary studies that ought to be included.

Dependency Injection (DI) is a commonly applied mechanism to decouple classes from their dependencies in order to provide higher modularization. However, bad DI practices often lead to negative consequences, such as increasing coupling. Although white literature conjectures about the existence of DI anti-patterns, there is no evidence on their practical relevance, usefulness, and generality. The objective of this study is to propose and evaluate a catalog of DI anti-patterns and associated refactorings. We reviewed existing reported DI anti-patterns in order to analyze their completeness. The limitations found in literature motivated proposing a novel catalog of 12 DI anti-patterns. We developed a tool to statically analyze the occurrence level of the candidate DI anti-patterns in both open-source and industry projects. Next, we survey practitioners to assess their perception on the relevance, usefulness, and their willingness on refactoring anti-pattern instances of the catalog. Our static code analyzer tool showed a relative recall of 92.19% and high average precision. It revealed that at least 9 different DI anti-patterns appeared frequently in the analyzed projects. Besides, our survey confirmed the perceived relevance of the catalog and developers expressed their willingness to refactor instances of anti-patterns from source code. The catalog contains DI anti-patterns that occur in practice and that are perceived as useful. Sharing it with practitioners may help them to avoid such anti-patterns, thus improving source-code quality.

Together with many success stories, promises such as the increase in production speed and the improvement in stakeholders’ collaboration have contributed to making agile a transformation in the software industry in which many companies want to take part. However, driven either by a natural and expected evolution or by contextual factors that challenge the adoption of agile methods as prescribed by their creator(s), software processes in practice mutate into hybrids over time. Are these still agile? In this article, we investigate the question: what makes a software development method agile? We present an empirical study grounded in a large-scale international survey that aims to identify software development methods and practices that improve or tame agility. Based on 556 data points, we analyze the perceived degree of agility in the implementation of standard project disciplines and its relation to used development methods and practices. Our findings suggest that only a small number of participants operate their projects in a purely traditional or agile manner (under 15 percent). That said, most project disciplines and most practices show a clear trend towards increasing degrees of agility. Compared to the methods used to develop software, the selection of practices has a stronger effect on the degree of agility of a given discipline. Finally, there are no methods or practices that explicitly guarantee or prevent agility. We conclude that agility cannot be defined solely at the process level. Additional factors need to be taken into account when trying to implement or improve agility in a software company. Finally, we discuss the field of software process-related research in the light of our findings and present a roadmap for future research.

While most research shows positive effects of gamification, the focus on its adverse effects is considerably smaller. Having this in mind, we conducted a systematic mapping study of the negative effects of game design elements on education/learning systems. The study revealed 77 papers reporting undesired effects of game design elements. We found that badges, competitions, Ieaderboards, and points are the game design elements most often reported as causing negative effects. The most cited negative effects were lack of effect, lack of understanding, irrelevance, motivational issues, and worsened performance. The ethical issue of cheating was also often reported. As part of our results, we map the relations between game design elements and the negative effects that they may cause. Our mapping study can help gamification designers make more informed decisions when selecting game design elements to be included in education/learning systems, raising awareness on potential negative effects.

[Background] Many startup environments and even traditional software companies have embraced the use of MVPs (Minimum Viable Products) to allow quickly experimenting solution options. The MVP concept has influenced the way in which development teams apply Software Engineering (SE) practices. However, the overall understanding of this influence of MVPs on SE practices is still poor. [Objective] Our goal is to characterize the publication landscape on practices that have been used in the context of software MVPs. [Method] We conducted a systematic mapping study using a hybrid search strategy that consists of a database search and parallel forward and backward snowballing. [Results] We identified 33 papers, published between 2013 and 2020. We observed some trends related to MVP ideation and evaluation practices. For instance, regarding ideation, we found six different approaches (e.g., Design Thinking, Lean Inception) and mainly informal end-user involvement practices (e.g., workshops, interviews). For evaluation there is an emphasis on end-user validations based on practices such as usability tests, A/B testing, and usage data analysis. However, there is still limited research related to MVP technical feasibility assessment and effort estimation. We also observed a lack of scientific rigor in many of the identified studies. [Conclusion] Our analysis suggests that there are opportunities for solution proposals to address gaps concerning technical feasibility assessment and effort estimation. Also, more effort needs to be invested into empirically evaluating the existing MVP-related practices.

Machine learning (ML) has become a core feature for today’s real-world applications, making it a trending topic for the software engineering community. Requirements Engineering (RE) is no stranger to this and its main conferences have included workshops aiming at discussing RE in the context of ML. However, current research on the intersection between RE and ML mainly focuses on using ML techniques to support RE activities rather than on exploring how RE can improve the development of ML-based systems. This paper concerns a systematic mapping study aiming at characterizing the publication landscape of RE for ML-based systems, outlining research contributions and contemporary gaps for future research. In total, we identified 35 studies that met our inclusion criteria. We found several different types of contributions, in the form of analyses, approaches, checklists and guidelines, quality models, and taxonomies. We discuss gaps by mapping these contributions against the RE topics to which they were contributing and their type of empirical evaluation. We also identified quality characteristics that are particularly relevant for the ML context (e.g., data quality, explainability, fairness, safety, and transparency). Main reported challenges are related to the lack of validated RE techniques, the fragmented and incomplete understanding of NFRs for ML, and difficulties in handling customer expectations. There is a need for future research on the topic to reveal best practices and to propose and investigate approaches that are suitable to be used in practice.

Microservices have become a popular architectural style for data-driven applications, given their ability to functionally decompose an application into small and autonomous services to achieve scalability, strong isolation, and specialization of database systems to the workloads and data formats of each service. Despite the accelerating industrial adoption of this architectural style, an investigation of the state of the practice and challenges practitioners face regarding data management in microservices is lacking. To bridge this gap, we conducted a systematic literature review of representative articles reporting the adoption of microservices, we analyzed a set of popular open-source microservice applications, and we conducted an online survey to cross-validate the findings of the previous steps with the perceptions and experiences of over 120 experienced practitioners and researchers. Through this process, we were able to categorize the state of practice of data management in microservices and observe several foundational challenges that cannot be solved by software engineering practices alone, but rather require system-level support to alleviate the burden imposed on practitioners. We discuss the shortcomings of state-of-the-art database systems regarding microservices and we conclude by devising a set of features for microservice-oriented database systems.

Petrobras is Brazil's largest publicly-held company, operating in the oil, natural gas, and energy industry. Internal efforts enabled Petrobras to identify Digital Transformation (DT) opportunities to further promote their operational excellence. While addressing these opportunities typically requires Research and Development (R&D) uncertainties that could lead traditional R&D cooperation terms to be negotiated in years, there are time-to-market constraints for fast-paced deliveries to experiment solution options. Having this in mind, they partnered up with PUC-Rio to establish a new DT initiative. [Goal] The goal of this paper is to present the Lean R&D approach, tailored within the new initiative to meet the aforementioned DT needs. [Method] We designed Lean R&D integrating the following building blocks: (i) Lean Inceptions, to allow stakeholders to jointly outline a Minimal Viable Product (MVP); (ii) parallel technical feasibility assessment and conception phases, allowing to `fail fast'; (iii) scrum-based development management; and (iv) strategically aligned continuous experimentation to test business hypotheses. We report on first experiences of applying Lean R&D in practice. [Results] Lean R&D enabled addressing research-related uncertainties early and to efficiently deliver valuable MVPs within fast-paced four months cycles. [Conclusions] In our first experiences Lean R&D showed itself suitable for supporting DT initiatives. However, more formal case studies are needed. The business strategy alignment and the continuous support of a highly qualified research team were considered key success factors.

Context: Data-intensive systems, a.k.a. big data systems (BDS), are software systems that handle a large volume of data in the presence of performance quality attributes, such as scalability and availability. Before the advent of big data management systems (e.g. Cassandra) and frameworks (e.g. Spark), organizations had to cope with large data volumes with custom-tailored solutions. In particular, a decade ago, Tecgraf/PUC-Rio developed a system to monitor truck fleet in real-time and proactively detect events from the positioning data received. Over the years, the system evolved into a complex and large obsolescent code base involving a costly maintenance process. Goal: We report our experience on replacing a legacy BDS with a microservice-based event-driven system. Method: We applied action research, investigating the reasons that motivate the adoption of a microservice-based event-driven architecture, intervening to define the new architecture, and documenting the challenges and lessons learned. Results: We perceived that the resulting architecture enabled easier maintenance and faultisolation. However, the myriad of technologies and the complex data flow were perceived as drawbacks. Based on the challenges faced, we highlight opportunities to improve the design of big data reactive systems. Conclusions: We believe that our experience provides helpful takeaways for practitioners modernizing systems with data-intensive requirements.

Companies adopting agile development tend to face challenges in complying with security norms. Existing research either focuses on how to integrate security into agile methods or on discussing compliance issues of agile methods but independently of the regulation type, in particular of security standards. A comprehensive overview of this scattered field is still missing and we know little about how to achieve security compliance in agile software development. Existing secondary studies (mapping studies and literature reviews) analyze publications on secure agile development, but they do not analyze implications of security standard compliance, e.g., integration of specific standard requirements or compliance assessments. To close this gap, we report on a systematic mapping study. Starting with a set of 2,383 papers, our work distills 11 relevant publications addressing security compliance in agile software development. With this study, we contribute by describing the maturity of the field, as well as domains where security compliant agile software engineering was investigated. Moreover, we make explicit which phases of a secure development process are covered by the field and which agile principles are analyzed when aiming at compliance with international security standards, country-specific security regulations, industry-specific security standards, and other well-known security frameworks.

Abstract Context As a novel coronavirus swept the world in early 2020, thousands of software developers began working from home. Many did so on short notice, under difficult and stressful conditions. Objective This study investigates the effects of the pandemic on developers’ wellbeing and productivity. Method A questionnaire survey was created mainly from existing, validated scales and translated into 12 languages. The data was analyzed using non-parametric inferential statistics and structural equation modeling. Results The questionnaire received 2225 usable responses from 53 countries. Factor analysis supported the validity of the scales and the structural model achieved a good fit (CFI = 0.961, RMSEA = 0.051, SRMR = 0.067). Confirmatory results include: (1) the pandemic has had a negative effect on developers’ wellbeing and productivity; (2) productivity and wellbeing are closely related; (3) disaster preparedness, fear related to the pandemic and home office ergonomics all affect wellbeing or productivity. Exploratory analysis suggests that: (1) women, parents and people with disabilities may be disproportionately affected; (2) different people need different kinds of support. Conclusions To improve employee productivity, software companies should focus on maximizing employee wellbeing and improving the ergonomics of employees’ home offices. Women, parents and disabled persons may require extra support.

Systematic Literature Reviews (SLRs) have been adopted within Software Engineering (SE) for more than a decade to provide meaningful summaries of evidence on several topics. Many of these SLRs are now potentially not fully up-to-date, and there are no standard proposals on how to update SLRs in SE. The objective of this paper is to propose guidelines on how to best search for evidence when updating SLRs in SE, and to evaluate these guidelines using an SLR that was not employed during the formulation of the guidelines. To propose our guidelines, we compare and discuss outcomes from applying different search strategies to identify primary studies in a published SLR, an SLR update, and two replications in the area of effort estimation. These guidelines are then evaluated using an SLR in the area of software ecosystems, its update and a replication. The use of a single iteration forward snowballing with Google Scholar, and employing as a seed set the original SLR and its primary studies is the most cost-effective way to search for new evidence when updating SLRs. Furthermore, the importance of having more than one researcher involved in the selection of papers when applying the inclusion and exclusion criteria is highlighted through the results. Our proposed guidelines formulated based upon an effort estimation SLR, its update and two replications, were supported when using an SLR in the area of software ecosystems, its update and a replication. Therefore, we put forward that our guidelines ought to be adopted for updating SLRs in SE.

Background: Replication is essential to build knowledge in empirical science. Experiment replications reported in the software engineering context present variabilities on their design elements, e.g., variables, materials. The understanding of these variabilities is required to plan experimental replications within a research program. However, the lack of an explicit representation of experiments' variabilities and commonalities is likely to hamper their understanding and replication planning. Aims: The goal of this paper is to explore the use of Variability Modeling Approaches (VMAs) to represent experiment families (i.e., an original study and its replications) and to investigate the feasibility of using VMAs to support experiment replication planning. Method: We selected two experiment families, analyzed their commonalities and variabilities, and represented them using a set of well-known VMAs: Feature Model, Decision Model, and Orthogonal Variability Model. Based on the resulting models, we conducted a preliminary comparison of using such alternative VMAs to support replication planning. Results: Subjects were able to plan consistent experiment replications with the VMAs as support. Additionally, through a qualitative analysis, we identified and discuss advantages and limitations of using the VMAs. Conclusions: It is feasible to represent experiment families and to plan replications using VMAs. Based on our emerging results, we conclude that the Feature Model VMA provides the most suitable representation. Furthermore, we identified benefits in a potential merge between the Feature Model and Decision Model VMAs to provide more details to support replication planning.

Background: Code refactoring aims to improve code structures via code transformations. A single transformation rarely suffices to fully remove code smells that reveal poor code structures. Most transformations are applied in batches, i.e. sets of interrelated transformations, rather than in isolation. Nevertheless, empirical knowledge on batch application, or batch refactoring, is scarce. Such scarceness helps little to improve current refactoring practices. Aims: We analyzed 57 open and closed software projects. We aimed to understand batch application from two perspectives: characteristics that typically constitute a batch (e.g., the variety of transformation types employed), and the batch effect on smells. Method: We analyzed 19 smell types and 13 transformation types. We identified 4,607 batches, each applied by the same developer on the same code element (method or class); we expected to have batches whose transformations are closely interrelated. We computed (1) the frequency in which five batch characteristic manifest, (2) the probability of each batch characteristics to remove smells, and (3) the frequency in which batches introduce and remove smells. Results: Most batches are quite simple: although most batches are applied on more than one method (90%), they are usually composed of the same transformation type (72%) and only two transformations (57%). Batches applied on a single method are 2.6 times more prone to fully remove smells than batches affecting more than one method. Surprisingly, batches mostly ended up introducing (51%) or not fully removing (38%) smells. Conclusions: The batch simplicity suggests that developers have sub-explored the combinations of transformations within a batch. We summarized some batches that may fully remove smells, so that developers can incorporate them into current refactoring practices.

Background: Over the years, several studies were conducted aiming at understanding the Technical Debt (TD) phenomenon and its implications on software development. Most of these studies focus on source code related TD types. The absence of empirical studies on usability debt motivated our research. Aims: The goal of this paper is to provide an initial usability debt characterization in software projects regarding its occurrence, type, and resolution effort. Method: We conducted a multi-case study, analyzing TD items of five software projects from four different companies. Results: We identified and classified 145 TD items in the projects. The analysis of these items allowed us to observe that the frequency of usability TD items ranged from 10.4% to 20.8% in the projects. The usability debt items cover a range of usability issues, violating eight out of the ten Nielsen usability heuristics. Regarding effort for paying the TD, usability debt items require a relatively low effort, ranging from 5.1% to 6.7% of the total TD resolution effort in the analyzed projects. Conclusions: Usability TD items are frequent, concern relevant usability issues and typically require low effort for their payment. Hence, paying this type of TD should receive high priority in TD management strategies.

[Context] Systematic Literature Reviews (SLRs) have been adopted within the Software Engineering (SE) domain for more than a decade to provide meaningful summaries of evidence on several topics. Many of these SLRs are now outdated, and there are no standard proposals on how to update SLRs in SE. [Objective] The goal of this paper is to provide recommendations on how to best to search for evidence when updating SLRs in SE. [Method] To achieve our goal, we compare and discuss outcomes from applying different search strategies to identifying primary studies in a previously published SLR update on effort estimation. [Results] The use of a single iteration forward snowballing with Google Scholar, and employing the original SLR and its primary studies as a seed set seems to be the most cost-effective way to search for new evidence when updating SLRs. [Conclusions] The recommendations can be used to support decisions on how to update SLRs in SE.

Defects in requirements specifications can have severe consequences during the software development lifecycle. Some of them result in overall project failure due to incorrect or missing quality characteristics such as security. There are several concerns that make security difficult to deal with; for instance, (1) when stakeholders discuss general requirements in meetings, they are often unaware that they should also discuss security-related topics, and (2) they typically do not have enough expertise in security. This often leads to unspecified or ill-defined security-related aspects. These concerns become even more challenging in agile contexts, where lightweight documentation is typically involved. The goal of this paper is to design and evaluate an approach for reviewing security-related aspects in agile requirements specifications of web applications. The approach considers user stories and security specifications as input and relates those user stories to security properties via Natural Language Processing. Based on the related security properties, our approach then identifies high-level security requirements from the Open Web Application Security Project to be verified and generates a reading technique to support reviewers in detecting defects. We evaluate our approach via two controlled experiment trials. We compare the effectiveness and efficiency of novice inspectors verifying security aspects in agile requirements using our approach against using the complete list of high-level security requirements. The (statistically significant) results indicate that using our approach has a positive impact (with large effect size) on the performance of inspectors in terms of effectiveness and efficiency.

Requirements Engineering (RE) has established itself as a software engineering discipline over the past decades. While researchers have been investigating the RE discipline with a plethora of empirical studies, attempts to systematically derive an empirical theory in context of the RE discipline have just recently been started. However, such a theory is needed if we are to define and motivate guidance in performing high quality RE research and practice. We aim at providing an empirical and externally valid foundation for a theory of RE practice, which helps software engineers establish effective and efficient RE processes in a problem-driven manner. We designed a survey instrument and an engineer-focused theory that was first piloted in Germany and, after making substantial modifications, has now been replicated in 10 countries worldwide. We have a theory in the form of a set of propositions inferred from our experiences and available studies, as well as the results from our pilot study in Germany. We evaluate the propositions with bootstrapped confidence intervals and derive potential explanations for the propositions. In this article, we report on the design of the family of surveys, its underlying theory, and the full results obtained from the replication studies conducted in 10 countries with participants from 228 organisations. Our results represent a substantial step forward towards developing an empirical theory of RE practice. The results reveal, for example, that there are no strong differences between organisations in different countries and regions, that interviews, facilitated meetings and prototyping are the most used elicitation techniques, that requirements are often documented textually, that traces between requirements and code or design documents are common, that requirements specifications themselves are rarely changed and that requirements engineering (process) improvement endeavours are mostly internally driven. Our study establishes a theory that can be used as starting point for many further studies for more detailed investigations. Practitioners can use the results as theory-supported guidance on selecting suitable RE methods and techniques.

Background: Unhandled exceptions affect the reliability of web applications. Several studies have measured the reliability of web applications in use against unhandled exceptions, showing a recurrence of the problem during the maintenance phase. Detecting latent unhandled exceptions automatically is difficult and application-specific. Hence, general approaches to deal with defects in web applications do not treat unhandled exceptions appropriately. Aims: To design and evaluate a method that can support finding, correcting, and preventing unhandled exceptions in web applications. Method: We applied the design science engineering cycle to design a method called Pattern-Driven Maintenance (PDM). PDM relies on identifying defect patterns based on application server logs and producing static analysis rules that can be used for prevention. We applied PDM to two industrial web applications involving different companies and technologies, measuring the reliability improvement and the precision of the produced static analysis rules. Results: In both cases, our approach allowed identifying defect patterns and finding latent unhandled exceptions to be fixed in the source code, enabling to completely eliminate the pattern-related failures and improving the application reliability. The static analysis rules produced by PDM achieved a precision of 59-68% in the first application and 89-100% in the second, where lessons learnt from the first evaluation were addressed. Conclusions: The results strengthen our confidence that PDM can help maintainers to improve the reliability for unhandled exceptions in other existing web applications.

[Context] Data is being collected at an unprecedented scale. Data sets are becoming so large and complex that traditionally engineered systems may be inadequate to deal with them. While software engineering comprises a large set of approaches to support engineering robust software systems, there is no comprehensive overview of approaches that have been proposed and/or applied in the context of engineering big data systems. [Goal] This study aims at surveying existing research on big data software engineering to unveil and characterize the development approaches and major contributions. [Method] We conducted a systematic mapping study, identifying 52 related research papers, dated from 2011 to 2016. We classified and analyzed the identified approaches, their objectives, application domains, development lifecycle phase, and type of contribution. [Results] As a result, we outline the current state of the art and gaps on employing software engineering approaches to develop big data systems. For instance, we observed that the major challenges are in the area of software architecture and that more experimentation is needed to assess the classified approaches. [Conclusion] The results of this systematic mapping provide an overview on existing approaches to support building big data systems and helps to steer future research based on the identified gaps.

[Background] The rapidly changing business environments in which many companies operate is challenging traditional Requirements Engineering (RE) approaches. This gave rise to agile approaches for RE. Security, at the same time, is an essential non-functional requirement that still tends to be difficult to address in agile development contexts. Given the fuzzy notion of "agile" in context of RE and the difficulties of appropriately handling security requirements, the overall understanding of how to handle security requirements in agile RE is still vague. [Objective] Our goal is to characterize the publication landscape of approaches that handle security requirements in agile software projects. [Method] We conducted a systematic mapping to outline relevant work and contemporary gaps for future research. [Results] In total, we identified 21 studies that met our inclusion criteria, dated from 2005 to 2017. We found that the approaches typically involve modifying agile methods, introducing new artifacts (e.g., extending the concept of user story to abuser story), or introducing guidelines to handle security issues. We also identified limitations of using these approaches related to environment, people, effort and resources. [Conclusion] Our analysis suggests that more effort needs to be invested into empirically evaluating the existing approaches and that there is an avenue for future research in the direction of mitigating the identified limitations.

The prevalence of design problems may cause re-engineering or even discontinuation of the system. Due to missing, informal or outdated design documentation, developers often have to rely on the source code to identify design problems. Therefore, developers have to analyze different symptoms that manifest in several code elements, which may quickly turn into a complex task. Although researchers have been investigating techniques to help developers in identifying design problems, there is little knowledge on how developers actually proceed to identify design problems. In order to tackle this problem, we conducted a multi-trial industrial experiment with professionals from 5 software companies to build a grounded theory. The resulting theory offers explanations on how developers identify design problems in practice. For instance, it reveals the characteristics of symptoms that developers consider helpful. Moreover, developers often combine different types of symptoms to identify a single design problem. This knowledge serves as a basis to further understand the phenomena and advance towards more effective identification techniques.

A common software product line strategy involves plugin-based web systems that support simple and quick incorporation of custom behaviors. As a result, they have been widely adopted to create web-based applications. Indeed, the popularity of ecosystems that support plugin-based development (e.g., WordPress) is largely due to the number of customization options available as community-contributed plugins. However, plugin-related vulnerabilities tend to be recurrent, exploitable and hard to be detected and may lead to severe consequences for the customized product. Hence, there is a need to further understand such vulnerabilities to enable preventing relevant security threats. Therefore, we conducted an exploratory study to characterize vulnerabilities caused by plugins in web-based systems. To this end, we went over WordPress vulnerability bulletins cataloged by the National Vulnerability Database as well as associated patches maintained by the WordPress plugins repository. We identified the main types of vulnerabilities caused by plugins as well as their impact and the size of the patch to fix the vulnerability. Moreover, we identified the most common security-related topics discussed among WordPress developers. We observed that, while plugin-related vulnerabilities may have severe consequences and might remain unnoticed for years before being fixed, they can commonly be mitigated with small and localized changes to the source code. The characterization helps to provide an understanding on how typical plugin-based vulnerabilities manifest themselves in practice. Such information can be helpful to steer future research on plugin-based vulnerability detection and prevention.

[Background] Systematic Literature Reviews (SLRs) are one of the important pillars when employing an evidence-based paradigm in Software Engineering. To date most SLRs have been conducted using a search strategy involving several digital libraries. However, significant issues have been reported for digital libraries and applying such search strategy requires substantial effort. On the other hand, snowballing has recently arisen as a potentially more efficient alternative or complementary solution. Nevertheless, it requires a relevant seed set of papers. [Aims] This paper proposes and evaluates a hybrid search strategy combining searching in a specific digital library (Scopus) with backward and forward snowballing. [Method] The proposed hybrid strategy was applied to two previously published SLRs that adopted database searches. We investigate whether it is able to retrieve the same included papers with lower effort in terms of the number of analysed papers. The two selected SLRs relate respectively to elicitation techniques (not confined to Software Engineering (SE)) and to a specific SE topic on cost estimation. [Results] Our results provide preliminary support for the proposed hybrid search strategy as being suitable for SLRs investigating a specific research topic within the SE domain. Furthermore, it helps overcoming existing issues with using digital libraries in SE. [Conclusions] The hybrid search strategy provides competitive results, similar to using several digital libraries. However, further investigation is needed to evaluate the hybrid search strategy.

Background: A Systematic Literature Review (SLR) is a methodology used to aggregate relevant evidence related to one or more research questions. Whenever new evidence is published after the completion of a SLR, this SLR should be updated in order to preserve its value. However, updating SLRs involves significant effort. Objective: The goal of this paper is to investigate the application of forward snowballing to support the update of SLRs. Method: We compare outcomes of an update achieved using the forward snowballing versus a published update using the search-based approach, i.e., searching for studies in electronic databases using a search string. Results: Forward snowballing showed a higher precision and a slightly lower recall. It reduced in more than five times the number of primary studies to filter however missed one relevant study. Conclusions: Due to its high precision, we believe that the use of forward snowballing considerably reduces the effort in updating SLRs in Software Engineering; however the risk of missing relevant papers should not be underrated.

[Context] Problems in Requirements Engineering (RE) can lead to serious consequences during the software development lifecycle. [Goal] The goal of this paper is to propose empirically-based guidelines that can be used by different types of organisations according to their size (small, medium or large) and process model (agile or plan-driven) to help them in preventing such problems. [Method] We analysed data from a survey on RE problems answered by 228 organisations in 10 different countries. [Results] We identified the most critical RE problems, their causes and mitigation actions, organizing this information by clusters of size and process model. Finally, we analysed the causes and mitigation actions of the critical problems of each cluster to get further insights into how to prevent them. [Conclusions] Based on our results, we suggest preliminary guidelines for preventing critical RE problems in response to context characteristics of the companies.

Context] Many software projects fail due to problems in requirements engineering (RE).[Objective] The goal of this paper is to gather information on relevant RE problems and to represent knowledge on their most common causes.[Method] We replicated a global family of RE surveys in Brazil and used the data to identify critical RE problems and to build probabilistic causeeffect diagrams to represent knowledge on their common causes.[Results] The survey was answered by 74 different organizations, including small, medium and very large sized companies, conducting both, plan-driven and agile development.The most critical RE problems, according to those organizations, are related to communication and to incomplete or underspecified requirements.We provide the full probabilistic cause-effect diagrams with knowledge on common causes of the most critical identified RE problems online.[Conclusion] We believe that the knowledge presented in the diagrams can be helpful to support organizations in conducting causal analysis sessions by providing an initial understanding on what usually causes critical RE problems.

[Objective] The objective of this paper is to extend a previously conducted systematic literature review (SLR) that investigated under what circumstances individual organizations would be able to rely on cross-company based estimation models. [Method] We applied the same methodology used in the SLR we are extending herein (covering the period 2006-2013) based on primary studies that compared predictions from cross-company models with predictions from within-company models constructed from analysis of project data. [Results] We identified 11 additional papers; however two of these did not present independent results and one had inconclusive findings. Two of the remaining eight papers presented both, trials where cross-company predictions were not significantly different from within-company predictions and others where they were significantly different. Four found that cross-company models gave prediction accuracy significantly different from within-company models (one of them in favor of cross-company models), while two found no significant difference. The main pattern when examining the study related factors was that studies where cross-company predictions were significantly different from within-company predictions employed larger within-company data sets. [Conclusions] Overall, half of the analyzed evidence indicated that cross-company estimation models are not significantly worse than within-company estimation models. Moreover, there is some evidence that sample size does not imply in higher estimation accuracy, and that samples for building estimation models should be carefully selected/filtered based on quality control and project similarity aspects. The results need to be combined with the findings from the SLR we are extending to allow further investigating this topic.

[Context] Experiment planners need to be aware of relevant Threats to Validity (TTVs), so they can devise effective control actions or accept the risk. [Objective] The aim of this paper is to introduce a TTV knowledge base (KB) that supports experiment planners in identifying relevant TTVs in their research context and actions to control these TTVs. [Method] We identified requirements, designed and populated a TTV KB with data extracted during a systematic review: 63 TTVs and 149 control actions from 206 peer-reviewed published software engineering experiments. We conducted an initial proof of concept on the feasibility of using the TTV KB and analyzed its content. [Results] The proof of concept and content analysis provided indications that experiment planners can benefit from an extensible TTV KB for identifying relevant TTVs and control actions in their specific context. [Conclusions] The TTV KB should be further evaluated and evolved in a variety of software engineering contexts.

Context. Software engineering researchers conduct systematic literature reviews (SLRs) to build bodies of knowledge (BoKs). Unfortunately, relevant knowledge collected in the SLR process is not publicly available, which considerably slows down building BoKs incrementally. Objective. We present and evaluate the Systematic Knowledge Engineering (SKE) process to support efficiently building BoKs from published research. Method. SKE is based on the SLR process and on Knowledge Engineering practices to build a Knowledge Base (KB) by reusing intermediate data extraction results from SLRs. We evaluated the feasibility of applying SKE by building a Software Inspection BoK KB from published experiments and a Software Product Line BoK KB from published experience reports. We compared the effort, benefits, and risks of building BoK KBs regarding the SKE and the traditional SLR processes. Results. The application of SKE for incrementally collecting and organizing knowledge in the context of a BoK was feasible for different domains and different types of evidence. While the efforts for conducting the SKE and traditional SLR processes are comparable, SKE provides significant benefits for building BoKs. Conclusions. SKE enables researchers in a scientific community to reuse and incrementally build knowledge in a BoK. SKE is ready to be evaluated in other software engineering domains.

Experimentation in software engineering represents a means to understand the impact of adopting software technologies. In this paper we describe an on-going experimentation work aiming at to address a real industry need in Brazil: the understanding of how performance variables (such as cost, productivity, and quality) are affected by the adoption of a nationwide software development reference model. The main strategy consists in integrating the application of carefully planned surveys into the model implementation and assessment procedures. Besides the expected performance results, we hope it can be useful to reinforce to the Brazilian software industry the importance of experimentation to support software technologies evaluation.

Evaluation of group support is a complex task and its design must be carefully planned in order to produce useful results. Many factors may influence evaluation design, some of which have been suggested in the literature. However, in a particular evaluation context, additional factors may arise, influencing and even compromising the evaluation design. We argue that preliminary evaluations must be performed before they are conducted in a real setting in order to verify if its design will provide the expected measurements and results. This paper presents the evaluation process of awareness information resources within software diagram peer reviews. The paper describes the evaluation context, objectives, plan and design together with enactment of a preliminary case study following this design. From this preliminary study, we outline issues that arose during its enactment that influenced refinement of the evaluation design.