Abstract The productivity of software developers is influenced by various factors encompassing technical, organizational, and individual aspects. Among these factors, mental health has emerged as a critical element for sustaining performance and well-being in this context. This work primary focus lies in understanding developers’ perceptions and acceptance of metrics related to mental health and productivity. Firstly, a mapping study was conducted to map the existing knowledge in this area. A total of 178 papers were retrieved using a predefined search string. After applying strict inclusion and exclusion criteria, five secondary studies were selected and 99 factors influencing productivity and/or mental health in the workplace were identified. Secondly, a metrics catalog was developed based on these influencing factors, combining established indicators from literature with new metrics designed to monitor developers’ mental health and productivity. The catalog includes 12 metrics along with their respective measurement formulas. Thirdly, an assessment of this catalog was performed through a survey and structured interviews with industry professionals, gathering insights on the applicability and relevance of the proposed metrics. The survey was completed by 47 software developers, 22 of whom also participated in the interviews. Our results reveal that while developers largely recognized the value of the catalog, resistance emerged when these metrics were positioned as evaluative mechanisms in the workplace.

Abstract Social coding platforms like GitHub facilitate collaborative software development through pull requests (PRs), which generate discussions that significantly impact code quality, requirements, and design. Such conversations become a rich source of insights for improving development practices and predicting project outcomes and are subject to several human aspects that have been linked to code quality and PR acceptance. Sentiment analysis is one of the many ways to try to understand these human aspects. However, PR discussions are multifaceted, often involving technical jargon and aspects which limits the utility of general-purpose sentiment analysis tools. This has led to the creation of SE-specific tools, but recent studies have also observed that they demonstrate limited effectiveness. Thus, this study explores the potential of using large language models (LLMs) for this purpose, given their enhanced contextual understanding and ability to process technical language. We evaluated ten LLMs across proprietary and open-source categories, using two complementary datasets: a curated Gold dataset and the PRemo dataset, which captures real-world PR discussions. The models were assessed under zero-shot, few-shot and chain-of-thought prompting techniques on 8,913 messages. In addition, we establish baselines by evaluating fine-tuned transformer-based models. Results show that GPT-4o achieved the highest overall performance across the LLMs, though smaller models, such as Mistral Small and Deepseek-R1 32B delivered competitive results. Transformer-based models achieved excellent performance on the Gold dataset but exhibited degradation on the PRemo dataset. Finally, we conducted a qualitative analysis of misclassified instances, revealing recurring challenges related to technical terminology, sentiment-charged keywords, message length, and contextual ambiguity. These findings suggest that model selection should balance performance requirements against practical constraints, rather than defaulting to the largest available models.

Open software development platforms, such as GitHub, foster developers’ collaboration in coding tasks through pull requests (PR). PRs serve as a mechanism for code contributions and structured discussions. These discussions often involve developers exchanging messages that contain questions. Despite the importance of questions, existing studies have not gathered insights on what are recurring types of questions in GitHub PR discussions. In this paper, we address this gap by leveraging the well-established 5W2H framework, which consists of seven questions classes. Our goal is to analyze the distribution of these question types within GitHub PRs and assess whether language models, specifically GPT-3.5 and GPT-4o, can accurately classify them. We conducted our study using a dataset derived from nine GitHub projects from the Netflix and Google ecosystems. Our findings reveal that both GPT-3.5 and GPT-4o perform well in identifying What and Who classes, which were the most frequently occurring question types in developers’ discussions. However, both models struggled with less common categories, failing to detect How Much, and showing inconsistencies in classifying Where, When, Why, and How. These findings suggest that improving classification accuracy may require incorporating domain-specific context and refining prompt engineering techniques.

Modern software systems are highly configurable, offering a vast number of configuration options that can be customized to meet specific functional and non-functional requirements. To support the configuration process, several automated software approaches based on machine learning have been proposed in the literature. These approaches aim to assist developers by predicting non-functional properties based on configuration settings. A recent study demonstrated the potential of leveraging a subset of configuration options (a.k.a. features) to achieve accurate performance predictions in the Linux kernel. The promise of learning over a reduced set of features – instead of all features – is to obtain performance models that are faster to compute, simpler to interpret, and still accurate. Despite the encouraging results of the original study, several questions remain unresolved: Can the findings be generalized to other configurable systems other than Linux? Which learning algorithms deliver the most efficient results when working with a reduced number of features? What are the most effective sampling strategies for building accurate and efficient models? In this work, we extend the original study by conducting an in-depth analysis across eight configurable systems. We evaluate the impact of sampling strategies and learning algorithms on model accuracy and training efficiency. Our goal is to understand whether there is a dominant sampling strategy and learning algorithm for varying systems and performance targets. Our results reveal variability in optimal strategies across systems and advocate for tailored approaches rather than universal solutions.

Modern software development relies on cloud-based collaborative platforms (e.g., GitHub and GitLab). In these platforms, developers often employ a pull-based development approach, proposing changes via pull requests and engaging in communication via asynchronous message exchanges. Since communication is key for software development, studies have linked different types of sentiments embedded in the communication to their effects on software projects, such as bug-inducing commits or the non-acceptance of pull requests. In this context, sentiment analysis tools are paramount to detect the sentiment of developers’ messages and prevent potentially harmful impact. Unfortunately, existing state-of-the-art tools vary in terms of the nature of their data collection and labeling processes. Yet, there is no comprehensive study comparing the performance and generalizability of existing tools utilizing a dataset that was designed and systematically curated to this end, and in this specific context. Therefore, in this study, we design a methodology to assess the effectiveness of existing sentiment analysis tools in the context of pull request discussions. For that, we created a dataset that contains ≈ 1.8K manually labeled messages from 36 software projects. The messages were labeled by 19 experts (neuroscientists and software engineers), using a novel and systematic manual classification process designed to reduce subjectivity. By applying these existing tools to the dataset, we observed that while some tools ]perform acceptably, their performance is far from ideal, especially when classifying negative messages. This is interesting since negative sentiment is often related to a critical or unfavorable opinion. We also observed that some messages have characteristics that can make them harder to classify, causing disagreements between the experts and possible misclassifications by the tools, requiring more attention from researchers. Our contributions include valuable resources to pave the way to develop robust and mature sentiment analysis tools that capture/anticipate potential problems during software development.

Code smells are symptoms in the source code that indicate code quality degradation and, consequently, may affect code comprehension and maintenance. Moreover, when two or more code smells occur on the same piece of code, forming an agglomeration, they may be more harmful to the code quality. Although the impact of smells in isolation is well known, the impact of their agglomeration is still underexplored. Our goal with this study is to provide evidence of how agglomerations impact code stability, i.e. we investigate if agglomeration suffers more modifications along the system evolution, and in which intensity. For this purpose, we mined two years of commit history from 30 open-source Java systems from GitHub. To analyze code stability, we considered four measurements: the number of commits, lines of modified code, rate of modified classes, and the proportion of changes. We examined these measurements from two perspectives: by system and by aggregating all system data. Additionally, we further considered how a class created/deleted in this time span impacts our results. Our main findings are: (i) classes with two or more code smells of different types change more frequently and in more intensity than classes with a single smell or no smell; (ii) the stability of the class varies greatly with the system under analysis; (iii) when a smelly class was deleted in our time range, they usually had several lines of code added until it became unsustainable. We can conclude that agglomerations change with more frequency and intensity, raising maintenance and evolution costs. Consequently, this information can be used to prioritize code refactoring.

Robustness is critical for ensuring that software functions correctly under adverse conditions. Exception-handling mechanisms in programming languages enable developers to deal with these adverse conditions. However, implementing exception-related code can present significant challenges to developers. We investigated exception-related code contributions across Java projects in the Apache ecosystem. We analyzed exception-related pull requests (exception-PRs), which were detected using a validated heuristic. We produced a comprehensive dataset of 988 exception-PRs. We observed no statistically significant differences in complexity metrics between exception-PRs and non-exception-PRs. We also found no significant differences in developers' behavior metrics, indicating consistent engagement regardless of whether the pull request addressed exception-related code. A manual analysis revealed that most exception-PRs focused on system improvements rather than bug fixes, suggesting proactive efforts to enhance software robustness. Moreover, the most frequently addressed aspects of exceptional code in these exception-PRs were: (i) the external representation of adverse situations to end-users (more than 40% of the PRs) and (ii) the implementation of effective error-handling actions (nearly 35% of the PRs) to promote program recoverability. Interestingly, a significant proportion of exception-PRs simultaneously addressed multiple aspects. By understanding the nature and characteristics of exception-PRs, we expect to better support developers in managing erroneous conditions and improving software robustness.

The modernization of monolithic legacy systems with microservices has been a trend in recent years. As part of this modernization, identifying microservice candidates starting from legacy code is challenging, as maintainers may consider many criteria simultaneously. Multi-objective search-based approaches represent a promising state-of-the-art solution to support this decision-making process. However, the rationale to adopt each microservice candidate automatically identified by these approaches is poorly investigated in industrial cases. Furthermore, studies with these approaches have not carefully investigated how maintainers reason and make decisions when designing microservice architectures from legacy systems. To address this gap, we conducted an on-site case study with maintainers of an industrial legacy system to investigate the usefulness of automatically generated microservice architectures. We analyze design decisions pointed out by the maintainers when reasoning about microservice candidates using several criteria at the same time. Our study is the first to assess a search-based approach involving actual maintainers conceiving microservice architectures in an industrial setting. Therefore, firstly, we considered individual evaluation of microservice candidates to understand the rationale for identifying a service. Secondly, we conducted a focus group study with maintainers with the goal of investigating design decisions at an architectural level. The results show that:(i)the automated approach is able to identify useful microservices;(ii)the criteria observed by previous studies are, in fact, considered by maintainers; and(iii)the maintainer profiles, i.e., the preferred granularity for microservice, highly affect design decisions. Finally, we observed the maintainers needed little effort in adjusting the automatically identified microservices to make them adoptable. In addition to indicating a promising potential of search-based approaches to generate microservice architectures, our findings highlight the need for:(i)interactive and/or customizable approaches that enable maintainers to include their preferences during the search process, and(ii)flexible or automated selection of criteria that fits the scenario in which the modernization is taking place.

Modern programming languages, such as Java, use exception-handling mechanisms to guarantee the robustness of software systems. Although important, the quality of exception code is usually poor and neglected by developers. Indiscriminate robustness changes (e.g., the addition of empty catch blocks) can indicate design decisions that negatively impact the internal quality of software systems. As it is known in the literature, multiple occurrences of poor code structures, namely code smells, are strong indicators of design problems. Still, existing studies focus mainly on the correlation of maintainability smells with design problems. However, using only these smells may not be enough since developers need more context (e.g., system domain) to identify the problems in certain scenarios. Moreover, these studies do not explore how changes in the exceptional code of the methods combined with maintainability smells can give complementary evidence of design problems. By covering both regular and exception codes, the developer can have more context about the system and find complementary code smells that reinforce the presence of design problems. This work aims to leverage the identification of design problems by tracking poor robustness changes combined with maintainability smells. We investigated the correlation between robustness changes and maintainability smells on the commit history of more than 160k methods from different releases of 10 open-source software systems. We observed that maintainability smells can be worsened or even introduced when robustness changes are performed. This scenario mainly happened for the smells Feature Envy, Long Method, and Dispersed Coupling. We also analyzed the co-occurrence between robustness and maintainability smells. We identified that the empty catch block and catch throwable robustness smells were the ones that co-occurred the most with maintainability smells related to the Concern Overload and Misplaced Concern design problems. The contribution of our work is to reveal that poor exception code, usually neglected by developers, negatively impacts the quality of methods and classes, signaled by the maintainability smells. Therefore, existing code smell detecting tools can be enhanced to leverage robustness changes to identify design problems.

Developers constantly perform code changes throughout the lifetime of a project. These changes may induce the introduction of design problems (design decay) over time, which may be reduced or accelerated by interacting with different factors (e.g., refactorings) that underlie each change. However, existing studies lack evidence about how these factors interact and influence design decay. Thus, this paper reports a study aimed at investigating whether and how (associations of) process and developer factors influence design decay. We studied seven software systems, containing an average of 45K commits in more than six years of project history. Design decay was characterized in terms of five internal quality attributes: cohesion, coupling, complexity, inheritance, and size. We observed and characterized 12 (sub-)factors and how they associate with design decay. To this end, we employed association rule mining. Moreover, we also differentiate between the associations found on modules with varying levels of decay. Process- and developer-related factors played a key role in discriminating these different levels of design decay. Then, we focused on analyzing the effects of potentially interacting factors regarding slightly- and largely-decayed modules. Finally, we observed diverging decay patterns in these modules. For example, individually, the developer-related sub-factor that represented first-time contributors, as well as the process-related one that represented the size of a change did not have negative effects on the changed classes. However, when analyzing specific factor interactions, we saw that changes in which both of these factors interacted tended to have a negative effect on the code, leading to decay.

With large scale and complex configurable systems, it is hard for users to choose the right combination of options (i.e., configurations) in order to obtain the wanted trade-off between functionality and performance goals such as speed or size. Machine learning can help in relating these goals to the configurable system options, and thus, predict the effect of options on the outcome, typically after a costly training step. However, many configurable systems evolve at such a rapid pace that it is impractical to retrain a new model from scratch for each new version. In this paper, we propose a new method to enable transfer learning of binary size predictions among versions of the same configurable system. Taking the extreme case of the Linux kernel with its$\approx 14,500$configuration options, we first investigate how binary size predictions of kernel size degrade over successive versions. We show that the direct reuse of an accurate prediction model from 2017 quickly becomes inaccurate when Linux evolves, up to a 32% mean error by August 2020. We thus propose a new approach for transfer evolution-aware model shifting (tEAMS). It leverages the structure of a configurable system to transfer an initial predictive model towards its future versions with a minimal amount of extra processing for each version. We show thattEAMSvastly outperforms state of the art approaches over the 3 years history of Linux kernels, from 4.13 to 5.8.

Companies have adopted modern code review as a key technique for continuously monitoring and improving the quality of software changes. One of the main motivations for this is the early detection of design impactful changes, to prevent that design-degrading ones prevail after each code review. Even though design degradation symptoms often lead to changes' rejections, practices of modern code review alone are actually not sufficient to avoid or mitigate design decay. Software design degrades whenever one or more symptoms of poor structural decisions, usually represented by smells, end up being introduced by a change. Design degradation may be related to both technical and social aspects in collaborative code reviews. Unfortunately, there is no study that investigates if code review stakeholders, e.g, reviewers, could benefit from approaches to distinguish and predict design impactful changes with technical and/or social aspects. By analyzing 57,498 reviewed code changes from seven open-source systems, we report an investigation on prediction of design impactful changes in modern code review. We evaluated the use of six ML algorithms to predict design impactful changes. We also extracted and assessed 41 different features based on both social and technical aspects. Our results show that Random Forest and Gradient Boosting are the best algorithms. We also observed that the use of technical features results in more precise predictions. However, the use of social features alone, which are available even before the code review starts (e.g., for team managers or change assigners), also leads to highly-accurate prediction. Therefore social and/or technical prediction models can be used to support further design inspection of suspicious changes early in a code review process. Finally, we provide an enriched dataset that allows researchers to investigate the context behind design impactful changes during the code review process.

Microservices are small and autonomous services that communicate through lightweight protocols. Companies have often been adopting microservices to incrementally redesign legacy systems as part of a modernization process. Microservices promote better reuse and customization of existing features while increasing business capabilities, if appropriate design decisions are made. There are some partially-automated approaches supporting the re-design of legacy features into microservices. However, they fail in covering two key aspects: (i) provide an architectural design of the features being redesigned, and (ii) simultaneously support relevant criteria, e.g., feature modularization and decrease of network communication overhead. Also, these two aspects tend to be poorly discussed along industrial case studies. To fulfill these gaps, we propose a redesign strategy to support the re-engineering of features legacy code as microservices. This strategy covers key possibly-conflicting criteria on microservice-based architectures. We employ search-based optimization to deal with such conflicting criteria. The output of the strategy is a set of redesign candidates of legacy features as microservices. We reflect upon the benefits and drawbacks of the proposed strategy through an industrial case study. In particular, we perform an in-depth analysis of the resulting microservice candidates, and a discussion about their potential for customization and reuse. The reflections/discussions are also supported by observations of developers involved in the process.

Petrobras is Brazil's largest publicly-held company, operating in the oil, natural gas, and energy industry. Internal efforts enabled Petrobras to identify Digital Transformation (DT) opportunities to further promote their operational excellence. While addressing these opportunities typically requires Research and Development (R&D) uncertainties that could lead traditional R&D cooperation terms to be negotiated in years, there are time-to-market constraints for fast-paced deliveries to experiment solution options. Having this in mind, they partnered up with PUC-Rio to establish a new DT initiative. [Goal] The goal of this paper is to present the Lean R&D approach, tailored within the new initiative to meet the aforementioned DT needs. [Method] We designed Lean R&D integrating the following building blocks: (i) Lean Inceptions, to allow stakeholders to jointly outline a Minimal Viable Product (MVP); (ii) parallel technical feasibility assessment and conception phases, allowing to `fail fast'; (iii) scrum-based development management; and (iv) strategically aligned continuous experimentation to test business hypotheses. We report on first experiences of applying Lean R&D in practice. [Results] Lean R&D enabled addressing research-related uncertainties early and to efficiently deliver valuable MVPs within fast-paced four months cycles. [Conclusions] In our first experiences Lean R&D showed itself suitable for supporting DT initiatives. However, more formal case studies are needed. The business strategy alignment and the continuous support of a highly qualified research team were considered key success factors.

The expensive maintenance of legacy systems leads companies to migrate such systems to microservice architectures. This migration requires the identification of system's legacy parts to become microservices. However, the successful identification of microservices, which are promising to be adoptable in practice, requires the simultaneous satisfaction of many criteria, such as coupling, cohesion, reuse and communication overhead. Search-based microservice identification has been recently investigated to address this problem. However, state-of-the-art search-based approaches are limited as they only consider one or two criteria (namely cohesion and coupling), possibly not fulfilling the practical needs of developers. To overcome these limitations, we propose toMicroservices, a many-objective search-based approach that considers five criteria, the most cited by practitioners in recent studies. Our approach was evaluated in a real-life industrial legacy system undergoing a microservice migration process. The performance of toMicroservices was quantitatively compared to a baseline. We also gathered qualitative evidence based on developers' perceptions, who judged the adoptability of the recommended microservices. The results show that our approach is both: (i) very similar to the most recent proposed approach on optimizing the traditional criteria of coupling and cohesion, but (ii) much better when taking into account all the five criteria. Finally, most of the microservice candidates were considered adoptable by practitioners.

Numerous software systems are highly configurable and provide a myriad of configuration options that users can tune to fit their functional and performance requirements (e.g., execution time). Measuring all configurations of a system is the most obvious way to understand the effect of options and their interactions, but is too costly or infeasible in practice. Numerous works thus propose to measure only a few configurations (a sample) to learn and predict the performance of any combination of options' values. A challenging issue is to sample a small and representative set of configurations that leads to a good accuracy of performance prediction models. A recent study devised a new algorithm, called distance-based sampling, that obtains state-of-the-art accurate performance predictions on different subject systems. In this paper, we replicate this study through an in-depth analysis of x264, a popular and configurable video encoder. We systematically measure all 1,152 configurations of x264 with 17 input videos and two quantitative properties (encoding time and encoding size). Our goal is to understand whether there is a dominant sampling strategy over the very same subject system (x264), i.e., whatever the workload and targeted performance properties. The findings from this study show that random sampling leads to more accurate performance models. However, without considering random, there is no single "dominant" sampling, instead different strategies perform best on different inputs and non-functional properties, further challenging practitioners and researchers.

The Software Product Line (SPL) configuration field is an active area of research and has attracted both practitioners and researchers attention in the last years. A key part of an SPL configuration is a feature model that represents features and their dependencies (i.e., SPL configuration rules). This model can be extended by adding Non-Functional Properties (NFPs) as feature attributes resulting in Extended Feature Models (EFMs). Configuring products from an EFM requires considering the configuration rules of the model and satisfying the product functional and non-functional requirements. Although the configuration of a product arising from EFMs may reduce the space of valid configurations, selecting the most appropriate set of features is still an overwhelming task due to many factors including technical limitations and diversity of contexts. Consequently, configuring large and complex SPLs by using configurators is often beyond the users' capabilities of identifying valid combinations of features that match their (non-functional) requirements. To overcome this limitation, several approaches have modeled the product configuration task as a combinatorial optimization problem and proposed constraint programming algorithms to automatically derive a configuration. Although these approaches do not require any user intervention to guarantee the optimality of the generated configuration, due to the NP-hard computational complexity of finding an optimal variant, exact approaches have inefficient exponential time. Thus, to improve scalability and performance issues, we introduced the adoption of a greedy heuristic algorithm and a biased random-key genetic algorithm (BRKGA). Our experiment results show that our proposed heuristics found optimal solutions for all instances where those are known. For the instances where optimal solutions are not known, the greedy heuristic outperformed the best solution obtained by a one-hour run of the exact algorithm by up to 67.89%. Although the BRKGA heuristic slightly outperformed the greedy heuristic, it has shown larger running times (especially on the largest instances). Therefore, to ensure a good user experience and enable a very fast configuration task, we extended a state-of-the-art configurator with the proposed greedy heuristic approach.

Dynamic software product lines demand self-adaptation of their behavior to deal with runtime contextual changes in their environment and offer a personalized product to the user. However, taking user preferences and context into account impedes the manual configuration process, and thus, an efficient and automated procedure is required. To automate the configuration process, context-aware recommendation techniques have been acknowledged as an effective mean to provide suggestions to a user based on their recognized context. In this work, we propose a collaborative filtering method based on tensor factorization that allows an integration of contextual data by modeling an N-dimensional tensor User-Feature-Context instead of the traditional two-dimensional User-Feature matrix. In the proposed approach, different types of non-functional properties are considered as additional contextual dimensions. Moreover, we show how to self-configure software product lines by applying our N-dimensional tensor factorization recommendation approach. We evaluate our approach by means of an empirical study using two datasets of configurations derived for medium-sized product lines. Our results reveal significant improvements in the predictive accuracy of the configuration over a state-of-the-art non-contextual matrix factorization approach. Moreover, it can scale up to a 7-dimensional tensor containing hundred of configurations in a couple of milliseconds.

In the last decades, software product lines (SPL) have proven to be an efficient software development technique in industries due its capability to increase quality and productivity and decrease cost and time-to-market through extensive reuse of software artifacts. To achieve these benefits, tool support is fundamental to guide industries during the SPL development life-cycle. However, many different SPL tools are available nowadays and the adoption of the appropriate tool is a big challenge in industries. In order to support engineers choosing a tool that best fits their needs, this paper presents the results of a controlled empirical study to assess two Eclipse-based tools, namely FeatureIDE and pure::variants. This empirical study involved 84 students who used and evaluated both tools. The main weakness we observe in both tools are the lack adequate mechanisms for managing the variability, such as for product configuration. As a strength, we observe the automated analysis and the feature model editor.

Today’s competitive marketplace requires the industry to understand unique and particular needs of their customers. Product line practices enable companies to create individual products for every customer by providing an interdependent set of features. Users configure personalized products by consecutively selecting desired features based on their individual needs. However, as most features are interdependent, users must understand the impact of their gradual selections in order to make valid decisions. Thus, especially when dealing with large feature models, specialized assistance is needed to guide the users in configuring their product. Recently, recommender systems have proved to be an appropriate mean to assist users in finding information and making decisions. In this paper, we propose an advanced feature recommender system that provides personalized recommendations to users. In detail, we offer four main contributions: (i) We provide a recommender system that suggests relevant features to ease the decision-making process. (ii) Based on this system, we provide visual support to users that guides them through the decision-making process and allows them to focus on valid and relevant parts of the configuration space. (iii) We provide an interactive open-source configurator tool encompassing all those features. (iv) In order to demonstrate the performance of our approach, we compare three different recommender algorithms in two real case studies derived from business experience.

Open online courses are a method of online lecturing whose application in education is not bounded by space and location constraints. The successful implementation of open courses requires conceptual changes in how instructors and students behave in open unbounded education environment. There are some emerging open courses for teaching specific topics of Software Engineering. However, it is still limited the knowledge about the best practices for learning Software Engineering processes, methods, and tools in such an open environment. To address this limitation, this paper presents and evaluates an open course for Introduction to Software Engineering. The presented open course has over 250 online students registered and is based on a face-to-face equivalent. The online course is currently composed of 44 video lectures, 160 questions in 16 quizzes, and several discussion topics. We evaluate this course by comparing the students' performance in online vs. face-to-face equivalent courses. Our results indicated that students who had access to online content achieve similar or better performance than students taking only the face-to-face course.