COVID-19 accelerated the digital-health market, resetting user expectations at a pace that makes traditional release planning brittle. In partnership with Curatio, we conducted a 12-month longitudinal case study to evaluate and improve the release process for their flagship app, Stronger Together . First, we mined public app store data to map the product space and quantify the reuse value of 134 candidate features. Next, we formulated a bi-objective optimization that maximizes reuse value while preserving feature cohesion, yielding a data-driven “Plan K” for the next release. Finally, we compared this plan with the two releases Curatio shipped, combining artifact analysis with semi-structured interviews. Five of the seven recommended features were adopted, capturing 80% of the predicted reuse value; two high-impact items were deferred because of technical and regulatory constraints. The evaluation identifies four recurrent “release traps” that explain why theoretically optimal plans can fail in practice and demonstrates how good-enough, continuously replanned roadmaps can deliver most of the benefits with significantly less risk. The study contributes a replicable appraisal method for feature-reuse decisions and offers actionable guidance for industrial teams facing the same balance of legacy maintenance, rapid innovation, and market uncertainty.

As Artificial Intelligence (AI) becomes increasingly embedded in healthcare technologies, understanding the maturity of AI in patient-centric applications is critical for evaluating its trustworthiness, transparency, and real-world impact. In this study, we investigate the integration and maturity of AI feature integration in 116 patient-centric healthcare applications. Using Large Language Models (LLMs), we extracted key functional features, which are then categorized into different stages of the Gartner AI maturity model. Our results show that over 86.21% of applications remain at the early stages of AI integration, while only 13.79% demonstrate advanced AI integration.

Software documentation provides guidance on the proper use of tools or services. With the rapid growth of machine learning libraries, individuals from various fields are incorporating machine learning into their workflows through programming. However, many of these users lack software engineering experience, affecting the usability of the documentation. Traditionally, software developers have created documentation primarily for their peers, making it challenging for others to interpret and effectively use these resources. Moreover, no study has specifically focused on machine learning software documentation or on analyzing the backgrounds of developers who rely on such documentation, highlighting a critical gap in understanding how to make these resources more accessible. In this study, we examined customization trends in TensorFlow tutorials and compared these artifacts to analyze content and design differences. We also analyzed Stack Overflow questions related to TensorFlow documentation to understand the types of questions and the backgrounds of the developers asking them. Further, we developed two taxonomies based on the nature and triggers of the questions for machine learning software.Our findings showed no significant differences in the content or the nature of the questions across different tutorials. Our results show that 24.9% of the questions concern errors and exceptions, while 64.3% relates to inadequate and non-generalizable examples in the documentation. Despite efforts to create customized documentation, our analysis indicates that current TensorFlow documentation does not effectively support its target users.

An academic contribution to computer science becomes impactful when incorporated into a real software project. For machine learning (ML), open-source frameworks facilitate researchers to exploit and share their research output with other researchers and practitioners. However, such contributionsas other changes-need to be properly reviewed. This paper reports preliminary findings of an investigation conducted on Tensorflow aimed at analyzing how contributions originating from scientific articles are reviewed and how such a review process compares with code review of conventional software systems. We have quantitatively and qualitatively analyzed 16 cases in which ideas/solutions from articles made into TensorFlow after a pull request review, investigating (i) the nature of pull request review comments, (ii) the role of the reviewer, and (iii) the artifacts being reviewed or shared during the review process. The results show how, in line with previous investigations on the development process of ML systems, the code review process involves the interaction of data scientists and academics with software developers. Also, it interleaves phases assessing the scientific merits and compatibility of the article's solution with conventional code review focused on code readability and maintainability issues.

The integration of generative AI into developer forums like Stack Overflow presents an opportunity to enhance problem-solving by allowing users to post screenshots of code or Integrated Development Environments (IDEs) instead of traditional text-based queries. This study evaluates the effectiveness of various large language models (LLMs)—specifically LLAMA, GEMINI, and GPT-4o in interpreting such visual inputs. We employ prompt engineering techniques, including in-context learning, chain-of-thought prompting, and few-shot learning, to assess each model’s responsiveness and accuracy. Our findings show that while GPT-4o shows promising capabilities, achieving over $60 \%$ similarity to baseline questions for $51.75 \%$ of the tested images, challenges remain in obtaining consistent and accurate interpretations for more complex images. This research advances our understanding of the feasibility of using generative AI for image-centric problem-solving in developer communities, highlighting both the potential benefits and current limitations of this approach while envisioning a future where visual-based debugging copilot tools become a reality.

The rapid advancements in large language models (LLMs) have greatly expanded the potential for automated code-related tasks. Two primary methodologies are used in this domain: prompt engineering and fine-tuning. Prompt engineering involves applying different strategies to query LLMs, like Chat-GPT, while fine-tuning further adapts pre-trained models, such as CodeBERT, by training them on task-specific data. Despite the growth in the area, there remains a lack of comprehensive comparative analysis between the approaches for code models. In this paper, we evaluate GPT-4 using three prompt engineering strategies-basic prompting, in-context learning, and taskspecific prompting-and compare it against 17 fine-tuned models across three code-related tasks: code summarization, generation, and translation. Our results indicate that GPT-4 with prompt engineering does not consistently outperform fine-tuned models. For instance, in code generation, GPT-4 is outperformed by finetuned models by 28.3% points on the MBPP dataset. It also shows mixed results for code translation tasks. Additionally, a user study was conducted involving 27 graduate students and 10 industry practitioners. The study revealed that GPT-4 with conversational prompts, incorporating human feedback during interaction, significantly improved performance compared to automated prompting. Participants often provided explicit instructions or added context during these interactions. These findings suggest that GPT-4 with conversational prompting holds significant promise for automated code-related tasks, whereas fully automated prompt engineering without human involvement still requires further investigation.

ABSTRACT Background The trend of sharing images and image‐based social networks has eventually changed the landscape of social networks. Objective This study focuses on three primary objectives: (i) identifying issue reports that benefit from image sharing and processing in Bugzilla, (ii) identifying the type of image that would improve the bug report, and (iii) conducting a comprehensive qualitative and quantitative evaluation of the tool's performance and impact. Methods We trained machine‐learning and deep‐learning models on a dataset of 34,540 Bugzilla issue reports. The results are evaluated using quantitative performance metrics and a qualitative survey. Results Our prediction method achieves an F1‐score of 0.77 and about 75% of participants found it practically useful in our study. Conclusion This study and its associated dataset and methodology represent the first research on recommending images to developers for enhanced issue report communication. Our results illuminate a promising trajectory for enhanced and visual productivity tools for developers.

In the rapidly evolving landscape of developer communities, Q&A platforms serve as crucial resources for crowdsourcing developers’ knowledge. A notable trend is the increasing use of images to convey complex queries more effectively. However, the current state-of-the-art method of duplicate question detection has not kept pace with this shift, which predominantly concentrates on text-based analysis. Inspired by advancements in image processing and numerous studies in software engineering illustrating the promising future of image-based communication on social coding platforms, we delved into image-based techniques for identifying duplicate questions on Stack Overflow. When focusing solely on text analysis of Stack Overflow questions and omitting the use of images, our automated models overlook a significant aspect of the question. Previous research has demonstrated the complementary nature of images to text. To address this, we implemented two methods of image analysis: first, integrating the text from images into the question text, and second, evaluating the images based on their visual content using image captions. After a rigorous evaluation of our model, it became evident that the efficiency improvements achieved were relatively modest, approximately an average of 1%. This marginal enhancement falls short of what could be deemed a substantial impact. As an encouraging aspect, our work lays the foundation for easy replication and hypothesis validation, allowing future research to build upon our approach and explore novel solutions for more effective image-driven duplicate question detection.

Context: GitHub, renowned for facilitating collaborative code version control and software production in software teams, expanded its services in 2017 by introducing GitHub Marketplace. This online platform hosts automation tools to assist developers with the production of their GitHub-hosted projects, and it has become a valuable source of information on the tools used in the Open Source Software (OSS) community. Objective: In this exploratory study , we introduce GitHub Marketplace as a software marketplace by exploring the Characteristics, Features, and Policies of the platform comprehensively, identifying common themes in production automation. Further, we explore popular tools among practitioners and researchers and highlight disparities in the approach to these tools between industry and academia. Method: We adopted the conceptual framework of software app stores from previous studies and used that to examine 8,318 automated production tools (440 Apps and 7,878 Actions) across 32 categories on GitHub Marketplace. We explored and described the policies of this marketplace as a unique platform where developers share production tools for the use of other developers. Furthermore, we conducted a systematic mapping of 515 research papers published from 2000 to 2021 and compared open-source academic production tools with those available in the marketplace. Results: We found that although some of the automation topics in literature are widely used in practice, they have yet to align with the state-of-practice for automated production. We discovered that practitioners often use automation tools for tasks like “Continuous Integration” and “Utilities”, while researchers tend to focus more on “Code Quality” and “Testing”. Conclusion: Our study illuminates the landscape of open-source tools for automation production. We also explored the disparities between industry trends and researchers’ priorities. Recognizing these distinctions can empower researchers to build on existing work and guide practitioners in selecting tools that meet their specific needs. Bridging this gap between industry and academia helps with further innovation in the field and ensures that research remains pertinent to the evolving challenges in software production.

Abstract Visual content in the form of images and videos has taken over general‐purpose social networks in a variety of ways, streamlining and enriching online communications. We are interested to understand if and to what extent the use of images is popular and helpful in social coding platforms. We mined 9 years of data from two popular software developers' platforms: the Mozilla issue tracking system, that is, Bugzilla, and the most well‐known platform for developers' Q/A, that is, Stack Overflow. We further triangulated and extended our mining results by performing a survey with 168 software developers. We observed that, between 2013 and 2022, the number of posts containing image data on Bugzilla and Stack Overflow doubled. Furthermore, we found that sharing images makes other developers engage more and faster with the content. In the majority of cases in which an image is included in a developer's post, the information in that image is complementary to the text provided. Finally, our results showed that when an image is shared, understanding the content without the information in the image is unlikely for 86.9% of the cases. Based on these observations, we discuss the importance of considering visual content when analyzing developers and designing automation tools.

GitHub Actions is a powerful tool for automating workflows on GitHub repositories, with thousands of Actions currently available on the GitHub Marketplace. So far, the research community has conducted mining studies on Actions, with much of the focus on CI/CD. However, the motivation and best practices of developers for using, developing, and debugging Actions are unknown. To address this gap, we conducted a survey study with 90 Action users and developers. Our findings indicate that developers prefer Actions with verified creators and more stars when choosing between similar Actions, and often switch to alternative Actions when faced with bugs or a lack of documentation. We also found that developers find the composition of YAML files, which are essential for Action integration, challenging and error-prone. They primarily rely on Q&A forums to fix issues with these YAML files. Finally, we observed that developers would not likely adopt Actions when there are concerns around complexity and security risks. Our study summarizes developers’ perceptions, decision-making process, and challenges in using, developing, and debugging Actions. We provide recommendations for improving the visibility, re-usability, documentation, and support surrounding GitHub Actions.

Evolving software with an increasing number of features is harder to understand and thus harder to use. Software release planning has been concerned with planning these additions. Moreover, software of increasing size takes more effort to be maintained. In the domain of mobile apps, too much functionality can easily impact usability, maintainability, and resource consumption. Hence, it is important to understand the extent to which the law of continuous growth applies to mobile apps. Previous work showed that the deletion of functionality is common and sometimes driven by user reviews. However, it is unknown whether these deletions are visible or important to the app users. In this study, we surveyed 297 mobile app users to understand the significance of functionality deletion for them. Our results showed that for most users, the deletion of features corresponds with negative sentiments and change in usage and even churn. Motivated by these preliminary results, we propose Radiation to input user reviews and recommend if any functionality should be deleted from an app's User Interface (UI). We evaluate Radiation using historical data and surveying developers' opinions. From the analysis of 190,062 reviews from 115 randomly selected apps, we show that Radiation can recommend functionality deletion with an average F-Score of 74% and if sufficiently many negative user reviews suggest so.

The COVID−19 pandemic has accelerated the adoption of digital health solutions. This has presented significant challenges for software development teams to swiftly adjust to the market needs and demand. To address these challenges, product management teams have had to adapt their approach to software development, reshaping their processes to meet the demands of the pandemic. Brighsquid implemented a new task assignment process aimed at enhancing developer accountability toward the customer. To assess the impact of this change on code ownership, we conducted a code change analysis. Additionally, we surveyed 67 developers to investigate the relationship between accountability and ownership more broadly. The findings of our case study indicate that the revised assignment model not only increased the perceived sense of accountability within the production team but also improved code resilience against ownership changes. Moreover, the survey results revealed that a majority of the participating developers (67.5%) associated perceived accountability with artifact ownership.

Background: Code reviewing is an essential part of software development to ensure software quality. However, the abundance of review tasks and the intensity of the workload for reviewers negatively impact the quality of the reviews. The short review text is often unactionable. Aims: We propose the Example Driven Review Explanation (EDRE) method to facilitate the code review process by adding additional explanations through examples. EDRE recommends similar code reviews as examples to further explain a review and help a developer to understand the received reviews with less communication overhead. Method: Through an empirical study in an industrial setting and by analyzing 3,722 Code reviews across three open-source projects, we compared five methods of data retrieval, text classification, and text recommendation. Results: EDRE using TF-IDF word embedding along with an SVM classifier can provide practical examples for each code review with 92% F-score and 90% Accuracy. Conclusions: The example-based explanation is an established method for assisting experts in explaining decisions. EDRE can accurately provide a set of context-specific examples to facilitate the code review process in software teams.

Together with many success stories, promises such as the increase in production speed and the improvement in stakeholders’ collaboration have contributed to making agile a transformation in the software industry in which many companies want to take part. However, driven either by a natural and expected evolution or by contextual factors that challenge the adoption of agile methods as prescribed by their creator(s), software processes in practice mutate into hybrids over time. Are these still agile? In this article, we investigate the question: what makes a software development method agile? We present an empirical study grounded in a large-scale international survey that aims to identify software development methods and practices that improve or tame agility. Based on 556 data points, we analyze the perceived degree of agility in the implementation of standard project disciplines and its relation to used development methods and practices. Our findings suggest that only a small number of participants operate their projects in a purely traditional or agile manner (under 15 percent). That said, most project disciplines and most practices show a clear trend towards increasing degrees of agility. Compared to the methods used to develop software, the selection of practices has a stronger effect on the degree of agility of a given discipline. Finally, there are no methods or practices that explicitly guarantee or prevent agility. We conclude that agility cannot be defined solely at the process level. Additional factors need to be taken into account when trying to implement or improve agility in a software company. Finally, we discuss the field of software process-related research in the light of our findings and present a roadmap for future research.

Machine Learning software documentation is different from most of the documentations that were studied in software engineering research. Often, the users of these documentations are not software experts. The increasing interest in using data science and in particular, machine learning in different fields attracted scientists and engineers with various levels of knowledge about programming and software engineering. Our ultimate goal is automated generation and adaptation of machine learning software documents for users with different levels of expertise. We are interested in understanding the nature and triggers of the problems and the impact of the users' levels of expertise in the process of documentation evolution. We will investigate the Stack Overflow Q&As and classify the documentation related Q/As within the machine learning domain to understand the types and triggers of the problems as well as the potential change requests to the documentation. We intend to use the results for building on top of the state of the art techniques for automatic documentation generation and extending on the adoption, summarization, and explanation of software functionalities.

Requirements Engineering (RE) has established itself as a software engineering discipline over the past decades. While researchers have been investigating the RE discipline with a plethora of empirical studies, attempts to systematically derive an empirical theory in context of the RE discipline have just recently been started. However, such a theory is needed if we are to define and motivate guidance in performing high quality RE research and practice. We aim at providing an empirical and externally valid foundation for a theory of RE practice, which helps software engineers establish effective and efficient RE processes in a problem-driven manner. We designed a survey instrument and an engineer-focused theory that was first piloted in Germany and, after making substantial modifications, has now been replicated in 10 countries worldwide. We have a theory in the form of a set of propositions inferred from our experiences and available studies, as well as the results from our pilot study in Germany. We evaluate the propositions with bootstrapped confidence intervals and derive potential explanations for the propositions. In this article, we report on the design of the family of surveys, its underlying theory, and the full results obtained from the replication studies conducted in 10 countries with participants from 228 organisations. Our results represent a substantial step forward towards developing an empirical theory of RE practice. The results reveal, for example, that there are no strong differences between organisations in different countries and regions, that interviews, facilitated meetings and prototyping are the most used elicitation techniques, that requirements are often documented textually, that traces between requirements and code or design documents are common, that requirements specifications themselves are rarely changed and that requirements engineering (process) improvement endeavours are mostly internally driven. Our study establishes a theory that can be used as starting point for many further studies for more detailed investigations. Practitioners can use the results as theory-supported guidance on selecting suitable RE methods and techniques.

Maximizing satisfaction from offering features as part of the upcoming release(s) is different from minimizing dissatisfaction gained from not offering features. This asymmetric behavior has never been utilized for product release planning. We study Asymmetric Release Planning (ARP) by accommodating asymmetric feature evaluation. We formulated and solved ARP as a bi-criteria optimization problem. In its essence, it is the search for optimized trade-offs between maximum stakeholder satisfaction and minimum dissatisfaction. Different techniques including a continuous variant of Kano analysis are available to predict the impact on satisfaction and dissatisfaction with a product release from offering or not offering a feature. As a proof of concept,we validated the proposed solution approach called Satisfaction-Dissatisfaction Optimizer (SDO) via a real-world case study project. From running three replications with varying effort capacities, we demonstrate that SDO generates optimized trade-off solutions being (i) of a different value profile and different structure, (ii) superior to the application of random search and heuristics in terms of quality and completeness, and (iii) superior to the usage of manually generated solutions generated from managers of the case study company. A survey with 20 stakeholders evaluated the applicability and usefulness of the generated results.

One of Lehman's laws of software evolution is that the functionality of programs has to increase over time to maintain user satisfaction. In the domain of mobile apps, though, too much functionality can easily impact usability, resource consumption, and maintenance effort. Hence, does the law of continuous growth apply there? This paper shows that in mobile apps, deletion of functionality is actually common, challenging Lehman's law. We analyzed user driven requests for deletions which were found in 213,866 commits from 1,519 open source Android mobile apps from a total of 14,238 releases. We applied hybrid (open and closed) card sorting and created taxonomies for nature and causes of deletions. We found that functionality deletions are mostly motivated by unneeded functionality, poor user experience, and compatibility issues. We also performed a survey with 106 mobile app developers. We found that 78.3% of developers consider deletion of functionality to be equally or more important than the addition of new functionality. Developers confirmed that they plan for deletions. This implies the need to re-think the process of planning for the next release, overcoming the simplistic assumptions to exclusively look at adding functionality to maximize the value of upcoming releases. Our work is the first to study the phenomenon of functionality deletion and opens the door to a wider perspective on software evolution.

The goal of RE Cares is to apply our requirements engineering and design and prototyping skills to a problem of societal importance to stakeholders residing in the RE conference locale.

This paper outlines the objectives, structure, and panelists of the data driven requirements engineering panel. This panel is intended to address challenges and opportunities that requirements engineering researchers face in accessing data.

Deciding about the content and readiness when shipping a new product release can have a strong impact on the success (or failure) of the product. Having formerly analyzed the state-of-the art in this area, the objective for this paper was to better understand the process and rationale of real-world release decisions and to what extent research on release readiness is aligned with industrial needs. We designed two rounds of surveys with focus on the current (Survey-A) and the desired (Survey-B) process of how to make release readiness decisions. We received 49 and 40 valid responses for Survey-A and Survey-B, respectively.

Background: Understanding and controlling the impact of change decides about the success or failure of evolving products. The problem magnifies for start-ups operating with limited resources. Their usual focus is on Minimum Viable Product (MVP's) providing specialized functionality, thus have little expense available for handling changes. Aims: Change Impact Analysis (CIA) refers to the identification of source code files impacted when implementing a change request. We extend this question to predict not only affected files, but also the effort needed for implementing the change, and the duration necessary for that. Method: This study evaluates the performance of three textual similarity techniques for CIA based on Bag of words in combination with either topic modeling or file coupling. Results: The approaches are applied on data from two industrial projects. The data comes as part of an industrial collaboration project with Brightsquid, a Canadian start-up company specializing in secure communication solutions. Performance analysis shows that combining textual similarity with file coupling improves impact prediction, resulting in Recall of 67%. Effort and duration can be predicted with 84% and 72% accuracy using textual similarity only. Conclusions: The relative effort invested into CIA for predicting impacted files can be reduced by extending its applicability to multiple dimensions which include impacted files, effort, and duration.

Background: Several mobile app releases do not find their way to the end users. Our analysis of 11,514 releases across 917 open source mobile apps revealed that 44.3% of releases created in GitHub never shipped to the app store (market). Aims: We introduce "marketability" of open source mobile apps as a new release decision problem. Considering app stores as a complex system with unknown treatments, we evaluate performance of predictive models and analogical reasoning for marketability decisions. Method: We performed a survey with 22 release engineers to identify the importance of marketability release decision. We compared different classifiers to predict release marketability. For guiding the transition of not successfully marketable releases into successful ones, we used analogical reasoning. We evaluated our results both internally (over time) and externally (by developers). Results: Random forest classification performed best with F1 score of 78%. Analyzing 58 releases over time showed that, for 81% of them, analogical reasoning could correctly identify changes in the majority of release attributes. A survey with seven developers showed the usefulness of our method for supporting real world decisions. Conclusions: Marketability decisions of mobile apps can be supported by using predictive analytics and by considering and adopting similar experience from the past.

Functionality of software products often does not match user needs and expectations. The closed set-up of systems and information is replaced by wide access to data of users and competitor products. This shift offers completely new opportunities to approach requirements elicitation and subsequent planning of software functionality. This is, in particular true for app store markets. App stores are markets for many small sized software products which provide an open platform for users to provide feedback on using apps. Moreover, the functionality and status of similar software products can be retrieved. While this is a competitive risk, it is at the same time an opportunity.In this paper, we envision a new release planning approach that leverages the new opportunities for decision making. We propose a new model using bi-criterion integer programming. We make suggestions for optimized super app functionality that are based on two key aspects: (i) the estimated value of features, and (ii) the cohesiveness between newly added features and cohesiveness between existing and the features to be added. The information on these attributes comes from reasoning on feature composition of existing similar apps. The approach is applicable to the development of new product releases as well as to the creation of completely new apps. We illustrate the applicability of our model by a small example and outline directions for future research.

Industry and society are facing radical changes due to fast growing digital technologies and its ubiquity. Products and services will increasingly augment and integrate the real world with the digital world. This digital transformation has reached all business areas. Companies and consumers expect to obtain innovation, market penetration, cost reductions and more flexibility. The relationship between RE and society is bi-directional. In this talk, we discuss the evolving role of RE by referring to a quarter century of impressive research. We discuss the increasing scope and responsibility of our discipline, serving as the bridge between the general public and technical teams and providing a response to the dramatic changes in our society.

Large software organizations such as Facebook or Netflix, who otherwise make daily or even hourly releases of their web applications using continuous delivery, have had to invest heavily into a customized release strategy for their mobile apps, because the vetting process of app stores introduces lag and uncertainty into the release process. Amidst these large, resourceful organizations, it is unknown how the average mobile app developer organizes her app's releases, even though an incorrect strategy might bring a premature app update to the market that drives away customers towards the heavy market competition. To understand the common release strategies used for mobile apps, the rationale behind them and their perceived impact on users, we performed two surveys with users and developers. We found that half of the developers have a clear strategy for their mobile app releases, since especially the more experienced developers believe that it affects user feedback. We also found that users are aware of new app updates, yet only half of the surveyed users enables automatic updating of apps. While the release date and frequency is not a decisive factor to install an app, users prefer to install apps that were updated more recently and less frequently. Our study suggests that an app's release strategy is a factor that affects the ongoing success of mobile apps.

Open innovation is the collective term describing business collaboration which combines internal and external ideas into architectures and systems. Despite the wide interest in several domains and the unquestionable potential that open innovation can bring to the software industry, open innovation remains greatly unexplored in the software engineering literature. While the business view of open innovation proved to be beneficial, the software engineering community needs support in understanding what tools, techniques and methods are well suited or can enable open innovation on both strategic and operational levels of software engineering. OISE 2015 is the first step toward raising awareness about open innovation in software engineering academic and industrial communities.