Background: Modern software increasingly relies on Developer Prompts (Dev Prompts)-snippets of natural language embedded directly in source code-to leverage the capabilities of Large Language Models (LLMs) for tasks like classification, summarization, and content generation. Yet, despite the rapid adoption of LLMs and Dev Prompts, it remains unclear to what extent these prompts unintentionally encode biases, invite injection attacks, or underperform due to sub-optimal phrasing. Aims: To address this gap, we present a large-scale empirical analysis of Dev Prompts found in real open-source software projects to assess the prevalence of bias, security vulnerabilities, and performance issues. Then, we propose and validate approaches to mitigate these issues, and demonstrate the practical feasibility of addressing them. Method: We systematically sampled 2,320 Dev Prompts from a set of 40,573 found in open-source software projects, to identify the prevalence of the aforementioned issues. We also implemented a lightweight tool that automatically rewrites flawed prompts. Results: We find evidence of easy-to-fix issues across multiple dimensions: 3.46% of prompts contain language likely to lead to biased model responses, while over 10.75% are vulnerable to straightforward injection attacks, and we posit that many more are amenable to performance improvement through minor adjustments. Our prototype successfully mitigated bias in 68.29% of cases, prevented injection vulnerabilities in 41.81%, and improved performance in 37.1% of tested prompts. Conclusions: Our findings highlight an urgent need for future research and dedicated tool-support to help software developers write safer, fairer, and more effective prompts. To facilitate ongoing work in this emerging area, we share our data and analysis infrastructure publicly. We encourage the community to further explore the implications of Dev Prompts in modern software.

In modern software engineering, build systems play the crucial role of facilitating the conversion of source code into software artifacts. Recent research has explored high-level causes of build failures, but has largely overlooked the structural properties of build files. Akin to source code, build systems face technical debt challenges that hinder maintenance and optimization. While refactoring is often seen as a key tool for addressing technical debt in source code, there is a significant research gap regarding the specific refactoring changes developers apply to build code and whether these refactorings effectively address technical debt.In this paper, we address this gap by examining refactorings applied to build scripts in open-source projects, covering the widely used build systems of Gradle, Ant, and Maven. Additionally, we investigate whether these refactorings are used to tackle technical debts in build systems. Our analysis was conducted on 725 examined build-file-related commits. We identified 24 build-related refactorings, which we divided into 6 main categories. These refactorings are organized into the first empirically derived taxonomy of build system refactorings. Furthermore, we investigate how developers employ these refactoring types to address technical debts via a manual commitanalysis and a developer survey. In this context, we identified 5 technical debts addressed by these refactorings and discussed their correlation with the different refactorings. Finally, we introduce BuildRefMiner, an LLM-powered tool leveraging GPT40 to automate the detection of refactorings within build systems. We evaluated its performance and found that it achieves an F1 score of 0.76 across all build systems.This study will serve as a foundational building block for guiding future research and practice in the maintenance and optimization of build systems. BuildRefMiner and the replication package for this study are available at [1]

Virtual Reality (VR) is an emerging technique that provides a unique real-time experience for users. VR technologies have provided revolutionary user experiences in various scenarios (e.g., training, education, gaming, etc.). However, testing VR applications is challenging due to their nature which necessitates physical interactivity, and their reliance on specific hardware systems. Despite the recent advancements in VR technology and its usage scenarios, we still know little about VR application testing. To fill up this knowledge gap, we performed an empirical study on 314 open-source VR applications. Our analysis identified that 79% of the VR projects evaluated did not have any automatic tests, and for the VR projects that did, the median functional-method to test-method ratios were lower than those of other project types. Moreover, we uncovered tool support issues concerning the measurement of VR code coverage, and the assertion density results we were able to generate were relatively low, with an average of 17.63%. Finally, through a manual analysis of 370 test cases, we identified the different categories of test cases being used to validate VR application quality attributes. Furthermore, we extracted which of these categories are VR-attention, meaning that test writers need to pay special attention to VR characteristics when writing tests of these categories. We believe that our findings constitute a call to action for the VR development community to improve their automatic testing practices and provide directions for software engineering researchers to develop advanced techniques for automatic test case generation and test quality analysis for VR applications. Our replication package containing the dataset we used, software tools we developed, and the results we found, is accessible at ‍https://doi.org/10.6084/m9.figshare.19678938.

Background: Continuous Integration (CI) has become widely adopted to enable faster code change integration. Meanwhile, Machine Learning (ML) is being used by software applications for previously unsolvable real-world scenarios. ML projects employ development processes different from those of traditional software projects, but they too require multiple iterations in their development, and may benefit from CI. Aims: While there are many works covering CI within traditional software, none of them empirically explored the adoption of CI and its associated issues within ML projects. To address this knowledge gap, we performed an empirical analysis comparing CI adoption between ML and Non-ML projects. Method: We developed TraVanalyzer, the first Travis CI configuration analyzer, to analyze the CI practices of ML projects, and developed a CI log analyzer to identify the different CI problems of ML projects. Results: We found that Travis CI is the most popular CI tool for ML projects, and that their CI adoption lags behind that of Non-ML projects, but that ML projects which adopted CI, used it for building, testing, code analysis, and automatic deployment more than Non-ML projects. Furthermore, while CI in ML projects is as likely to experience problems as CI in Non-ML projects, it has more varied reasons for build-breakage. The most frequent CI failures of ML projects are due to testing-related problems, similar to Non-ML and OSS CI failures. Conclusion: To the best of our knowledge, this is the first work that has analyzed ML projects’ CI usage, practices, and issues, and contextualized its results by comparing them with similar Non-ML projects. It provides findings for researchers and ML developers to identify possible improvement scopes for CI in ML projects.

Refactoring is widely adopted nowadays in industry to restructure the code and meet high quality while preserving the external behavior. Many of the existing refactoring tools and research are based on search-based techniques to find relevant recommendations by finding trade-offs between different quality attributes. While these techniques show promising results on open-source and industry projects, they lack explanations of the recommended changes which can impact their trustworthiness when adopted in practice by developers. Furthermore, most of the adopted search-based techniques are based on random population generation and random change operators (e.g., crossover and mutation). However, it is critical to understand which good refactoring patterns may exist when applying change operators to either keep them or exchange with other solutions rather than destroying them with random changes. In this paper, we propose knowledge-informed change operators and an improved seeding mechanism that we integrated in a multi-objective genetic algorithm. We also provide explanations for refactoring solutions. First, we generate association rules using the Apriori algorithm to find relationships between applied refactorings in previous commits, their locations, and their rationale (quality improvements). Then, we use these rules to 1) initialize the population, 2) improve the change operators and seeding mechanisms of the multi-objective search in order to preserve and exchange good patterns in the refactoring solutions, and 3) explain how a sequence of refactorings collaborate in order to improve the quality of the system (e.g., fitness functions). The validation on large open-source systems shows that X-SBR provides refactoring solutions of a better quality than those given by the state-of-the-art techniques in terms of reducing the invalid refactorings, improving the quality, and increasing trustworthiness of the developers in the suggested refactorings via the provided explanations.