Large Language Models (LLMs) have gained attention for addressing coding problems, but their effectiveness in fixing code maintainability remains unclear. This study evaluates LLMs capability to resolve 127 maintainability issues from 10 GitHub repositories. We use zero-shot prompting for Copilot Chat and Llama 3.1, and few-shot prompting with Llama only. The LLM-generated solutions are assessed for compilation errors, test failures, and new maintainability problems. Llama with few-shot prompting successfully fixed 44.9 % of the methods, while Copilot Chat and Llama zero-shot fixed 32.29 % and 30 %, respectively. However, most solutions introduced errors or new maintainability issues. We also conducted a human study with 45 participants to evaluate the readability of 51 LLM-generated solutions. The human study showed that 68.63 % of participants observed improved readability. Overall, while LLMs show potential for fixing maintainability issues, their introduction of errors highlights their current limitations.

Summary Software defect prediction is a subject of study involving the interplay of software engineering and machine learning. The current literature proposed numerous machine learning models to predict software defects from software data, such as commits and code metrics. Further, the most recent literature employs explainability techniques to understand why machine learning models made such predictions (i.e., predicting the likelihood of a defect). As a result, developers are expected to reason on the software features that may relate to defects in the source code. However, little is known about the developers' perception of these machine learning models and their explanations. To explore this issue, we focus on a survey with experienced developers to understand how they evaluate each quality attribute for the defect prediction. We chose the developers based on their contributions at GitHub, where they contributed to at least 10 repositories in the past 2 years. The results show that developers tend to evaluate code complexity as the most important quality attribute to avoid defects compared with the other target attributes such as source code size, coupling, and documentation. At the end, a thematic analysis reveals that developers evaluate testing the code as a relevant aspect not covered by the static software features. We conclude that, qualitatively, there exists a misalignment between developers' perceptions and the outputs of machine learning models. For instance, while machine learning models assign high importance to documentation, developers often overlook documentation and prioritize assessing the complexity of the code instead.

Abstract Software defect and code smell prediction help developers identify problems in the code and fix them before they degrade the quality or the user experience. The prediction of software defects and code smells is challenging, since it involves many factors inherent to the development process. Many studies propose machine learning models for defects and code smells. However, we have not found studies that explore and compare these machine learning models, nor that focus on the explainability of the models. This analysis allows us to verify which features and quality attributes influence software defects and code smells. Hence, developers can use this information to predict if a class may be faulty or smelly through the evaluation of a few features and quality attributes. In this study, we fill this gap by comparing machine learning models for predicting defects and seven code smells. We trained in a dataset composed of 19,024 classes and 70 software features that range from different quality attributes extracted from 14 Java open-source projects. We then ensemble five machine learning models and employed explainability concepts to explore the redundancies in the models using the top-10 software features and quality attributes that are known to contribute to the defects and code smell predictions. Furthermore, we conclude that although the quality attributes vary among the models, the complexity, documentation, and size are the most relevant. More specifically, Nesting Level Else-If is the only software feature relevant to all models.

Software defect prediction represents an area of interest in both academia and the software industry. Thus, software defects are prevalent in software development and might generate numerous difficulties for users and developers apart. The current literature offers multiple alternative approaches to predict the likelihood of defects in the source code. Most of these studies concentrate on predicting defects from a broad set of software features. As a result, the individual discriminating power of software features is still unknown as some perform well only with specific projects or metrics. In this study, we applied machine learning techniques in a popular dataset. This data has information about software defects in five Java projects, containing 5,371 classes and 37 software features. To this aim, we convey an exploratory investigation that produced hundreds of thousands of machine learning models from a diverse collection of software features. These models are random in the sense that they promptly select the features from the entire pool of features. Even though the immense majority of models are ineffective, we could produce several models that yield accurate predictions, thus classifying defects from Java project classes. Among these accurate models, our results indicate that change metric features are more present than entropy or class-level metrics. We concentrated our analysis on models that rank a randomly chosen defective class higher than a casually selected clean class with over 80% accuracy. We also report and discuss some features contributing to the explanation of model decisions. Therefore, our study promotes reasoning on which features support predicting defects in these projects. Finally, we present the implications of our work to practitioners.

Software maintenance plays an important role during software development and life cycle. Indeed, previous works show that maintenance activities consume most of the software budget. Therefore, understanding how these activities are performed can help software managers to previously plan and allocate resources in projects. Despite previous works, there is still a lack in accurate models to classify developers commits into maintenance activities. In the present article, we propose improvements in a state-of-the-art approach used to classify commits. Particularly, we include three additional features in the classification model and we use XGBoost, a boosting tree learning algorithm, for classification. Experimental results show that our approach outperforms the state-of-the-art baseline achieving more than 77% of accuracy and more than 64% in Kappa metric.