Library migration is the process of replacing one library with another library that provides similar functionality. Manual library migration is time consuming and error prone, as it requires developers to understand the APIs of both libraries, map them, and perform the necessary code transformations. Large Language Models (LLMs) are shown to be effective at generating and transforming code as well as finding similar code, which are necessary upstream tasks for library migration. Such capabilities suggest that LLMs may be suitable for library migration. Accordingly, this paper investigates the effectiveness of LLMs for migration between Python libraries. We evaluate three LLMs, LLama 3.1, GPT-4o mini, and GPT-4o on PyMigBench, where we migrate 321 real-world library migrations that include 2,989 migration-related code changes. To measure correctness, we (1) compare the LLM’s migrated code with the developers’ migrated code in the benchmark and (2) run the unit tests available in the client repositories. We find that LLama 3.1, GPT-4o mini, and GPT-4o correctly migrate 89%, 89%, and 94% of the migration-related code changes, respectively. We also find that 36%, 52% and 64% of the LLama 3.1, GPT-4o mini, and GPT-4o migrations pass the same tests that passed in the developer’s migration. To ensure the LLMs are not reciting the migrations, we also evaluate them on 10 new repositories where the migration never happened. Overall, our results suggest that LLMs can be effective in migrating code between libraries, but we also identify some open challenges.

Obsolete and redundant tests increase regression testing costs. Therefore, developers should remove them from test suites; however, identifying these tests is non-trivial. Automated techniques for identifying obsolete and redundant tests could help developers reduce regression testing costs. Nonetheless, we have limited empirical evidence of how and why developers delete tests. Therefore, in this work, we first create DelTest, a dataset of 24,431 manually confirmed deleted tests, by analyzing 449,592 commits from seven open-source Java projects. We then perform an empirical study on DelTest to understand test deletion. Our findings show that test deletion frequency and the number of deleted tests vary significantly across projects, suggesting that test deletion is more likely driven by project-specific needs than the broader development cycle. Developers delete only one or two tests in most commits, suggesting test deletion is mostly small and incremental. In DelTest, 83.2% of tests are deleted along with the corresponding test classes, while $16.8 \%$ are deleted individually. We find that $91.4 \%$ of deleted tests in six projects are obsolete tests (i.e., production code is deleted), $7 \%$ are redundant tests (i.e., passing tests), and $1.6 \%$ are failing tests. The deletion of $20 \%$ of redundant tests reduces code coverage or mutation scores. We also evaluate test suite reduction (TSR) approaches on DelTEST and find that a TSR approach identifies up to 54% of the redundant tests. Our findings can help improve automated techniques for identifying obsolete and redundant tests.

Regression testing is critical for maintaining software quality. Therefore, developers must maintain efficient and effective regression test suites. However, the task can be tedious and challenging, especially in Continuous Integration (CI) where developers frequently change code. With each code change, developers may need first to identify all relevant tests for the code change and then identify specific tests among the relevant tests that need to be modified. This process can reduce developers' productivity if they do this manually. Developers may also not modify test suites due to time constraints, resulting in inefficient and ineffective test suites. In this paper, we propose a technique and tool, TREC, that identifies relevant tests for code changes in CI and recommends tests to developers for modifications. TREC performs method and test co-evolution and method call analysis to identify and recommend tests for code changes. We evaluate the effectiveness of TREC by recommending tests for 1,699 developers' modified methods in 437 commits from three open-source Java projects. We find TREC's recommended tests include 2,886 (84.39%) of the 3,420 developers' modified tests for the 1,699 methods, indicating TREC's capability to successfully identify and recommend the majority of tests that need to be modified for the methods. Demo: https://www.youtube.com/watch?v=WgIveZuPDBU

Writing effective unit tests is often tedious, difficult, and time-consuming. Test recommendation techniques facilitate this process by recommending existing manually written tests from other similar systems for developers to reuse. However, developers still have to put non-trivial effort into modifying the recommended tests. For example, they have to understand vari-ous code elements in the recommended tests to accurately replace them with semantically similar code elements from the target system. In this paper, we propose JTESTMIGRATOR, a technique to automatically migrate unit tests between semantically similar methods across applications. Given a source and a target method with similar functionality across applications, where the source method has some unit tests, JTESTMIGRATOR migrates unit tests by transforming the test code. JTESTMIGRATOR uses semantic similarity and type compatibility of code elements in the source and target systems to transform test code. We implement JTESTMIGRATOR to migrate JUnit tests and evaluate it on 104 tests for 42 methods across 5 popular libraries. JTESTMIGRATOR successfully migrates 76 (73%) of the tests for 32 (76%) of the methods across the 5 libraries. 10 (13%) of the successfully migrated tests increase the code coverage of 4 target methods.

Developers heavily rely on Application Programming Interfaces (APIs) from libraries to build their projects. However, libraries might become obsolete, or new libraries with better APIs might become available. In such cases, developers replace the used libraries with alternative libraries, a process known as library migration. Since manually migrating between libraries is tedious and error prone, there has been a lot of effort towards automated library migration. However, most of the current research on automated library migration focuses on Java libraries, and even more so on version migrations of the same library. Despite the increasing popularity of Python, limited research has investigated migration between Python libraries. To provide the necessary data for advancing the development of Python library migration tools, this paper contributes PyMigBench, a benchmark of real Python library migrations.PyMigBench contains 59 analogous library pairs and 75 real migrations with migration-related code changes in 161 Python files across 57 client repositories.

Unit tests play a critical role in improving software quality. However, writing effective unit tests from scratch is difficult and tedious. One way to reduce this difficulty is to recommend existing tests of semantically similar functions. However, modifying the recommended tests manually might still be difficult and tedious. For example, developers have to understand various code elements in the recommended tests to accurately replace them with semantically similar code elements from the target application. One way to mitigate the issue is by developing a test migration or reuse technique that could automatically transform the code elements in the recommended tests and migrate them to the target application. However, to develop such a technique, we first need to identify what types of code transformations are required to successfully migrate the recommended tests. Therefore, in this paper, we first recruit two external participants to create JTestMigBench, a benchmark of 510 manually migrated JUnit tests for 186 methods from five popular libraries. We then analyze the code changes in the migrated tests to create JTestMigTax, a taxonomy of test code transformation patterns. Our contributions provide the necessary foundations to develop automated unit test migration or reuse techniques.

While Application Programming Interfaces (APIs) allow easier reuse of existing functionality, developers might make mistakes in using these APIs (a.k.a. API misuses). If an API usage specification exists, then automatically detecting such misuses becomes feasible. Since manually encoding specifications is a tedious process, there has been a lot of research regarding pattern-based specification mining. However, while annotations are widely used in Java enterprise microservices frameworks, most of these pattern-based rule discovery techniques have not considered annotation-based API usage rules. In this industrial case study of MicroProfile, an open-source Java microservices framework developed by IBM and others, we investigate whether the idea of pattern-based discovery of rules can be applied to annotation-based API usages. We find that our pattern-based approach mines 23 candidate rules, among which 4 are fully valid specifications and 8 are partially valid specifications. Overall, our technique mines 12 valid rules, 10 of which are not even documented in the official MicroProfile documentation. To evaluate the usefulness of the mined rules, we scan MicroProfile client projects for violations. We find 100 violations of 5 rules in 16 projects. Our results suggest that the mined rules can be useful in detecting and preventing annotation-based API misuses.

Understanding the adoption and usage of any programming language feature is crucial for improving it. Existing studies indicate that Java annotations are widely used by developers. However, there is currently no empirical data on annotation usage in Android apps. Android apps are often smaller than general Java applications and typically use Android APIs or specific libraries catered to the mobile environment. Therefore, it is not clear if the results of existing Java studies hold for Android apps. In this paper, we investigate annotation practices in Android apps through an empirical study of 1,141 open-source apps. Using previously studied metrics, we first compare annotation usage in Android apps to existing results from general Java applications. Then, for the first time, we study why developers declare custom annotations. Our results show that the density of annotations and the values of various other annotation metrics are notably less in Android apps than in Java projects. Additionally, the types of annotations used in Android apps are different than those in Java, with many Android-specific annotations. These results imply that researchers may need to distinguish mobile apps while performing studies on programming language features. However, we also found examples of extreme usage of annotations with, for example, a large number of attributes, as well as a low adoption rate for most annotations. By looking at such results, annotation designers can assess adoption patterns and take various improvement measures, such as modularizing their offered annotations or cleaning up unused ones. Finally, we find that developers declare custom annotations in different apps but with the same purpose, which presents an opportunity for annotation designers to create new annotations.

Each Android app must have an Android manifest file. It is one of the most important configuration files manually written by developers. In addition to various configuration parameters required to run an app, it also contains configuration parameters which are used to implement security, compatibility, and accessibility of an app. Any mistakes in writing the manifest file can cause serious implications in terms of security, reliability, and availability of an app. In this paper, we study and report different types of mistakes committed by developers in writing Android manifest files. The study was performed on 13,483 real-world Android apps. We also present an open source rule-based static analysis tool which detects developer mistakes in the manifest file. The tool generates a warning message if it detects any misconfigurations in the manifest file. We used the tool to perform the empirical study and it generated total 59,547 configuration errors in 11,110 apps. Only 2,373 apps, among studied apps, do not have any configuration errors.