Modern software systems increasingly depend on packages released on code sharing platforms such as GitHub, Bitbucket, and GitLab. To minimize the risk of lurking defects in such packages, strong test suites covering the normal as well as the exceptional paths are needed. In this paper we explore the potential of using tests from dependent projects to increase the code coverage of base packages. We extracted 4 popular Python packages from GitHub together with 14 dependent projects and analyzed the code coverage of the available tests. We observed that adopting the tests of the dependent projects in the test suite of the base library, would increase the line coverage in 9 out of 14 (64%) of the cases and the mutation coverage in all of them (100%). Our results suggest that a tool which would generate tests for the base package based on the tests in the dependent projects, would help to strengthen the test suite.

Previous research has demonstrated that the test coverage of libraries can be expanded by using existing test inputs from their dependent projects. In this paper, we propose an algorithm for test transplantation based on test slicing. The algorithm extracts test inputs, isolates them by creating mocks, and then transplants the test code onto the test suite of the libraries. To achieve test slicing, we dynamically execute the tests in the dependent project and create its graph of histories. Then, we traverse back from the interesting object state and collect the corresponding edges. Finally, we reverse the collected edges and create a sequence of method calls to reconstruct the same object state. We have implemented a proof-of-concept in Pharo-Smalltalk, in this paper we discuss the lessons learned so far.

Software test amplification is the act of strength-ening manually written test-cases to exercise the boundary conditions of the system under test. It has been demonstrated by the research community to work for the programming language Java, relying on the static type system to safely transform the code under test. In dynamically typed languages, such type decla-rations are not available, and as a consequence test amplification has yet to find its way to programming languages like Smalltalk, Python, Ruby and Javascript. The AnSyMo research group has created two proof of concept tools for languages without a static type system: AmPyfier (for Python) and Small-Amp (for Pharo-Smalltalk). In this tool demonstration paper we explain how we relied on profiling libraries present in the respective eco-systems to infer the necessary type information for enabling full-blown test amplification.

Abstract Some test amplification tools extend a manually created test suite with additional test cases to increase the code coverage. The technique is effective, in the sense that it suggests strong and understandable test cases, generally adopted by software engineers. Unfortunately, the current state-of-the-art for test amplification heavily relies on program analysis techniques which benefit a lot from explicit type declarations present in statically typed languages. In dynamically typed languages, such type declarations are not available and as a consequence test amplification has yet to find its way to programming languages like Smalltalk, Python, Ruby and Javascript. We propose to exploit profiling information —readily obtainable by executing the associated test suite— to infer the necessary type information creating special test inputs with corresponding assertions. We evaluated this approach on 52 selected test classes from 13 mature projects in the Pharo ecosystem containing approximately 400 test methods. We show the improvement in killing new mutants and mutation coverage at least in 28 out of 52 test classes (≈ 53%). Moreover, these generated tests are understandable by humans: 8 out of 11 pull-requests submitted were merged into the main code base (≈ 72%). These results are comparable to the state-of-the-art, hence we conclude that test amplification is feasible for dynamically typed languages.

Abstract Test amplification aims to automatically improve a test suite. One technique generates new test methods through transformations of the original tests. These test amplification tools heavily rely on analysis techniques that benefit a lot from type declarations present in the source code of projects written in statically typed languages. In dynamically typed languages, such type declarations are not available, and therefore, research regarding test amplification for those languages is sparse. Recent work has brought test amplification to the dynamically typed language Pharo Smalltalk by introducing the concept of dynamic type profiling. The technique is dependent on Pharo‐specific frameworks and has not yet been generalized to other languages. Another significant downside in test amplification tools based on the mutation score of a test suite is their high time cost. In this paper, we present AmPyfier, a tool that brings test amplification and type profiling to the dynamically typed language Python. AmPyfier introduces multi‐metric selection in order to increase the time efficiency of test amplification. We evaluated AmPyfier on 11 open‐source projects and found that AmPyfier could strengthen 37 out of 54 test classes. Multi‐metric selection decreased the time cost ranging from 17% to 98% as opposed to selection based on the full mutation score.