Developers may choose to implement a library despite the existence of similar libraries, considering factors such as computational performance, language or platform dependency, accuracy, convenience, and completeness of an API. As a result, GitHub hosts several library projects that have overlaps in their functionalities. These overlaps have been of interest to developers from the perspective of code reuse or the preference of one implementation over the other. Through an empirical study, we explore the extent and nature of existence of these similarities in the library functions. We have further studied whether the similarity of functions across different libraries and their associated test suites can be leveraged to reveal defects in one another. We see scope for effectively using the mining of test suites from the perspective of revealing defects in a program or its documentation. Another noteworthy observation made in the study is that similar functions may exist across libraries implemented in the same language as well as in different languages. Identifying the challenges that lie in building a testing tool, we automate the entire process inMetallicus, a test mining and recommendation tool.Metallicusreturns a test suite for the given input of a query function and a template for its test suite. On a dataset of query functions taken from libraries implemented in Java or Python,Metallicusrevealed 46 defects.

A software system evolves over time due to factors such as bug-fixes, enhancements, optimizations and deprecation. As entities interact in a software repository, the alterations made at one point may require the changes to be reflected at various other points to maintain consistency. However, often less attention is given to making appropriate changes to the documentation associated with the functions. Inconsistent documentation is undesirable, since documentation serves as a useful source of information about the functionality. This paper presents a study on the prevalence of function documentations that are indirectly or implicitly dependent on entities other than the associated function. We observe a substantial presence of such documentations, with 62% of the studied Javadoc comments being dependent on other entities, as studied in 11 open-source repositories implemented in Java. We comprehensively analyze the nature of documentation updates made in 1288 commit logs and study patterns to reason about the cause of dependency in the documentation. Our findings from the observed patterns may be applied to suggest documentations that should be updated on making a change in the repository.

The thesis aims to provide test generation techniques, beyond the consideration of coverage-based criterion, with an objective to highlight inconsistencies in the documentation and the implementation. We leverage the domain knowledge gained from developers' expertise and existing resources to generate test-cases.

Developers may choose to implement a library, despite the existence of similar libraries, considering factors such as computational performance, language or platform dependency, and accuracy. As a result, GitHub is a host to several library projects that have overlaps in the functionalities. These overlaps have been of interest to developers from the perspective of code reuse or preferring one implementation over the other. We present an empirical study to explore the extent and nature of existence of these similarities in the library functions. We have further studied whether the similarity among functions across different libraries and their associated test suites can be leveraged to reveal defects in one another. Applying a natural language processing based approach on the documentations associated with functions, we have extracted matching functions across 12 libraries, available on GitHub, over 2 programming languages and 3 themes. Our empirical evaluation indicates existence of a significant number of similar functions across libraries in same as well as different programming languages where a language can influence the extent of existence of similarities. The test suites from another library can serve as an effective source of defect revealing tests. The study resulted in revealing 72 defects in 12 libraries. Further, we analyzed the source of origination of the defect revealing tests. We deduce that issue reports and pull requests can be beneficial in attaining quality test cases not only to test the libraries in which these issues are reported but also for other libraries that are similar in theme.

Automated testing techniques are often assessed on coverage based metrics. However, despite giving good coverage, the test cases may miss the gap between functional specification and the code implementation. This gap may be subtle in nature, arising due to the absence of logical checks, either in the implementation or in the specification, resulting in inconsistencies in the input definition. The inconsistencies may be prevalent especially for structured inputs, commonly specified using string-based data types. Our study on defects reported over popular libraries reveals that such gaps may not be limited to input validation checks. We propose a test generation technique for structured string inputs where we infer inconsistencies in input definition to expose semantic gaps in the method under test and the method specification. We assess this technique using our tool SEGATE, Semantic Gap Tester. SEGATE uses static analysis and automaton modeling to infer the gap and generate test cases. On our benchmark dataset, comprising of defects reported in 15 popular open-source libraries, written in Java, SEGATE was able to generate tests to expose 80% of the defects.