Microservice-based systems have emerged as an effective architecture for countless industry applications. They provide applications as small, independent, and modular services. With the increasing interest in such systems, it is important to tackle challenges related to their quality assurance. However, to advance research in this area, systems are required to evaluate new approaches and tools. In this paper, we perform a systematic literature search for systems used in research for testing and monitoring microservice-based systems to aid future research. We provide an overview of the found studies and the systems used in their evaluation. We compose a list of publicly available systems and their characteristics, like size, available tests, and technologies used. Finally, we investigated the context in which these systems were used to provide insights in their usage and additional data that is available for them.

Abstract Context The readability of source code is key for understanding and maintaining software systems and tests. Although several studies investigate the readability of source code, there is limited research specifically on the readability of test code and related influence factors. Objective In this paper, we aim at investigating the factors that influence the readability of test code from an academic perspective based on scientific literature sources and complemented by practical views, as discussed in grey literature. Methods First, we perform a Systematic Mapping Study (SMS) with a focus on scientific literature. Second, we extend this study by reviewing grey literature sources for practical aspects on test code readability and understandability. Finally, we conduct a controlled experiment on the readability of a selected set of test cases to collect additional knowledge on influence factors discussed in practice. Results The result set of the SMS includes 19 primary studies from the scientific literature for further analysis. The grey literature search reveals 62 sources for information on test code readability. Based on an analysis of these sources, we identified a combined set of 14 factors that influence the readability of test code. 7 of these factors were found in scientific and grey literature, while some factors were mainly discussed in academia (2) or industry (5) with only limited overlap. The controlled experiment on practically relevant influence factors showed that the investigated factors have a significant impact on readability for half of the selected test cases. Conclusion Our review of scientific and grey literature showed that test code readability is of interest for academia and industry with a consensus on key influence factors. However, we also found factors only discussed by practitioners. For some of these factors we were able to confirm an impact on readability in a first experiment. Therefore, we see the need to bring together academic and industry viewpoints to achieve a common view on the readability of software test code.

The readability of software code is a key success criterion for understanding and maintaining software systems and tests. In industry practice, a limited number of guidelines aim for improving and assessing the readability of software (test) code. Although several studies focus on investigating the readability of software code, we observed limited research work that focuses on the readability of software test code. In this paper we focus on systematically investigating the characteristics, factors, and assessment criteria that have an impact on the readability of test code. We build on a Systematic Mapping Study (SMS) to identify key characteristics, factors, and assessment criteria that have an impact on test code readability, legibility, and understandability to support and improve maintenance tasks. The result set includes 16 studies for further analysis. The majority of publications focuses on readability investigations of automatically generated test code (88%), often evaluated with surveys to access the readability of test code (44 %). Although several approaches aim at assessing the readability with focus on isolated factors, a combination of different readability aspects within an assessment framework can help to better assess and justify the readability of test code with focus on improving software and system test maintenance.