In long lifespan software systems, specification documents can be outdated or even missing. Developing new software releases or checking whether some user requirements are still valid becomes challenging in this context. This challenge can be addressed by extracting high-level observable capabilities of a system by mining its source code and the available source-level documentation. This paper presents feature extraction and traceability (FEAT), an approach that automatically extracts topoi, which are summaries of the main capabilities of a program, given under the form of collections of code functions along with an index. FEAT acts in two steps: first, clustering: by mining the available source code, possibly augmented with code-level comments, hierarchical agglomerative clustering groups similar code functions. In addition, this process gathers an index for each function. Second, entry point selection: functions within a cluster are then ranked and presented to validation engineers as topoi candidates. We implemented FEAT on top of a general-purpose test management and optimization platform and performed an experimental study over 15 open-source software projects amounting to more than 1 M lines of codes proving that automatically discovering topoi is feasible and meaningful on realistic projects.

Exhaustive testing of highly configurable software developed in continuous integration is rarely feasible in practice due to the configuration space of exponential size on the one hand, and strict time constraints on the other. This entails using selective testing techniques to determine the most failure-inducing test cases, conforming to highly-constrained time budget. These challenges have been well recognized by researchers, such that many different techniques have been proposed. In practice, however, there is a lack of efficient tools able to reduce high testing effort, without compromising software quality. In this paper we propose a test suite optimization technology TITAN, which increases the time-and cost-efficiency of testing highly configurable software developed in continuous integration. The technology implements practical test prioritization and minimization techniques, and provides test traceability and visualization for improving the quality of testing. We present the TITAN tool and discuss a set of methodological and technological challenges we have faced during TITAN development. We evaluate TITAN in testing of Cisco's highly configurable software with frequent high quality releases, and demonstrate the benefit of the approach in such a complex industry domain.

Data-intensive systems in e-governance collect and process data to ensure conformance to a set of business rules. Testers meticulously verify data in test databases, extracted from different steps of a live production stream , for correct application of business rules. We simplify the process by allowing testers to model a test domain on a relational database and automatically generate test cases representing data interactions satisfying combinatorial interaction coverage criteria. This paper also introduces test cases with self-referential interactions, which is a necessity in real-world databases. We verify these test cases using our human-in-the-loop tool, Depict. Depict, with expert assistance, generates complex SQL queries for test cases and produces a visual report of test case satisfaction. We apply the approach to two scenarios: 1) simplify and optimize a periodic archiving operation and 2) verify fault codes within the testing environment of the Custom directorate's TVINN system.

Testing data-intensive systems is paramount to increase our reliance on information processed in e-governance, scientific/ medical research, and social networks. A common practice in the industrial testing process is to use test databases copied from live production streams to test functionality of complex database applications that manage well-formedness of data and its adherence to business rules in these systems. This practice is often based on the assumption that the test database adequately covers realistic scenarios to test, hopefully, all functionality in these applications. There is a need to systematically evaluate this assumption. We present a tool-supported method to model realistic scenarios and verify whether copied test databases actually cover them and consequently facilitate adequate testing. We conceptualize realistic scenarios as data interactions between fields cross-cutting a complex database schema and model them as test cases in a classification tree model. We present a human-in the-loop tool, DEPICT, that uses the classification tree model as input to (a) facilitate interactive selection of a connected sub graph from often many possible paths of interactions between tables specified in the model (b) automatically generate SQL queries to create an inner join between tables in the connected sub graph (c) extract records from the join and generate a visual report of satisfied and unsatisfied interactions hence quantifying test adequacy of the test database. We report our experience as a qualitative evaluation of approach and with a large industrial database from the Norwegian Customs and Excise information system TVINN featuring large and complex databases with millions of records.