ABSTRACT System‐level testing of avionics software systems requires compliance with different international safety standards such as DO‐178C. An important consideration of the avionics industry is automated test data generation according to the criteria suggested by safety standards. One of the recommended criteria by DO‐178C is the modified condition/decision coverage (MC/DC) criterion. Current model‐based test data generation approaches use constraints written in Object Constraint Language (OCL) and apply search techniques to generate test data. These approaches either do not support MC/DC criterion or suffer from performance issues while generating test data for large‐scale avionics systems. In this paper, we propose an effective way to automate MC/DC test data generation during model‐based testing. We develop a strategy that utilizes case‐based reasoning (CBR) and range reduction heuristics designed to solve MC/DC‐tailored OCL constraints. We performed an empirical study to compare our proposed strategy for MC/DC test data generation using CBR, range reduction, both CBR and range reduction, with an original search algorithm, and random search. We also empirically compared our strategy with existing constraint‐solving approaches. The results show that both CBR and range reduction for MC/DC test data generation outperform the baseline approach. Moreover, the combination of both CBR and range reduction for MC/DC test data generation is an effective approach compared to existing constraint solvers.

Summary The unmanned aircraft system (UAS) is rapidly gaining popularity in civil and military domains. A UAS consists of an application software that is responsible for defining a UAS mission and its expected behavior. A UAS during its mission experiences changes (or interruptions ) that require the unmanned aerial vehicle (UAV) in a UAS to self‐adapt, that is, to adjust both its behavior and position in real‐time, particularly for maintaining formation in the case of a UAS swarm. This adaptation is critical as the UAS operates in an open environment, interacting with humans, buildings, and neighboring UAVs. To verify if a UAS correctly makes an adaptation, it is important to test it. The current industrial practice for testing the self‐adaptive behaviors in UAS is to carry out testing activities manually. This is particularly true for existing UAS rather than newly developed ones. Manual testing is time‐consuming and allows the execution of a limited set of test cases. To address this problem, we propose an automated model‐based approach to test the self‐adaptive behavior of UAS application software. The work is conducted in collaboration with an industrial partner and demonstrated through a case study of UAS swarm formation flight application software. Further, the approach is verified on various self‐adaptive behaviors for three open‐source autopilots (i.e., Ardu‐Copter, Ardu‐Plane, and Quad‐Plane). Using the proposed model‐based testing approach we are able to test sixty unique self‐adaptive behaviors. The testing results show that around 80% of the behavior adaptations are correctly executed by UAS application software.

Abstract Model transformation testing has become crucial as model‐driven engineering has raised the abstraction level for developing software systems. Transformation is written to transform models from one level of abstraction to another, for example, model to model or model to code. A major challenge in testing the transformation is the creation of test models, such that (i) they conform to the source meta‐model (i.e., multiplicities and Object Constraint Language [OCL] constraints on meta‐model) and (ii) they provide coverage of the complete transformation (solving branch conditions for traversing all paths). Manual creation of test models requires a lot of time and effort. Still, the validity of the developed test models cannot be ensured. This paper aims to solve the above challenges using an automated search‐based strategy. The proposed approach is two‐stepped. First, valid test models are generated by solving source meta‐model constraints. Second, the generated models are evolved for achieving the structural coverage of the transformation by solving the branch conditions. A toolset model transformation testing environment (MOTTER) is developed to automate the search‐based solution. The proposed work is empirically evaluated on two case studies using four search algorithms. The result reflects that it successfully generates valid test models for achieving desired structural coverage with high performance on both the case studies.

Abstract Data science (DS) applications not only suffer from traditional software faults but may also suffer from data‐specific and model‐related faults. Fault models play an important role in evaluating and designing tests for testing DS applications. The existing fault models do not consider DS specific faults. In this study, we built a fault model DS applications. We investigate the faults by using diverse approaches: (i) a multi‐vocal literature survey of published literature, (ii) semi‐structured interviews of industry experts. The Multi‐vocal study allows us to synthesize the existing knowledge from researchers and practitioners. Qualitative data from semi‐structured interviews provide us with insights into the nature of faults encountered by practitioners. We combine the results of (i) and (ii) to derive a detailed fault model. The developed fault model is further validated through a quantitative survey of industry practitioners, and the respondents were asked to identify the faults from our proposed fault model that they have experienced and classify those faults based on their severity as perceived by practitioners and its frequency. The results show that practitioners consider prediction bias and model decay as the most severe faults while data sampling and splitting faults along with feature engineering faults are the most frequent.