Online model-based testing is one of the most suitable techniques to assess the proper behavior of service orchestrations. However, the diverse panorama in terms of modeling languages and test case generation tools is a limitation to widespread adoption. We advocate that the application of Model-Driven Engineering principles as meta-modeling and model transformation can cope with this problem, improving the interoperability of artifacts in the test case generation process, thus bringing benefits in case of agile development processes, where system and technology evolution is frequent. In this paper, we present our contribution to this idea, introducing i) a reference metamodel, which stores the business process behavior and the information to generate input models for testing tools, and ii) transformations from orchestration languages towards testing tools. The proposed approach is implemented in a testing framework and evaluated on a case study where multiple orchestrations are expressed in two languages. Also, the paper presents how test cases are appropriately generated and successfully executed, starting from an orchestration model as a consequence of successful transformations.

Service Oriented Architecture (SOA) is a popular design pattern that allows building applications composed of loosely-coupled and autonomous services. Such services may evolve and change at runtime, often outside the control of the owner of the application. Consequently, typical validation approaches, like offline testing performed before services deployment, are necessary but not sufficient: offline testing cannot assure the correct behavior of the SOA during its execution. To cope with the evolution of services and their orchestrations, in this paper we present a Self-Adaptive Model-BAsed online testing framework called SAMBA. SAMBA aims to assess the proper behavior of a SOA during its lifecycle executing model-based online testing at runtime, under the coordination of a MAPE-K control loop. SAMBA is assessed in a case study, where its detection capability are proved through functional, mutation and fault injection tests.

The wide spreading of mobile devices, such as smartphones and tablets, and their advancing capabilities, ranging from taking photos to accessing banking accounts, make them an attractive target for attackers. This, together with the fact that users frequently store critical information in such devices and that many organizations allow employees to use their personal devices to access the enterprise information infrastructure and applications, makes security assessment a key need. This paper proposes an approach for assessing the security risk posed by user-defined configurations in Android devices. The approach is based on the analysis of the risk (impact and likelihood) of user misconfiguration to harm the device or the user. The impact and likelihood values are defined based on a Multiple-Criteria Decision Analysis (MCDA) performed on the inputs provided by a set of security experts. A case study considering the user-defined configurations of 561 Android devices is presented, showing that the majority of the users neglect important and basic security configurations and that the proposed approach can be used in practice to characterize the security risk level of such devices.

The wide spreading of mobile devices, such as smart phones and tablets, and their always-advancing capabilities, ranging from taking photos to accessing banking accounts, makes them an attractive target for attackers. This, together with the fact that users frequently store critical personal information in such devices and that many organizations currently allow employees to use their personal devices to access the enterprise information infrastructure and applications, turns assessing the security of mobile devices into a key issue. In order to understand the common misconfiguration problems, this practical experience report presents a held analysis of 41 user-defined security settings of more than 500 Android devices. Findings suggest that most users neglect basic security configurations such as login mechanisms and (bat manufacturers should rethink their policies in terms of the security settings that can be modified by the users. The paper also proposes concrete security countermeasures to mitigate some of the identified misconfigurations.

A Web Service is a software system designed to support interoperable machine-to-machine interaction over a network, it also provides a standard means of interoperating between different software applications. However, Web Services have raised new challenges on information security, this technology is susceptible to XML Injection attacks, which would allow an attacker to collect and manipulate information to insert malicious code in either server-side or client-side, being one of the most employed attack against web applications according to the OWASP Top 10. Different studies have shown that the current testing techniques -- penetration testing and fuzzy scanning -- generate several false positives and negatives. However, the fault injection technique improve the robustness of web applications, through the greater flexibility to modify the test cases and to find software bugs. This work describes a fault injection technique for the evaluation of Web Services robustness with WS-Security (Username Token) and the development of a set of rules for vulnerability analysis, resulting on the improvement of the vulnerability detector accuracy. Our results show that 82% of web Services tested were vulnerable to XML Injection attacks.

In this paper a new multi-objective implementation of the generalized extremal optimization (GEO) algorithm, named M-GEOvsl, is presented. It was developed primarily to be used as a test case generator to find transition paths from extended finite state machines (EFSM), taking into account not only the transition to be covered but also the minimization of the test length. M-GEOvsl has the capability to deal with strings whose number of elements vary dynamically, making it possible to generate solutions with different lengths. The steps of the algorithm are described for a general multi-objective problem in which the solution length is an element to be optimized. Experiments were performed to generate test case from EFSM benchmark models using M-GEOvsl and the approach was compared with a related work.

Communication systems are inherently buggy. These flaws can lead to security breaches in applications, which a malicious user could exploit to cause security failures in the system and, under certain circumstances, to take complete control of the vulnerable system. In this paper, we introduce a novel attack injection approach based on attack modeling to perform security testing and detect potential security vulnerabilities. We use attack trees to describe the system flaws and derive corresponding attack scenarios. The attack scenarios are refined to executable scripts for testing tools that are in charge of injecting the attacks against the system. The approach is applied to the Wireless Application Protocol (WAP) currently used in low-tier mobile devices. We carried out experiments using real attacks such as Denial of Service (DoS) and Cipher Suite Rollback attacks. The experimental results show that the approach can achieve high efficiency in the role of uncovering vulnerabilities.

This paper presents a methodology and a set of tools for the modelling, validation and testing of Web service composition, conceived and developed within the French national project WebMov. This methodology includes several modelling techniques, based mainly on some variations of Timed Extended Finite State Machines (TEFSM) formalism, which provide a formal model of the BPEL description of Web services composition. These models are used as a reference for the application of different test generation and passive testing techniques for conformance and robustness checking. The whole WebMov methodology is integrated within a dedicated framework, composed by a set of tools that implement the model representation, the test generation and passive testing algorithms. This framework also permits the interaction of these tools to achieve specific modelling and testing activities in a complementary way. A case study based on a real service, a Travel Reservation Web Service, is presented as well as the results of the application of the proposed WebMov methodology and tools.

One important question in component-based software development is how to estimate the risk of using COTS components, as the components may have hidden faults and no source code available. This question is particularly relevant in scenarios where it is necessary to choose the most reliable COTS when several alternative components of equivalent functionality are available. This paper proposes a practical approach to assess the risk of using a given software component (COTS or non-COTS). Although we focus on comparing components, the methodology can be useful to assess the risk in individual modules. The proposed approach uses the injection of realistic software faults to assess the impact of possible component failures and uses software complexity metrics to estimate the probability of residual defects in software components. The proposed approach is demonstrated and evaluated in a comparison scenario using two real off-the-shelf components (the RTEMS and the RTLinux real-time operating system) in a realistic application of a satellite data handling application used by the European Space Agency.

No abstract available.

This paper presents an approach for automated test case generation using a software specification modeled in Statecharts. The steps defined in such approach involve: translation of Statecharts modeling into an XML-based language; and the PerformCharts tool generates FSMs based on control flow. These FSMs are the inputs for the Condado tool which generates test cases. The idea is to demonstrate that by using a higher-level technique, such as Statecharts, complex software can be represented with clarity and rich details. A case study was on an implementation of a protocol specified for communication between a scientific experiment and the On-Board Data Handling Computer of a satellite under development at National Institute for Space Research (INPE).

Fault injection is used to validate a system in the presence of faults. Jaca, a software injection tool developed in previous work, is used to inject faults at interfaces between classes of a system written in Java. We present a strategy for fault injection validation based on dependence analysis. The dependence analysis approach is used to help in reducing the number of experiments necessary to cover the system's interfaces. For the experiments we used a system that consists of two integrated components, an ODBMS performance test benchmark, Wisconsin 007 and an ODBMS, Ozone. The results of some experiments and their analysis are presented.

This work presents Jaca, a fault injection tool that aims to validate Java applications. Jaca injects high-level software faults by affecting attributes and methods of an object's public interface. Jaca is source-code independent: instrumentation needed for injection and monitoring purposes are introduced at bytecode level during load-time. Jaca's usefulness is shown in a case study used to validate a third-parties component. The case study consists of an OODBMS named Ozone as a component and a performance benchmark, Wisconsin OO7, which uses the OODBMS’ stored objects.

Jaca is a software fault injection tool that validates OO applications written in Java. Jaca's major goal is to inject faults using high-level programming features during runtime by corrupting attribute values, methods parameters or return values. Jaca's design was based on a set of patterns-the fault injection pattern system. This pattern describes a generic architecture defined from recurrent design aspects present in most fault injection tools. The objective was to reduce tool development time while enhancing qualities such as portability, extensibility, reusability, efficiency and robustness. The paper presents the pattern set and its use in Jaca's development. An extension of Jaca to consider injection at the assembly level is also presented to show how easy it is to add new features to the tool.

Component-based software engineering techniques are gaining substantial because of their potential to improve productivity and lower development costs of new software applications, yet satisfying high reliability requirements. A first step to address such high reliability requirements consists in reusing reliable components. To merit the attribute "reliable", a component should be extensively validated. As far as testing is the technique most commonly used for validation, this means that reusable components should well tested. For tests to be applied efficiently and on time, a component should be testable. This paper presents an approach to improve component testability by integrating testing resources into it, and hence obtaining a self-testable component. A prototyping tool, Concat, was developed to support the proposed approach. The tool is intended for OO components implemented in C++. Some preliminary results of an empirical evaluation of the fault detection effectiveness of the proposed testing approach are also discussed.

This paper presents an approach for the validation of OO applications by software-implemented fault injection (SWIFI) that is based on computational reflection. The primary motivation for the use of reflection is that it allows a clear separation between functional aspects and the later being related to necessary for fault injection and monitoring. Besides separation of concerns, the use of OO programming and reflection is also intended to provide more flexibility, extensibility, portability and reusability for the instrumentation features. Ease of use, not only to the instrumentation programmer but also to the user is also a goal. This paper presents FIRE, a prototyping tool that supports the proposed approach. FIRE was implemented using OpenC++1.2 and it's aimed to validate C++ applications. Preliminary results on the use of FIRE are also presented.

This paper addresses the problem of estimating the coverage of a fault tolerance mechanism through statistical professing of observations collected in fault injection experiments. A formal definition of coverage is given in terms of the fault and system activity sets that characterize the input space. Two categories of sampling techniques are considered for coverage estimation: sampling in the whole space and sampling in a space partitioned into classes. The estimators for each technique are compared by means of hypothetical examples. Techniques for early estimations of coverage are then studied. These techniques allow unbiased estimations of coverage to be made before all classes of the sampling space have been tested. Then, the "no-reply" problem that hampers most practical fault-injection experiments is discussed and an a posteriori stratification technique is proposed that allows the scope of incomplete tests to be widened by accounting for available structural information about the target system.>

The authors address the problem of validating the dependability of fault-tolerant computing systems, in particular, the validation of the fault-tolerance mechanisms. The proposed approach is based on the use of fault injection at the physical level on a hardware/software prototype of the system considered. The place of this approach in a validation-directed design process and with respect to related work on fault injection is clearly identified. The major requirements and problems related to the development and application of a validation methodology based on fault injection are presented and discussed. Emphasis is put on the definition, analysis, and use of the experimental dependability measures that can be obtained. The proposed methodology has been implemented through the realization of a general pin-level fault injection tool (MESSALINE), and its usefulness is demonstrated by the application of MESSALINE to the experimental validation of two systems: a subsystem of a centralized computerized interlocking system for railway control applications and a distributed system corresponding to the current implementation of the dependable communication system of the ESPRIT Delta-4 Project.>