Modern vehicles such as cars, ships, and planes are increasingly managed using Electronic Control Units (ECUs) that communicate over a Controller Area Network (CAN) bus. While this approach offers enhanced functionality, efficiency, and robustness, it may also be used for unforeseen or malicious purposes ranging from aftermarket modifications to full-fledged attacks threatening passengers’ safety. The ability to conduct in-depth tests is thus vital to protect against these issues. However, much of the functionality of ECUs is proprietary or undocumented. To alleviate this obstacle, this work presents a reverse engineering approach using high-coverage test sets produced using Combinatorial Testing (CT) methods. Our results indicate that this technique is promising for exciting unknown functionality, although challenges regarding the presence of hidden state and high-accuracy oracles are yet to be overcome.

A smart contract is often used to handle financial transactions. Unlike traditional programs, contract codes cannot be changed after deployment. It is crucial to test smart contracts thoroughly before deployment. In this paper, we present a fuzzing approach to testing smart contracts. Our fuzzing approach utilizes constraint solving, selective state exploration, and combinatorial testing to improve code coverage. Constraint solving generates test inputs that meet preconditions in a smart contract. Selective state exploration allows different state-dependent behaviors to be exercised while alleviating the state explosion problem. Combinatorial testing is used to exercise parameter interactions in a systematic manner. We implemented our approach in a tool called MagicMirror and evaluated our approach using more than 2,000 contracts. The experimental results show that MagicMirror effectively achieves high code coverage and detects vulnerabilities.

Covering arrays (CAs) are discrete objects appearing in combinatorial design theory that find practical applications, most prominently in software testing. The generation of optimized CAs is a difficult combinatorial optimization problem being subject to ongoing research. Previous studies have shown that many different algorithmic approaches are best suited for different instances of CAs. In this extended abstract we describe the IPO-MAXSAT algorithm, which adopts the prominent IPO strategy for CA generation and uses MaxSAT solving to optimize the occurring sub-problems.

Summary In this paper, we report on applying combinatorial testing to Internet of Things (IoT) home automation hub systems. We detail how to create a dedicated input parameter model of an IoT home automation hub system for use with combinatorial test case generation strategies. Further, we developed an automated test execution framework and two test oracles for evaluation purposes. We applied and evaluated our proposed methodological approach to a real‐world IoT system and analysed the obtained results of various combinatorial test sets with different properties generated based on the derived input model. Additionally, we compare these results to a random testing approach. Our empirical testing evaluations revealed multiple errors in the tested devices and also showed that all considered approaches performed nearly equally well.

Summary This work presents an extended and enhanced gray‐box combinatorial security testing methodology for SQL injection vulnerabilities in web applications. We propose multiple new attack grammars modelling SQLi attacks against MySQL‐compatible databases, each one targeting a different injection context. Additionally, these grammars are also dynamically refined at the beginning of each attack against an endpoint of a web application, as a further optimization of the used attack model by taking into account the specifics of the generated query of that endpoint. Our goal is to enhance existing combinatorial approaches for detecting SQL injection vulnerabilities. The newly developed methodology is implemented in a prototype security testing tool called SQLInjector+, which is an extension of an earlier prototype developed by us in prior work. This improved tool can attack (i.e. test) any web application that uses a MySQL‐compatible database management system. We evaluate our revised approach and improved prototype tool in a case study comprising of different kinds of web applications to which SQLi is a potential security threat. The case study contains the well‐known verification framework WAVSEP among other five real‐world web applications and one web application firewall. Our generated attack vectors, constructed via combinatorial methods applied to our improved and dynamically optimized attack grammars, are capable of injecting every known vulnerable endpoint in WAVSEP and also of finding new vulnerable parameters in some of the real‐world applications investigated in this paper. Our approach performs equally well or better when compared with existing state‐of‐art of SQL injection security testing tools (sqlmap, w3af, wapiti and fuzzdb) across all tested web applications in the case study.

Current web crawlers, document databases and change monitoring systems for web sites are commonly limited to static content and analysis of code as retrieved from the server, an approach that is not suitable for modern dynamic web applications. The canonical representation of the contents of a single web page at any given time is an instance of the Document Object Model (DOM), a tree structure that forms the basis for rendering and processing of the page within the browser and is updated when content is modified. This work presents DOMdiff, an algorithm to identify changes between two different DOM instances, as well as a method to classify these changes in terms of a ranking that represents the distance between the two trees. We compare a manually derived classifier with the results of PRank, a ranked version of the Perceptron algorithm, a simple machine learning approach that generates a multiclass classifier based on formulae in a constrained predicate logic, and the established statistical classifier C5.0. Our results indicate that DOMdiff is suitable to large-scale change identification and that entropy-based statistical classifiers are more accurate than our simple predicate-based classifier for the problem at hand, but require a larger decision tree. We additionally identify a shortcoming of PRank when handling features with low information gain/high entropy.

Correct behaviour of X.509 certificate validation code in SSL/TLS implementations is crucial to ensure secure communication channels. Recently, there have been major efforts in testing these implementations, namely frankencerts and mucerts, which provide new ways to generate test certificates which are likely to reveal errors in the implementations of X.509 validation logic. However, it remains a significant challenge to generate effective test certificates. In this paper, we explore the applicability of a prominent combinatorial method, namely combinatorial testing, for testing of X.509 certificates. We demonstrate that combinatorial testing provides the theoretical guarantees for revealing errors in the certificate validation logic of SSL/TLS implementations. Our findings indicate that the introduced combinatorial testing constructs, coveringcerts, compare favorably to existing testing methods by encapsulating the semantics of the validation logic in the input model and employing combinatorial strategies that significantly reduce the number of tests needed. Besides the foundations of our approach, we also report on experiments that indicate its practical use.

The expectation and the variance of the syndrome weight distribution of linear codes after transmission of codewords through a binary symmetric channel are derived exactly in closed form as functions of the code's parity-check matrix and of the degree distributions of the associated Tanner graph. The influence of (check) regularity of the Tanner graph is studied. Special attention is payed to Tanner graphs that have no cycles of length four. We further study the equivalence of some classes of combinatorial designs and important classes of LDPC codes and apply our general results to those more specific structures. Simulations validate the analytical results and show that the actual cumulative distribution function of the syndrome weight is close to that of a normal distribution.

We present a coverage measurement for TLS cipher suites recommendations provided by various regulatory and intelligence organizations such as the IETF, Mozilla, ENISA, German BSI, and USA NSA. These cipher suites are measured and analyzed using a combinatorial approach, which was made feasible via developing the necessary input models. Besides shedding light on the coverage achieved by the proposed recommendations, we discuss implications towards aspects of test quality. One of them relates to the testing of a TLS implementation, where a system designer or tester should expand the TLS cipher suite registry and integrate the information back to the TLS implementation itself such that the (overall) testing effort is reduced.

Contemporary hardware design shares many similarities with software development. The injection of malicious functionality (Trojans) in FPGA designs is a realistic threat. Established techniques for testing correctness do not cope well with Trojans, since Trojans are not captured in the system model. Furthermore, a well-designed Trojan activates under rare conditions and can escape detection during testing. Such conditions cannot be exhaustively searched, especially in the case of cryptographic core implementations with hundreds of inputs. In this paper, we explore the applicability of a prominent combinatorial strategy, namely combinatorial testing, for FPGA Trojan detection. We demonstrate that combinatorial testing provides the theoretical guarantees for exciting a Trojan of specific lengths by covering all input combinations. Our findings indicate that combinatorial testing constructs can improve the existing FPGA Trojan detection capabilities by reducing significantly the number of tests needed. Besides the foundations of our approach, we also report on first experiments that indicate its practical use.

Security testing of web applications remains a major problem of software engineering. In order to reveal vulnerabilities, manual and automatic testing approaches use different strategies for detection of certain kinds of inputs that might lead to a security breach. In this paper we compared a state-of-the-art manual testing tool with an automated one that is based on model-based testing. The first tool requires user input from the tester whereas the second one reduces the necessary amount of manual manipulation. Both approaches depend on the corresponding test case generation technique and its produced inputs are executed against the system under test (SUT). For this case we enhance a novel technique, which combines a combinatorial testing technique for input generation and a model-based technique for test execution. In this work the input parameter modelling is improved by adding constraints to generate more comprehensive and sophisticated testing inputs. The evaluated results indicate that both techniques succeed in detecting security leaks in web applications with different results, depending on the background logic of the testing approach. Last but not least, we claim that attack pattern-based combinatorial testing with constraints can be an alternative method for web application security testing, especially when we compare our method to other test generation techniques like fuzz testing.

In the past few decades, design theory has grown to encompass a wide variety of research directions. It comes as no surprise that applications in coding theory and communications continue to arise, and also that designs have found applications in new areas. Computer science has provided a new source of applications of designs, and simultaneously a field of new and challenging problems in design theory. In this paper, we revisit a construction for orthogonal designs using the multiplication tables of Cayley-Dickson algebras of dimension $2^n$. The desired orthogonal designs can be described by a system of equations with the aid of a Groebner basis computation. For orders greater than 16 the combinatorial explosion of the problem gives rise to equations that are unfeasible to be handled by traditional search algorithms. However, the structural properties of the designs make this problem possible to be tackled in terms of rewriting techniques, by equational unification. We establish connections between central concepts of design theory and equational unification where equivalence operations of designs point to the computation of a minimal complete set of unifiers. These connections make viable the computation of some types of orthogonal designs that have not been found before with the aforementioned algebraic modelling.

In this work, we initially describe a method patented in cite{Revell2012} for key generation in a symmetric channel between different nodes that are located within a trusted network. In the aftermath, we describe the different phases of the invented method in cryptographic terms and we analyze security aspects with respect to a proper implementation. We conclude by giving some arguments that justify the usage of this method in enterprise communication systems.

Nowadays, variable selection is fundamental to large dimensional statistical modelling problems, since large databases exist in diverse fields of science. In this paper, we benefit from the use of data mining tools and experimental designs in databases in order to select the most relevant variables for classification in regression problems in cases where observations and labels of a real-world dataset are available. Specifically, this study is of particular interest to use health data to identify the most significant variables containing all the necessary important information for classification and prediction of new data with respect to a certain effect (survival or death). The main goal is to determine the most important variables using methods that arise from the field of design of experiments combined with algorithmic concepts derived from data mining and metaheuristics. Our approach seems promising, since we are able to retrieve an optimal plan using only 6 runs of the available 8862 runs.