Abstract The choice of one’s programming language and relative ecosystem of libraries can affect the likelihood of encountering a critical vulnerability. Simply counting the vulnerabilities by mining a software repository is not enough, and case-control studies are a well-accepted methodology to determine relative risk. Yet, they require the ability to compare ‘equals with equals’ as a library for text processing is likely subject to less security scrutiny than a library for web applications. To compare libraries, we implemented a human-guided protocol to transfer classification categories from an ecosystem to libraries of another ecosystem. By building of this categorization, we performed a case-control study with the vulnerabilities available on Snyk and with status ’reviewed’ in the Github security Advisories till 2024. We mapped 76 Java/Maven libraries and 221 Python/PyPI packages as ’cases’ (libraries with vulnerabilities with a CVSS critical score) compared them against 58 Java/Maven and 166 Python/PyPI ’controls’ (Only with a high CVSS score). We found and overall the odds ratio of ending with a critical vulnerability is slightly higher when using a Java/Maven library in comparison to using a Python/PyPi package (1.13x). We refine the analysis to understand possible reasons for our result by using the CVSS vector metric. A possible explanation is that a vulnerability with low attack complexity has disproportionately higher chances to be critical in Java/Maven (38.9x) than in Python/PyPI (5.9x). Such results might be explained by the lack of past security interest in the Python ecosystem. By using the introduction of the OWASP dependency checker in 2023 for Python as possible indication of community interest, we found a risk reversal: after 2023 the risk of ending with a critical vulnerability (as opposed to just a high severity one) is significantly higher (2.4x) for a Python/PyPI package than for a Java/Maven library. To allow replication and updates, we make the dataset and the protocol individual steps available as open data.

Abstract Statistical model checking estimates probabilities and expectations of interest in probabilistic system models by using random simulations. Its results come with statistical guarantees. However, many tools use unsound statistical methods that produce incorrect results more often than they claim. In this paper, we provide a comprehensive overview of tools and their correctness, as well as of sound methods available for estimating probabilities from the literature. For expected rewards, we investigate how to bound the path reward distribution to apply sound statistical methods for bounded distributions, of which we recommend the Dvoretzky-Kiefer-Wolfowitz inequality that has not been used in SMC so far. We prove that even reachability rewards can be bounded in theory, and formalise the concept of limit-PAC procedures for a practical solution. The modes SMC tool implements our methods and recommendations, which we use to experimentally confirm our results.

To address the issue of educating and training new experts in cybersecurity, it is crucial to identify the specific educational needs of the various professions that exist in the field. We measure these needs by analysing six cybersecurity-related job profiles—each with its own specific skill requirements—that have been assessed by academic and industrial organisations from the cybersecurity community in 14 European countries. We find that it is possible to identify a series of “transversal” skills relevant to all job profiles, and thus of utmost importance in the cybersecurity curricula. However, we also observe that academic and industrial priorities differ substantially, and that skills related to the area of Human security do not rank particularly high, possibly exposing the difficulty of integrating such concepts in traditional education.

Numerous analysis methods for quantitative attack tree analysis have been proposed. These algorithms compute relevant security metrics, i.e., performance indicators that quantify how good the security of a system is; typical metrics being the most likely attack, the cheapest, or the most damaging one. However, existing methods are only geared towards specific metrics or do not work on general attack trees. This article classifies attack trees in two dimensions: proper trees versus directed acyclic graphs (i.e., with shared subtrees); and static versus dynamic gates. For three out of these four classes, we propose novel algorithms that work over a generic attribute domain, encompassing a large number of concrete security metrics defined on the attack tree semantics; dynamic attack trees with directed acyclic graph structure are left as an open problem. We also analyse the computational complexity of our methods.

Abstract Dynamic fault trees (DFTs) are widely adopted in industry to assess the dependability of safety-critical equipment. Since many systems are too large to be studied numerically, DFTs dependability is often analysed using Monte Carlo simulation. A bottleneck here is that many simulation samples are required in the case of rare events, e.g. in highly reliable systems where components seldom fail. Rare event simulation (RES) provides techniques to reduce the number of samples in the case of rare events. In this article, we present a RES technique based on importance splitting to study failures in highly reliable DFTs, more precisely, on a variant of repairable fault trees (RFT). Whereas RES usually requires meta-information from an expert, our method is fully automatic. For this, we propose two different methods to derive the so-called importance function. On the one hand, we propose to cleverly exploit the RFT structure to compositionally construct such function. On the other hand, we explore different importance functions derived in different ways from the minimal cut sets of the tree, i.e., the minimal units that determine its failure. We handle RFTs with Markovian and non-Markovian failure and repair distributions—for which no numerical methods exist—and implement the techniques on a toolchain that includes the RES engine FIG, for which we also present improvements. We finally show the efficiency of our approach in several case studies.

Numerous analysis methods for quantitative attack tree analysis have been proposed. These algorithms compute relevant security metrics, i.e. performance indicators that quantity how good the security of a system is, such as the most likely attack, the cheapest, or the most damaging one. This paper classifies attack trees in two dimensions: proper trees vs. directed acyclic graphs (i.e. with shared subtrees); and static vs. dynamic gates. For each class, we propose novel algorithms that work over a generic attribute domain, encompassing a large number of concrete security metrics defined on the attack tree semantics. We also analyse the computational complexity of our methods.

Abstract Statistical model checking uses Monte Carlo simulation to analyse stochastic formal models. It avoids state space explosion, but requires rare event simulation techniques to efficiently estimate very low probabilities. One such technique is $$\textsc {Restart}$$ R E S T A R T . Villén-Altamirano recently showed—by way of a theoretical study and ad-hoc implementation—that a generalisation of $$\textsc {Restart}$$ R E S T A R T to prolonged retrials offers improved performance. In this paper, we demonstrate our independent replication of the original experimental results. We implemented $$\textsc {Restart}$$ R E S T A R T with prolonged retrials in the and tools, and apply them to the models used originally. To do so, we had to resolve ambiguities in the original work, and refine our setup multiple times. We ultimately confirm the previous results, but our experience also highlights the need for precise documentation of experiments to enable replicability in computer science.

Fault Tree Analysis (FTA) is a prominent technique in industrial and scientific risk assessment. Repairable Fault Trees (RFT) enhance the classical Fault Tree (FT) model by introducing the possibility to describe complex dependent repairs of system components. Usual frameworks for analyzing FTs such as BDD, SBDD, and Markov chains fail to assess the desired properties over RFT complex models, either because these become too large, or due to cyclic behaviour introduced by dependent repairs. Simulation is another way to carry out this kind of analysis. In this paper we review the RFT model with Repair Boxes as introduced by Daniele Codetta-Raiteri. We present compositional semantics for this model in terms of Input/Output Stochastic Automata, which allows for the modelling of events occurring according to general continuous distribution. Moreover, we prove that the semantics generates (weakly) deterministic models, hence suitable for discrete event simulation, and prominently for rare event simulation using the FIG tool.

This paper introduces the statistical model checker FIGV, that estimates transient and steady-state reachability properties in stochastic automata. This software tool specialises in Rare Event Simulation via importance splitting, and implements the algorithms RESTART and Fixed Effort. FIG is push-button automatic since the user need not define an importance function: this function is derived from the model specification plus the property query. The tool operates with Input/Output Stochastic Automata with Urgency, aka IOSA models, described either in the native syntax or in the JANI exchange format. The theory backing FIG has demonstrated good efficiency, comparable to optimal importance splitting implemented ad hoc for specific models. Written in C++, FIG can outperform other state-of-the-art tools for Rare Event Simulation.

Dynamic fault trees (DFT) are widely adopted in industry to assess the dependability of safety-critical equipment. Since many systems are too large to be studied numerically, DFTs dependability is often analysed using Monte Carlo simulation. A bottleneck here is that many simulation samples are required in the case of rare events, e.g. in highly reliable systems where components fail seldomly. Rare event simulation (RES) provides techniques to reduce the number of samples in the case of rare events. We present a RES technique based on importance splitting, to study failures in highly reliable DFTs. Whereas RES usually requires meta-information from an expert, our method is fully automatic: By cleverly exploiting the fault tree structure we extract the so-called importance function. We handle DFTs with Markovian and non-Markovian failure and repair distributions—for which no numerical methods exist—and show the efficiency of our approach on several case studies.

Abstract Statistical model checking avoids the state space explosion problem in verification and naturally supports complex non-Markovian formalisms. Yet as a simulation-based approach, its runtime becomes excessive in the presence of rare events, and it cannot soundly analyse nondeterministic models. In this article, we present : a statistical model checker that combines fully automated importance splitting to estimate the probabilities of rare events with smart lightweight scheduler sampling to approximate optimal schedulers in nondeterministic models. As part of the Modest Toolset, it supports a variety of input formalisms natively and via the Jani exchange format. A modular software architecture allows its various features to be flexibly combined. We highlight its capabilities using experiments across multi-core and distributed setups on three case studies and report on an extensive performance comparison with three current statistical model checkers.