We present an approach for verifying systems at runtime. Our approach targets distributed systems whose components communicate with monitors over unreliable channels, where messages can be delayed, reordered, or even lost. Furthermore, our approach handles an expressive specification language that extends the real-time logic MTL with freeze quantifiers for reasoning about data values. The logic’s main novelty is a new three-valued semantics that is well suited for runtime verification as it accounts for partial knowledge about a system’s behavior. Based on this semantics, we present online algorithms that reason soundly and completely about streams where events can occur out of order. We also evaluate our algorithms experimentally. Depending on the specification, our prototype implementation scales to out-of-order streams with hundreds to thousands of events per second.

The first international Competition on Runtime Verification (CRV) was held in September 2014, in Toronto, Canada, as a satellite event of the 14th international conference on Runtime Verification (RV’14). The event was organized in three tracks: (1) offline monitoring, (2) online monitoring of C programs, and (3) online monitoring of Java programs. In this paper, we report on the phases and rules, a description of the participating teams and their submitted benchmark, the (full) results, as well as the lessons learned from the competition.

No abstract written

Software-defined networking (SDN) eases network management by centralizing the control plane and separating it from the data plane. The separation of planes in SDN, however, introduces new vulnerabilities in SDN networks, since the difference in processing packets at each plane allows an adversary to fingerprint the network's packet-forwarding logic. In this paper, we study the feasibility of fingerprinting the controller-switch interactions by a remote adversary, whose aim is to acquire knowledge about specific flow rules that are installed at the switches. This knowledge empowers the adversary with a better understanding of the network's packet-forwarding logic and exposes the network to a number of threats. In this paper, we collect measurements from hosts located across the globe using a realistic SDN network comprising of OpenFlow hardware and software switches. We show that, by leveraging information from the RTT and packet-pair dispersion of the exchanged packets, fingerprinting attacks on SDN networks succeed with overwhelming probability. We additionally show that these attacks are not restricted to active adversaries, but can also be mounted by passive adversaries that only monitor traffic exchanged with the SDN network. Finally, we discuss the implications of these attacks on the security of SDN networks, and we present and evaluate an efficient countermeasure to strengthen SDN networks against fingerprinting. Our results demonstrate the effectiveness of our countermeasure in deterring fingerprinting attacks on SDN networks.

Prior runtime-verification approaches for distributed systems are limited as they do not account for network failures and they assume that system messages are received in the order they are sent. To overcome these limitations, we present an online algorithm for verifying observed system behavior at runtime with respect to specifications written in the real-time logic MTL that efficiently handles out-of-order message deliveries and operates in the presence of failures. Our algorithm uses a three-valued semantics for MTL, where the third truth value models knowledge gaps, and it resolves knowledge gaps as it propagates Boolean values through the formula structure. We establish the algorithm's soundness and provide completeness guarantees. We also show that it supports distributed system monitoring, where multiple monitors cooperate and exchange their observations and conclusions.

In this paper, we study the feasibility of fingerprinting of controller-switch interactions in SDN networks by a remote adversary whose aim is to acquire knowledge about specific flow rules that are installed at the switches. This knowledge empowers the adversary with a better understanding of the network's packet-forwarding logic and exposes the network to a number of threats. In our study, we collect measurements from hosts located across the globe using a realistic SDN network comprising of OpenFlow hardware switches. We show that, by leveraging information from the RTT and packet-pair dispersion of the exchanged packets, fingerprinting attacks on SDN networks succeed with overwhelming probability. We also show that these attacks are not restricted to active adversaries, but can be equally mounted by passive adversaries that only monitor traffic exchanged with the SDN network. Finally, we sketch an efficient countermeasure to strengthen SDN networks against fingerprinting.

Sharing network resources with user groups, divisions, or even other companies in software defined networking promises better network utilization. Resource sharing is effectively realized by empowering these tenants at the control plane with permissions for administrating network components. However, since the network resources at the data plane are shared and different tenants can have competing objectives, mechanisms are needed to protect the network resources from unauthorized access. In this paper, we propose mechanisms that focus on protecting the network flows, which are determined by the entries installed in the flow tables of the shared switches. To this end, we present an access control scheme, based on the OpenFlow model, for accessing the switches' flow tables and their entries. Our scheme accounts for various security requirements in multi-tenant networks, including requirements on sharing flow table entries for handling network flows, and the resolution of conflicts originating from the reconfiguration of network components.

Runtime monitoring is a general approach to verifying system properties at runtime by comparing system events against a specification formalizing which event sequences are allowed. We present a runtime monitoring algorithm for a safety fragment of metric first-order temporal logic that overcomes the limitations of prior monitoring algorithms with respect to the expressiveness of their property specification languages. Our approach, based on automatic structures, allows the unrestricted use of negation, universal and existential quantification over infinite domains, and the arbitrary nesting of both past and bounded future operators. Furthermore, we show how to use and optimize our approach for the common case where structures consist of only finite relations, over possibly infinite domains. We also report on case studies from the domain of security and compliance in which we empirically evaluate the presented algorithms. Taken together, our results show that metric first-order temporal logic can serve as an effective specification language for expressing and monitoring a wide variety of practically relevant system properties.

Checking whether one formal language is included in another is important in many verification tasks. In this article, we provide solutions for checking the inclusion of languages given by visibly pushdown automata over both finite and infinite words. Visibly pushdown automata are a richer automaton model than the classical finite-state automata, which allows one, for example, to reason about the nesting of procedure calls in the executions of recursive imperative programs. The presented solutions do not rely on explicit automaton constructions for determinization and complementation. Instead, they are more direct and generalize the so-called Ramsey-based inclusion-checking algorithms, which apply to classical finite-state automata and proved to be effective there to visibly pushdown automata. We also experimentally evaluate these algorithms, demonstrating the virtues of avoiding explicit determinization and complementation constructions.

We revisit Schneider’s work on policy enforcement by execution monitoring. We overcome limitations of Schneider’s setting by distinguishing between system actions that are controllable by an enforcement mechanism and those actions that are only observable, that is, the enforcement mechanism sees them but cannot prevent their execution. For this refined setting, we give necessary and sufficient conditions on when a security policy is enforceable. To state these conditions, we generalize the standard notion of safety properties. Our classification of system actions also allows one, for example, to reason about the enforceability of policies that involve timing constraints. Furthermore, for different specification languages, we investigate the decision problem of whether a given policy is enforceable. We provide complexity results and show how to synthesize an enforcement mechanism from an enforceable policy.

IT systems manage increasing amounts of sensitive data and there is a growing concern that they comply with policies that regulate data usage. In this paper, we use temporal logic to express policies and runtime monitoring to check system compliance. While well-established methods for monitoring linearly ordered system behavior exist, a major challenge is monitoring distributed and concurrent systems where actions are locally observed in the different system parts. These observations can only be partially ordered, while policy compliance may depend on the actions' actual order of appearance. Technically speaking, it is in general intractable to check compliance of partially ordered traces. We identify fragments of our policy specification language for which compliance can be checked efficiently, namely, by monitoring a single representative trace in which the observed actions are totally ordered. Through a case study we show that the fragments are capable of expressing nontrivial policies and that monitoring representative traces is feasible on real-world data.

We have previously presented a monitoring algorithm for compliance checking of policies formalized in an expressive metric first-order temporal logic. We explain here the steps required to go from the original algorithm to a working infrastructure capable of monitoring an existing distributed application producing millions of log entries per day. The main challenge is to correctly and efficiently monitor the trace interleavings obtained by totally ordering actions that happen at the same time. We provide solutions based on formula transformations and monitoring representative traces. We also report, for the first time, on statistics on the performance of our monitor on real-world data, providing evidence of its suitability for nontrivial applications.

We show the practical feasibility of monitoring complex security properties using a runtime monitoring approach for metric first-order temporal logic. In particular, we show how a wide variety of security policies can be naturally formalized in this expressive logic, ranging from traditional policies like Chinese Wall and separation of duty to more specialized usage-control and compliance requirements. We also explain how these formalizations can be directly used for monitoring and experimentally evaluate the performance of the resulting monitors.

Various logical theories can be decided by automata-theoretic methods. Notable examples are Presburger arithmetic FO$(Z,+,<)$ and the linear arithmetic over the reals FO$(R,+,<)$, for which effective decision procedures can be built using automata. Despite the practical use of automata to decide logical theories, many research questions are still only partly answered in this area. One of these questions is the complexity of such decision procedures and the related question about the minimal size of the automata of the languages that can be described by formulas in the respective logic. In this paper, we establish a double exponential upper bound on the automata size for FO$(R,+,<)$ and an exponential upper bound for the discrete order over the integers FO$(Z,<)$. The proofs of these upper bounds are based on Ehrenfeucht-Fraiss{'\e} games. The application of this mathematical tool has a similar flavor as in computational complexity theory, where it can often be used to establish tight upper bounds of the decision problem for logical theories.

We introduce a novel approach to the runtime monitoring of complex system properties. In particular, we present an online algorithm for a safety fragment of metric first-order temporal logic that is considerably more expressive than the logics supported by prior monitoring methods. Our approach, based on automatic structures, allows the unrestricted use of negation, universal and existential quantification over infinite domains, and the arbitrary nesting of both past and bounded future operators. Moreover, we show how to optimize our approach for the common case where structures consist of only finite relations, over possibly infinite domains. Under an additional restriction, we prove that the space consumed by our monitor is polynomially bounded by the cardinality of the data appearing in the processed prefix of the temporal structure being monitored.

Automata provide a decision procedure for Presburger arithmetic. However, until now only crude lower and upper bounds were known on the sizes of the automata produced by the automata-based approach for Presburger arithmetic. In this article, we give an upper bound on the number of states of the minimal deterministic automaton for a Presburger arithmetic formula. This bound depends on the length of the formula and the quantifiers occurring in it. We establish the upper bound by comparing the automata for Presburger arithmetic formulas with the formulas produced by a quantifier-elimination method. We show that our bound is tight, also for nondeterministic automata. Moreover, we provide automata constructions for atomic formulas and establish lower bounds for the automata for linear equations and inequations.

Automata provide an effective mechanization of decision procedures for Presburger arithmetic. However, only crude lower and upper bounds are known on the sizes of the automata produced by this approach. In this paper, we prove that the number of states of the minimal deterministic automaton for a Presburger arithmetic formula is triple exponentially bounded in the length of the formula. This upper bound is established by comparing the automata for Presburger arithmetic formulas with the formulas produced by a quantifier elimination method. We also show that this triple exponential bound is tight (even for nondeterministic automata). Moreover, we provide optimal automata constructions for linear equations and inequations.