Stateful stream process engines in high availability clusters (HACs) track a large number of concurrent flow states and replicate them to backups to provide reliable functionality. Under high traffic loads, existing solutions in such HACs are expensive owing to precise stateful replication. This work presents two novel methods to address this issue: randomization on replication representation and a replication scheme designed for when system becomes overloaded. A hashing structure called Multilevel Counting Bloom Filter (MLCBF) is proposed as a low resource-consuming solution of stateful replication. Its performance and tradeoffs are then evaluated based on theoretic analysis and extensive trace-based tests. Trace-based simulation reveals that MLCBF reduces network and memory requirements of replication typically by over 90 percent for URL categorization. Most importantly, MLCBF is quite as simple and practical for implementation and maintenance. Moreover, an adaptive scheme called dynamic lazy insertion is designed to prevent replication from overloading system continuously and optimize the throughput of HAC. Testbed evaluation demonstrates its feasibility and effectiveness in an overloaded HAC.

To provide the reliable connectivity between two endpoints over the Internet, a firewall cluster for stateful high availability removes the single-point failure by replicating and maintaining TCP connection states to a backup firewall node, at the expense of the costs of network and system resources. In this paper, through trace-based simulations on a prototype implementation, we evaluate the overheads of different state replication methods with a tunable time-triggering parameter. Our evaluation results show that the overheads of precise replication are very high, especially for short flows. We find that a compact data structure employing randomization, a small delay on the replication operations, and host-level aggregation yield significant overhead reductions. Typically, the policy of delayed replication reducing 50% and 74.4% of bandwidth costs only excludes 1.9% and 3.4% of the protection on the pass-through traffic, respectively. These schemes and policies are efficient for alleviating peak system load, reducing the replication bandwidth consumption and still protecting the majority of Internet traffic bytes.

This article presents a novel mechanism to perform packet content inspection by longest prefix matching (LPM) technology. It is done by transforming the automaton-based state table lookup problem into the famous LPM table lookup problem. Two key features, symbol-wise prefix and magic state are observed on the state table to make it possible to utilize IP lookup techniques for string matching. The proposed mechanism is verified to be effective through Lulea algorithm. Also, the practicability is evaluated by employing realistic attack signatures and traffic traces. The experimental results indicate that a state table constructed from the Snort 2.4 patterns can be converted into a prefix table that requires only 2.5% of the memory utilized in the original state table. Compared with the state-of-the-art researches, the proposed scheme has more than 3 times of efficiency, achieving a better balance between required memory size and throughput rate.