Modern techniques in intrusion and DoS (Denial of Service) detection tend to be either supervised or semi-supervised, i.e., they require training and labelled data. In this work, we study the problem of correlating security attacks with anomalies reported at runtime by a fully unsupervised outlier detection module, i.e., a component that does not require any training at all. Through a concrete proof-of-concept case study, we demonstrate that unsupervised anomaly detection is both efficient and effective, but still, it needs to be combined with additional mechanisms to yield a complete intrusion detection and prevention solution.

We introduce PROUD, standing for PaRallel OUtlier Detection for streams, which is an extensible engine for continuous multi-parameter parallel distance-based outlier (or anomaly) detection tailored to big data streams. PROUD is built on top of Flink. It defines a simple API for data ingestion. It supports a variety of parallel techniques, including novel ones, for continuous outlier detection that can be easily configured. In addition, it graphically reports metrics of interest and stores main results into a permanent store to enable future analysis. It can be easily extended to support additional techniques. Finally, it is publicly provided in open-source.

We investigate the problem of exact set similarity joins using a co-process CPU-GPU scheme. We focus on large instances of the problem, i.e., using datasets of >1M entries, which may take hours to complete if not approached with care, due to the inherent quadratic complexity of the problem. We introduce a novel CPU-GPU co-process scheme, which performs initial filtering and indexing on the CPU and delegates final verification to the GPU. Further, we show that this scheme improves upon the state-of-the-art in both the CPU and GPU standalone solutions in several cases.