Modern government and enterprise networks are the target of sophisticated multi-step attacks called Advanced Persistent Threats (APTs), designed and carried out by expert adversaries. The prolonged nature of APTs results in overwhelming the analyst with an increasingly impractical number of alerts. As a result, the challenge of APT detection is ideal for automation through artificial intelligence (AI). In this paper, we propose the first, up to our knowledge, end-to-end AI-assisted architecture for detecting APTs – RANK. We propose advanced algorithms and solutions for four consecutive sub-problems: 1) alert templating and merging, 2) alert graph construction, 3) alert graph partitioning into incidents, and 4) incident scoring and prioritization. Additionally, we discuss the necessary optimizations and techniques enabling the system to operate in a real-time fashion. We evaluate our architecture against the 2000 DARPA, Mordor, as well as a large number of real-world datasets from enterprise networks. Extensive results are provided showing four orders-of-magnitude reduction in the amount of data to be reviewed, innovative extraction and security-aware scoring of incidents. The extracted incidents can be further used for downstream tasks. In our experiments where we have access to a portion of alert labels, we are able achieve 87% balanced accuracy.

This paper describes an efficient shape representation framework for planar shapes using Voronoi skeletons. A new algorithm for the construction of the Voronoi diagram of a polygon with holes is first described. The main features of this algorithm are its robustness in handling the standard degenerate cases (colinearity of more than two points; co-circularity of more than three points), and its ease of implementation. This algorithm is then used to derive accurate and robust skeletons for planar shapes. The shape representation scheme using Voronoi skeletons possesses the important properties of connectivity as well as Euclidean metrics. The resultant representation is stable with respect to being invariant to perturbations along the boundary of the shape.>