Modern government and enterprise networks are the target of sophisticated multi-step attacks called Advanced Persistent Threats (APTs), designed and carried out by expert adversaries. The prolonged nature of APTs results in overwhelming the analyst with an increasingly impractical number of alerts. As a result, the challenge of APT detection is ideal for automation through artificial intelligence (AI). In this paper, we propose the first, up to our knowledge, end-to-end AI-assisted architecture for detecting APTs – RANK. We propose advanced algorithms and solutions for four consecutive sub-problems: 1) alert templating and merging, 2) alert graph construction, 3) alert graph partitioning into incidents, and 4) incident scoring and prioritization. Additionally, we discuss the necessary optimizations and techniques enabling the system to operate in a real-time fashion. We evaluate our architecture against the 2000 DARPA, Mordor, as well as a large number of real-world datasets from enterprise networks. Extensive results are provided showing four orders-of-magnitude reduction in the amount of data to be reviewed, innovative extraction and security-aware scoring of incidents. The extracted incidents can be further used for downstream tasks. In our experiments where we have access to a portion of alert labels, we are able achieve 87% balanced accuracy.

Advanced Persistent Threats (APTs) refer to sophisticated, prolonged and multi-step attacks, planned and executed by skilled adversaries targeting government and enterprise networks. Attack graphs' topologies can be leveraged to detect, explain and visualize the progress of such attacks. However, due to the abundance of false-positives, such graphs are usually overwhelmingly large and difficult for an analyst to understand. Graph partitioning refers to the problem of reducing the graph of alerts to a set of smaller incidents that are easier for an analyst to process and better represent the actual attack plan. Existing approaches are oblivious to the security-context of the problem at hand and result in graphs which, while smaller, make little sense from a security perspective. In this paper, we propose an optimization approach allowing us to generate security-aware partitions, utilizing aspects such as the kill chain progression, number of assets involved, as well as the size of the graph. Using real-world datasets, the results show that our approach produces graphs that are better at capturing the underlying attack compared to state-of-the-art approaches and are easier for the analyst to understand.

Intrusion Detection Systems (IDS) generate a high volume of alerts that security analysts do not have the resources to explore fully. Modelling attacks, especially the coordinated campaigns of Advanced Persistent Threats (APTs), in a visually-interpretable way is a useful approach for network security. Graph models combine multiple alerts and are well suited for visualization and interpretation, increasing security effectiveness. In this paper, we use feature embeddings, learned from network event logs, and community detection to construct and segment alert graphs of related alerts and networks hosts. We posit that such graphs can aid security analysts in investigating alerts and may capture multiple aspects of an APT attack. The eventual goal of this approach is to construct interpretable attack graphs and extract causality information to identify coordinated attacks.

Performing system diagnosis is a critical task in modern datacenters. Investigating individual resource behavior may not be efficient in detecting abnormal behavior in large and complex datacenters. In this paper, we propose a scalable graph based diagnosis framework to detect system anomalies in Software-Defined Infrastructure running in SAVI testbed. We have leveraged Graph Mining and Machine Learning techniques in our approach in order to detect different kinds of anomalies. We have experimentally tested our framework on several use cases: Webserver-Database workload pattern, bandwidth throttling between a pair of VMs, denial-of-service (DoS) attack on a webserver and Spark Job failure. Our framework was able to detect the aforementioned anomalies accurately.

Wireless virtualization is a promising approach to foster innovation and prevent the ossification of wireless networks. Within a virtualized wireless network, multiple network slices, or virtual operators (VO), are co- hosted on the same physical infrastructure. A fundamental question in this environment is which multiplexing technique, TDMA, FDMA or SDMA, should be used to slice the network among the VOs. Another related question is how should the stochastic arrival process affect the slicing and QoS criteria. To answer these two questions, we study the problem of QoS-aware joint admission control and network slicing. Due to the NP- hardness of the problem, we approach it using a heuristic algorithm composed of three steps: spectrum allocation, admission control and spatial multiplexing. The proposed algorithm incorporates the effects of QoS and stochastic traffic. We study through simulations the benefits of joint spatial- frequency multiplexing over the static frequency slicing approach. Finally, our simulation results help shed some light on the trade-offs between frequency and spatial multiplexing as well as between QoS and utilization.

The software-defined approach of cloud radio access networks (C-RANs) enables supporting multiple virtual operators (VOs) on the same physical infrastructure. In this shared environment, a coordinator is needed to manage the sharing of resources between the VOs. Designing a coordinator is about striking a good balance between the flexibility given to the VOs, and the efficiency of the resource utilization. In this paper, we study the problem of coordinated scheduling in the multi-operator cloud-RAN environment. We formulate the problem as a distributed scheduling performed by the VOs, after which they submit their resource requests to a centralized coordinator. The coordinator selects a subset of non-conflicting requests for transmission. We show that the problem in the general case is NP-hard. We then discuss two special cases and relate them to the existing communication protocols. By gaining insights from these two special cases, we propose a general heuristic, which works on any formulation of the problem, and is still able to provide close-to-optimum performance in the special cases we considered. The heuristic is shown to have some similarities to the neuro-computation techniques such as Hopfield-network. Finally, simulation results are provided to show the efficiency of the proposed algorithms.

Cloud Radio Access Networks (C-RAN) promise to leverage cloud computing capabilities for enhancing the quality and coverage of next generation 5G networks. 5G networks shall witness an increasing density of users and access points, very small latencies, more bandwidth resources, and the use of virtualized hardware for baseband processing. Within such an environment, the problem of scheduling the network users across the radio resources might become a bottleneck of the system. In this paper, we study the problem of fully distributed scheduling in C-RAN systems. The idea is that each user's base band processing unit (BBU) tries to guess whether its user should be scheduled or not. First, we focus on the case of maximum throughput scheduling and Rayleigh channels, and provide closed-form expression for the expected effective channel and signal-to-noise ratio (SNR) in the distributed scenario. In order to deal with general channels and schedulers, we adopt the classification techniques from machine learning. We discover an interesting relationship between the fairness of the scheduler, and its ability to be distributed. In particular, schedulers which are more fair are also more prune to prediction errors in the distributed scenario. Finally, we provide simulation results that show that distributed scheduling can provide up to 90% of the performance of the centralized case.

Cloud Radio Access Networks (Cloud-RAN) promise to leverage cloud computing capabilities to enhance the quality and coverage of wireless networks. A dense network of remote radio heads (RRHs) ensures less attenuation at the receiver side. However, two drawbacks are associated with such dense network: the first is the high energy consumption associated with such a large number of RRHs; the second is the interference experienced by the receiver due to close proximity of the transmitters. To address these challenges, we study the problem of joint activation and clustering of RRHs. Since the problem is NP-hard, we provide a two-step algorithm that can find an efficient solution. The first step uses linear-programming relaxation to find a feasible solution. The second step is a greedy approach to improve the utility function through gradual activation-clustering of RRHs. Our simulation results demonstrate the benefit in the joint design of activation and clustering over existing activation only approaches.

The great potential of multi-cell processing in Coordinated Multi-point (CoMP) transmission is hindered by the limited bandwidth of the backhaul network. In this paper, we tackle this problem from two perspectives. First, we provide a thorough analytical study of the expected capacity when employing zero forcing (ZF) beamforming based on quantized channel state information (CSI). We derive simple yet accurate expression for the average capacity per user when Zero-forcing precoders are employed. Then, we propose a low-complexity algorithm to share the backhaul bandwidth between the different users in a fair manner. The algorithm does water-filling for the backhaul bandwidth among users based on their signal-to-interference ratio (SIR), with a target of Equal SIR for all users. We analytically prove that the proposed algorithm is more fair than the standard techniques for backhaul sharing. Simulation results are provided to asses the analytical findings about the superiority of our algorithm in terms of both fairness and sum rate.

Congestion in a transportation network is usually the result of either an increase in traffic demand, i.e. the desire of drivers to use the transportation network, or a decrease in traffic supply, i.e. the traffic capacity, which is affected by weather conditions, incidents, etc. In either case, congestion reduces the efficiency of the transportation network and increases the travel time of vehicles in the network. In this paper, we leverage the benefits that Vehicle-to-Vehicle (V2V) and Vehicle-to-Infrastructure (V2I) communications provide in Intelligent Transportation Systems (ITS), to optimize traffic assignment in transportation networks. In particular, we formulate a convex optimization problem for a transportation network and minimize network criticality, a new graph metric that measures centrality. The robustness of this solution is studied and compared to that of the System Optimal Equilibrium (SOE) Optimization. The results show that using network criticality provides robustness (lack of sensitivity) to both increases in traffic demand and decreases in traffic supply, thus reducing traffic congestion.

Coordinated beamforming in multipoint MIMO networks has been introduced to increase the overall capacity of wireless networks. In coordinated beamforming, the channel state information between the different MIMO access points/base stations in one hand, and the mobile stations on the other hand, needs to be shared among the MIMO nodes. A “backhaul” between different MIMO access points is used to transfer the channel state information. The channel state information of different links is quantized with different quantization steps according to a specific optimization criteria. This information is then shared through the backhaul. In this paper, we study the problem of allocating the backhaul bandwidth among users in coordinated beamforming MIMO multipoint networks. First, we prove through mathematical analysis, that there are many allocations that can provide “near maximum sum rate”. These different allocations vary significantly in fairness between users. A “fair” allocation is an allocation that provides a small variance between the rates of different users. Motivated by this finding, we introduce two novel, low complexity, backhaul bandwidth distribution schemes that can achieve a very close to maximum sum rate, and at the same time, offer throughput fairness among users. Simulation results show that, for the same sum-rate, the proposed schemes can achieve more fairness among users when compared to the conventional scheme, which gives all users the same share of bandwidth. Moreover, we show that one of the proposed schemes, namely the Equal SIR scheme, can achieve zero variance among users in a wide range of backhaul bandwidths while keeping a very close to maximum sum rate.