Data streams are everywhere in modern technologies, spanning from industrial process control to network traffic analysis. Stream clustering is required to describe data streams in real time and maintain accurate knowledge of their underlying structures. However, data streams frequently exhibit non-stationarity, changes in distributions, and the emergence of new classes. These alterations—commonly referred to as "concept drift"—severely disturb algorithms, resulting in inconsistent outcomes and models. We present SDOstreamclust, an incremental algorithm for stream clustering. It inherits the distinctive features of methods founded on Sparse Data Observers, i.e., lightweight, intuitive, self-adjusting, resistant to noise, capable of identifying non-convex clusters, and constructed upon robust parameters and interpretable models. We compare SDOstreamclust with established algorithms and evaluate them with a broad collection of datasets, both real and synthetic. SDOstreamclust shows outstanding performances, a major adaptability to concept drift, and a superior parameter stability and robustness. Often ignored in the evaluation of new methods, concept drift is a major challenge for next-generation algorithms, since it is inherent to evolving data and a main cause of degradation in machine learning. Hence, SDOstreamclust emerges as a major alternative for unsupervised streaming data analysis.

Abstract Anomaly detection has become pervasive in modern technology, covering applications from cybersecurity, to medicine or system failure detection. Before outputting a binary outcome (i.e., anomalous or non-anomalous), most algorithms evaluate instances with outlierness scores. But what does a score of 0.8 mean? Or what is the practical difference compared to a score of 1.2? Score ranges are assumed non-linear and relative, their meaning established by weighting the whole dataset (or a dataset model). While this is perfectly true, algorithms also impose dynamics that decisively affect the meaning of outlierness scores. In this work, we aim to gain a better understanding of the effect that both algorithms and specific data particularities have on the meaning of scores. To this end, we compare established outlier detection algorithms and analyze them beyond common metrics related to accuracy. We disclose trends in their dynamics and study the evolution of their scores when facing changes that should render them invariant. For this purpose we abstract characteristic S-curves and propose indices related to discriminant power, bias, variance, coherence and robustness. We discovered that each studied algorithm shows biases and idiosyncrasies, which habitually persist regardless of the dataset used. We provide methods and descriptions that facilitate and extend a deeper understanding of how the discussed algorithms operate in practice. This information is key to decide which one to use, thus enabling a more effective and conscious incorporation of unsupervised learning in real environments.

Given a set of data points, clustering serves to discover groups based on pairwise similarities and the shapes drawn by the data in the feature space. In other words, it is a tool to describe data and reveal their intrinsic nature in terms of patterns or groups. In this paper, we review the methodology of clustering when used to explore a priori unknown data, i.e., we do not know how data spaces are manipulated, how algorithms are tuned, and how results are validated. Under this practical approach, we examine the advantages of SDOclust, a clustering method that stands out for its simplicity, lightness, no need for parameterization and not being subject to traditional clustering limitations. We test SDOclust and main established alternatives — HDBSCAN, k-means-, Fuzzy C-means, Hierarchical Clustering, CLASSIX, and N2D Deep Clustering — by extensive experimentation with more than 200 datasets, both real and synthetic, that have been collected from the literature on evaluation and represent different data analysis challenges. We submit only SDOclust to unfavorable testing conditions by denying it a parameter tuning phase. Nevertheless, its overall performance is excellent and positions it as one of the best general-purpose alternatives. With deep clustering as the consolidation of a new paradigm, trends in clustering consist mainly in projecting data into spaces that are easier to dissect. Therefore, in cases where the original space does not show clustering-friendly structures and when we can assume transformation costs, SDOclust easily adapts and is a most natural choice to perform the partitioning task.

Abstract Stream clustering is required in applications where data is generated continuously or periodically and must be processed considering its temporal nature. In the absence of a ground truth, internal validation is the only option to evaluate the quality of performances. Traditional internal validation is commonly used also in stream clustering, even in spite of the fact that it becomes inconsistent in the event of data evolution. Recent trends opt for incremental approaches, but these are closer to change detection rather than validation methods and limit themselves by imposing online validation on online analysis. In this work we study the impact of concept drift in the validation of stream clustering and propose the Temporal Silhouette index, therefore making internal validation conform to streaming data. We conduct tests with more than 200 datasets and contrast performances of four popular stream clustering algorithms with seven validation methods (three static internal, three incremental internal, one external) and the proposed index. Results show the suitability of the Temporal Silhouette index for stream clustering validation in the event of concept drift and different types of outliers. The demand for reliable unsupervised learning in applications that process data in streams is ever-increasing, and such reliability inevitably requires the use of validation. This fact highlights the significance of the novel approach proposed in this work.

The detection of anomalies in streaming data faces complexities that make traditional static methods unsuitable due to computational costs and nonstationarity. We test and evaluate eight state of the art algorithms against prominent challenges related to streaming data. Results show insights regarding accuracy, memory-dependency, parameterization, and pre-knowledge exploitation, thus revealing the high impact of some data characteristics to establish a most appropriate algorithm—namely: locality (i.e., whether outlierness is relative to local contexts), relativeness (i.e., if past data defines outlierness), and concept drift (if it is expected, its intensity and frequency). In most applied cases, such factors can be inferred in advance through the use of historical data and domain knowledge. Assuming the viability of the studied methods in terms of time efficiency, this work discloses key findings to achieve optimal designs of streaming data anomaly detection in real-life applications.

In many scenarios like wireless Internet access or encrypted VPN tunnels, encryption is performed on a per-packet basis. While this encryption approach effectively protects the confidentiality of the transmitted payload, it leaves traffic patterns involving inter-arrival times and packet lengths observable, e.g., to eavesdroppers on the air interface. It is a widespread belief that by only observing interleaved packets of different parallel flows, analysis and classification of the corresponding traffic by an eavesdropper is very difficult or close to impossible.In this paper, we show that it is indeed possible to separate packets belonging to different flows purely from patterns observed in the interleaved packet sequence. We devise a novel deep recurrent neural network architecture that allows us to detect individual anomalous packets in a flow. Based on this anomaly detector, we develop an algorithm to find a separation into flows that minimizes the anomaly score indicated by our model. Our experimental results obtained with synthetically crafted flows and real-world network traces indicate that our approach is indeed able to separate flows successfully with high accuracy.Being able to recover a flow's packet sequence from multiple interleaved flows, we show with this paper that the common packet-level encryption might be insufficient in scenarios where high levels of privacy have to be achieved. On the defender's side, our approach constitutes a valuable tool in encrypted traffic analysis, but also contributes a novel neural network architecture in the field of network intrusion detection in general.

Compact data models have become relevant due to the massive, ever-increasing generation of data. We propose Observers-based Data Modeling (ODM), a lightweight algorithm to extract low density data models (aka coresets) that are suitable for both static and stream data analysis. ODM coresets keep data internal structures while alleviating computational costs of machine learning during evaluation phases accounting for a O(n log n) worst-case complexity. We compare ODM with previous proposals in classification, clustering, and outlier detection. Results show the preponderance of ODM for obtaining the best trade-off in accuracy, versatility, and speed.

The Covid-19 crisis has challenged cyber security teaching by creating the need for secure remote access to existing cyber security laboratory infrastructure. In this paper, we present requirements, architecture and key functionalities of a secure remote laboratory access solution that has been instantiated successfully for two existing laboratories at TU Wien. The proposed design prioritizes security and privacy aspects while integrating with existing Moodle eLearning platforms to leverage available authentication and group collaboration features. Performance evaluations of the prototype implementation for real cyber security classes support a first estimate of dimensioning and resources that must be provisioned when implementing the proposed secure remote laboratory access.

In highly security-critical network environments, it is a popular design decision to offload cryptographic tasks like encryption or signature generation to a dedicated trusted module or key server with paramount security features, we in this paper refer to with the general term Cryptographic Key Management Device (CKMD). While this network design yields several benefits, we demonstrate that the use of popular counter mode encryption modes like CTR or GCM can show substantial shortcomings in terms of security when used in conjunction with this network design. In particular, we show how the use of authenticated encryption using GCM enables the possibility of establishing a subliminal channel by exploiting the authentication information within messages. We show how decoding of hidden information can proceed in addition to decryption of overt information without raising authentication failures.

In January 2017 encrypted Internet traffic surpassed non-encrypted traffic. Although encryption increases security, it also masks intrusions and attacks by blocking the access to packet contents and traffic features, therefore making data analysis unfeasible. In spite of the strong effect of encryption, its impact has been scarcely investigated in the field. In this paper we study how encryption affects flow feature spaces and machine learning-based attack detection. We propose a new cross-layer feature vector that simultaneously represents traffic at three different levels: application, conversation, and endpoint behavior. We analyze its behavior under TLS and IPSec encryption and evaluate the efficacy with recent network traffic datasets and by using Random Forests classifiers. The cross-layer multi-key approach shows excellent attack detection in spite of TLS encryption. When IPsec is applied, the reduced variant obtains satisfactory detection for botnets, yet considerable performance drops for other types of attacks. The high complexity of network traffic is unfeasible for monolithic data analysis solutions, therefore requiring cross-layer analysis for which the multi-key vector becomes a powerful profiling core.

The difficulty to validate clustering reliability hinders the adoption of clustering in real-life applications. We propose: (a) a set of symbolic representations to interpret problem spaces and (b) the CluReAL algorithm to refine any clustering result regardless of the used technique. Both approaches are grounded by recently published absolute cluster validity indices. Conducted experiments show how the refinement algorithm improves performances in a wide variety of scenarios and builds more interpretable solutions, whereas symbolic representations are shown to offer explainable summaries of problem contexts. Refinement and interpretability are both crucial to reduce failure and increase performance control and operational awareness in processes that depend on clustering.

The increasing number of different, incompatible congestion control algorithms has led to an increased deployment of fair queuing. Fair queuing isolates each network flow and can thus guarantee fairness for each flow even if the flows' congestion controls are not inherently fair. So far, each queue in the fair queuing system either has a fixed, static maximum size or is managed by an Active Queue Management (AQM) algorithm like CoDel. In this paper we design an AQM mechanism (Learning Fair Qdisc (LFQ)) that dynamically learns the optimal buffer size for each flow according to a specified reward function online. We show that our Deep Learning based algorithm can dynamically assign the optimal queue size to each flow depending on its congestion control, delay and bandwidth. Comparing to competing fair AQM schedulers, it provides significantly smaller queues while achieving the same or higher throughput.

The application of clustering involves the interpretation of objects placed in multi-dimensional spaces. The task of clustering itself is inherently submitted to subjectivity, the optimal solution can be extremely costly to discover and sometimes even unreachable or nonexistent. This fact introduces a trade-off between accuracy and computational effort, moreover given that engineering applications usually work well with suboptimal solutions. In such applied scenarios, cluster validation is mandatory to refine algorithms and ensure that solutions are meaningful. Validity indices are commonly intended to benchmark diverse clustering setups, therefore they are coefficients with a relative nature, i.e., useful when compared to one another. In this paper, we propose a validation methodology that enables absolute evaluations of clustering results. Our method performs geometric measurements of the solution space and provides a coherent interpretation of the data structure by using indices based on inter- and intra-cluster distances, density, and multimodality within clusters. Conducted tests and comparisons with well-known indices show that our validation methodology improves the robustness of the clustering application for knowledge discovery. While clustering is often performed as a black box technique, our index is construable and therefore allows for the implementation of systems enriched with self-checking capabilities.

This paper proposes Reactive Adaptive eXperience based congestion control (Rax), a new method of congestion control (CC) that uses online reinforcement learning (RL) to maintain an optimum congestion window with respect to a given reward function and based on current network conditions. We use a neural network based approach that can be initialized either with random weights or with a previously trained neural network to improve stability and convergence time. As the processing of rewards in CC depends on the arrival of acknowledgements, which are delayed and received one by one, the problem is not suitable for current implementations of Deep RL. As a remedy we propose Partial Action Learning, a formulation of Deep RL that supports delayed and partial rewards. We show that our method converges to a stable, close-to-optimum solution within minutes and outperforms existing CC algorithms in typical networks. Thus, this paper demonstrates that Deep RL can be done online and can compete with classic CC schemes such as Cubic.

The visualization of network traffic flows is an open problem that affects the control and administration of communication networks. Feature vectors used for representing traffic commonly have from tens to hundreds of dimensions and hardly tolerate visual conceptualizations. In this work we use neural networks to obtain extremely low-dimensional data representations that are meaningful from an attack-detection perspective. We focus on a simple Autoencoder architecture, as well as an extension that benefits from pre-knowledge, and evaluate their performances by comparing them with reductions based on Principal Component Analysis and Linear Discriminant Analysis. Experiments are conducted with a modern Intrusion Detection dataset that collects legitimate traffic mixed with a wide variety of attack classes. Results show that feature spaces can be strongly reduced up to two dimensions with tolerable classification degradation while providing a clear visualization of the data. Visualizing traffic flows in two-dimensional spaces is extremely useful to understand what is happening in networks, also to enhance and refocus classification, trigger refined analysis, and aid the security experts' decision-making. We additionally developed a tool prototype that covers such functions, therefore supporting the optimization of network traffic attack detectors in both design and application phases.

Internet Background Radiation (IBR) is observed in empty network address spaces. No traffic should arrive there, but it does in overwhelming quantities, gathering evidences of attacks, malwares and misconfigurations. The study of IBR helps to detect spreading network problems, common vulnerabilities and attack trends. However, network traffic data evolves quickly and is of high volume and diversity, i.e., an outstanding big data challenge. When used to assist network security, it also requires the online classification of dynamic streaming data. In this paper, we introduce an AGgregation & Mode (AGM) vector to represent network traffic. The AGM format characterizes IP hosts by extracting aggregated and mode values of IP header fields, and without inspecting payloads. We performed clustering and statistical analysis to explore six months of IBR from 2012 with the AGM mapping. The discovered patterns allow building a classification of IBR, which identifies phenomena that have been actively polluting the Internet for years. The AGM representation is light and tailored for monitoring and pattern discovery. We show that AGM vectors are suitable to analyze large volumes of network traffic: they capture permanent operations, such as long term scanning, as well as bursty events from targeted attacks and short term incidents.

With the expected massive increase in high-bandwidth applications over 5G cellular networks, the efficient use of radio-network and core-network infrastructures becomes essential. Group communication is a method for transmitting data efficiently from one source to many receivers. In this paper we study the security provided in terms of authenticity and integrity for group communication in 5G networks. We identify that the current security solutions involve trusting the benignity and operational security of network operators as well as its users since the current security solutions do not provide data origin authentication. Based on this insight, we present two attack scenarios in which an adversary exploits the provided level of authenticity such that arbitrary data can be injected maliciously while receivers consider the data as if they were sent by the claimed source. We evaluate potential approaches to provide data origin authentication in 5G and show that future research is required for a general solution.

Smart grids require communication networks for supervision functions and control operations. With this they become attractive targets for attackers. In newer power grids, State Estimation (SE) is often performed based on Kalman Filters (KFs) to deal with noisy measurement data and detect Bad Data (BD) due to failures in the measurement system. Nevertheless, in a setting where attackers can gain access to modify sensor data, they can exploit the fact that SE is used to process the data. In this paper, we show how an attacker can modify Phasor Measurement Unit (PMU) sensor data in a way that it remains undetected in the state estimation process. We show how anomaly detection methods based on innovation gain fail if an attacker is aware of the state estimation and uses the right strategy to circumvent detection.

Covert channels exploit communication protocols to clandestinely transfer information. They enable criminals to hide malicious activities and can be used for secret data exfiltration, malware spreading or for the stealthy establishment of command and control structures. In this paper we study covert timing channels from a statistical perspective and investigate whether they can be identified as anomalies with unsupervised learning methods. We use a testbed to generate covert timing channels based on seven popular techniques and inject them in real captured traffic. Final datasets are analyzed with diverse outlier detection and classification algorithms. Our results show that, based on their statistical properties, covert channels do not occupy low density regions or take extreme values in the problem space, and therefore are not detectable as strong anomalies. However, they present traceable profiles that can be abstracted by supervised learning models. Such findings reveal that facing the detection of novel (and classic) covert timing channels from an anomaly-detection perspective will probably fail or not suffice; instead, they must be identified based on the similarity to known schemes, using supervised and semi-supervised approaches.

To highlight a potential threat to hardware security, we propose a methodology to derive a trigger signal from the behavior of Verilog simulation models of field-programmable gate array (FPGA) primitives that behave X-optimistic. We demonstrate our methodology with an example trigger that is implemented using Xilinx 7 Series FPGAs. Experimental results show that it is easily possible to create a trigger signal that is '0' in simulation (pre- and post-synthesis), and '1' in hardware. We show that this kind of trigger is neither detectable by formal equivalence checks, nor by recent Trojan detection techniques. As a countermeasure, we propose to carefully reconsider the utilization of X-optimism in FPGA simulation models.

Due to the increasing dependency of critical infrastructure on synchronized clocks, network time synchronization protocols have become an attractive target for attackers. We identify data origin authentication as the key security objective and therefore conduct a comprehensive, theoretical evaluation of data origin authentication schemes from different application fields with regard to their applicability to secure broadcast time synchronization. Some evaluated schemes were found to be susceptible to message delay attacks in the context of time synchronization - including TESLA, the approach currently favored by the IETF NTP working group and also on the shortlist of the P1588 Security Subcommittee for PTP. Two of the evaluated schemes, however, come somewhat close to meeting the evaluation criteria derived from our time synchronization specific threat analysis, and therefore qualify as promising candidates to secure broadcast time synchronization.

Abstract Covert channels provide means to conceal information transfer between hosts and bypass security barriers in communication networks. Hidden communication is of paramount concern for governments and companies, because it can conceal data leakage and malware communication, which are crucial building blocks used in cyber crime. We propose detectors based on descriptive analytics of traffic (DAT) to facilitate revealing network and transport layer covert channels originated from a wide spectrum of published data‐hiding techniques. DAT detectors transform communication data into flexible feature vectors that represent traffic by a set of extracted calculations and estimations. For the case of covert channels, the core of the detection is performed by the combined application of autocorrelation calculations and multimodality measures built upon kernel density estimations and Pareto charts. DAT detectors are devised to be embedded as extensions of network intrusion detection systems, being able to perform fast, lightweight analysis of numerous flows. The present paper focuses specifically on TCP/IP traffic and provides suitable classifications of TCP/IP fields and related covert channel techniques from the perspective of the statistical detection. The proposed methodology is evaluated with public traffic datasets as well as covert channels generated according to main techniques described in the related literature. Copyright © 2016 John Wiley & Sons, Ltd.

Modern access network technologies like Long Term Evolution (LTE) and High Speed Packet Access (HSPA) use time-slotting mechanisms to optimize resource sharing and overall network performance. In time-slotted networks, the one-way delay of all packets in a packet stream depends on the absolute point in time when the first packet of the stream is sent. With appropriate feedback signals, applications can exploit this effect to reduce their effective end-to-end delay. Time-critical applications such as real-time sensor data acquisition or voice-over-IP (VoIP) communications can shift their acquisition interval in order to adapt to the network timing. Information about the actual time-slotting periods can be gathered by active network measurements or through implementation of cross-layer information exchange. In this paper, a method is proposed to determine the optimum send time for particular destinations and to support applications in adjusting their send time accordingly. Theoretical findings are supported by the offline analysis of measurement data and by a proof-of-concept implementation that confirms the feasibility and effectiveness of the proposed solution in operational LTE and HSPA networks.

YouTube is the most popular service in today's Internet. Google relies on its massive content delivery network (CDN) to push YouTube videos as close as possible to the end-users, both to improve their watching experience as well as to reduce the load on the core of the network, using dynamic server selection strategies. However, we show that such a dynamic approach can actually have negative effects on the end-user quality of experience (QoE). Through the comprehensive analysis of one month of YouTube flow traces collected at the network of a large European ISP, we report a real case study in which YouTube QoE-relevant degradation affecting a large number of users occurs as a result of Google's server selection strategies. We present an iterative and structured process to detect, characterize, and diagnose QoE-relevant anomalies in CDN distributed services such as YouTube. The overall process uses statistical analysis methodologies to unveil the root causes behind automatically detected problems linked to the dynamics of CDNs' server selection strategies.

The Internet has become a complex system with increasing numbers of end-systems, applications, protocols and types of networks. Although we have a good understanding of how data is transferred over the network we cannot observe what happens with our data after sending and before receiving it - how packets traverse through the network and with which QoS characteristics remains unknown. Towards this objective we have developed a multi-hop packet tracking system intended to be used in experimental facilities, such as PlanetLab, where we have made our first tests. This paper describes our packet tracking realization and the results from our prototype implementation.