The last decade has seen the increasing application of machine learning to various tasks, including network anomaly detection . But anomaly detection methods based on a single machine learning algorithm usually fail to achieve good results, since network traffic have complex and changeable patterns. Therefore, many solutions based on ensemble learning have been proposed to address this problem. However, most previous studies have the main drawback that they overlook the similarity between the weak classifiers , which may degrade the detection performance. What is more, most existing works use offline and supervised algorithms, which means a large number of computing resources and reliable labels are necessary during the training period. In this paper, we propose ADSim , an online, unsupervised, and similarity-aware network anomaly detection algorithm based on ensemble learning. For a similarity-aware scheme, the target of ADSim can be intuitively described as recognizing the similar weak classifiers during the training phase and treat them as a whole. To achieve this, ADSim first incrementally maintains a distance matrix to record the similarity between the classifiers in the training phase and uses Hierarchy Clustering to group the similar classifiers. In the detecting phase, each cluster will be assigned a weight depending on the consistency of the detection results of the classifiers within it. Moreover, the working procedure of ADSim is online and unsupervised, which significantly improves its practicality. We test ADSim on two datasets, MAWILab and CIC-IDS-2017. The results show that ADSim outperforms the state-of-the-art ensemble learning methods and has ideal runtime performance.

With Internet-of-Things (IoT) devices gaining popularity, dedicated monitoring systems which accurately detect intrusion traffic for them are in high demand. Existing methods mainly use statistical spatial-temporal traffic features and machine learning models. Their practicality has been limited due to the lack of detection ability for stealthy and tricky attacks, diagnostic utility and long-term performance. To address these problems and motivated by the simplicity of mini IoT devices, we propose to construct fully packet-level models to profile traffic patterns for IoT devices by constructing automaton for short flow and long flow, where the length and direction of each packet are the representative features. We apply these fine-grained models to design and develop a traffic monitoring system, namelyIoTa, to detect intrusion traffic for IoT devices.IoTamatches the ongoing traffic with patterns extracted from normal traffic traces. With visible and interactive traffic profiles,IoTacan generate interpretable alerts and is available for long-term use under reasonable human efforts. Evaluations on dozens of common IoT devices show thatIoTacan achieve excellent detection accuracy (nearly perfect recalls and always over 0.999 precisions) for various intrusion traffic covering the complete kill chains. Incorrect detection results can be compensated for by error recovery mechanisms and the understandable alert context can be used by the operator to enhance the system. The diagnostic utility and little alert weariness are recognized by the experienced operators.

Recent years have witnessed the proliferation of smart home ecosystems. Well-characterized traffic generated by smart home devices has promoted the development of security enhancing techniques for smart homes but exposes users to the privacy disclosure risk at the same time. Malicious eavesdroppers can infer working states of smart home devices and user activities based on spatial-temporal traffic characteristics. Existing countermeasures towards this kind of side channel attack ignore the utility of smart home traffic profiles and signatures for network management and attempt to completely eliminate them. In this paper, we give a comprehensive study on the trade offs between the usability of smart home traffic for security monitoring and its privacy threat. We propose to monitor the smart homes under differential privacy. Based on our solution, decoy traffic can be generated in a controlled manner so as to confound the attackers without disturbing the running monitoring systems. We prototyped our proposal and demonstrate its effectiveness empirically. An interview study is also conducted to learn the user acceptance of the proposed privacy preserving mechanism.

A tremendous amount of Internet-of-Things (IoT) devices have been deployed in recent years, bringing new challenges for network management and cyber security. It is important for network managers to know what types of IoT devices are connecting to the network. Despite much research efforts, previous works place more emphasis on accuracy but ignore some other performance indicators also in high demand, like efficiency, robustness, adaptability to special scenarios and extensibility for new devices. In this paper, we propose a practical IoT device identification system, namely ByteIoT, based on a simple but well-organized traffic feature, i.e., the frequency distribution of bidirectional packet lengths. ByteIoT applies k-nearest neighbors algorithm as the classifier to gain extensibility and adaptability. We evaluate ByteIoT on several datasets and the results show that ByteIoT can outperform other state-of-the-art methods in the aspects of accuracy, efficiency, extensibility and adaptability.

Fast IPv4 scanning significantly improves network measurement and security research. Nevertheless, it is infeasible to perform brute-force scanning of the IPv6 address space. Alternatively, one can find active IPv6 addresses through scanning the candidate addresses generated by state-of-the-art algorithms. However, the probing efficiency of such algorithms is often very low. In this paper, our objective is to improve the probing efficiency of IPv6 addresses. We first perform a longitudinal active measurement study and build a high-quality dataset, hitlist, including more than 1.95B IPv6 addresses distributed in 58.2K BGP prefixes and collected over 17 months period. Different from the previous works, we probe the announced BGP prefixes using a pattern-based algorithm. This results in a dataset without uneven address distribution and low active rates. Further, we propose an efficient address generation algorithm, DET, which builds a density space tree to learn high-density address regions of the seed addresses with linear time complexity and improves the active addresses’ probing efficiency. We then compare our algorithm DET against state-of-the-art algorithms on the public hitlist and our hitlist by scanning 50M addresses. Our analysis shows that DET increases the de-aliased active address ratio and active address (including aliased addresses) ratio by 10%, and 14%, respectively. Furthermore, we develop a fingerprint-based method to detect aliased prefixes. The proposed method for the first time directly verifies whether the prefix is aliased or not. Our method finds that 10.64% of the public aliased prefixes are false positive.

With the rapid proliferation of IoT devices, device management and network security are becoming significant challenges. Knowing how many IoT devices are in the network and whether they are behaving normally is significant. IoT device identification is the first step to achieve these goals. Previous IoT identification works mainly use supervised learning and need lots of labeled data. Considering collecting labeled data is time-consuming and cannot be scaled, in this paper, we propose an IoT identification model based on semi-supervised learning. The model can differentiate IoT and non-IoT and classify specific IoT devices based on time interval features, traffic volume features, protocol features and TLS related features. The evaluation in a public dataset shows that our model only needs 5% labeled data and gets accuracy over 99%.