The last decade has seen the increasing application of machine learning to various tasks, including network anomaly detection . But anomaly detection methods based on a single machine learning algorithm usually fail to achieve good results, since network traffic have complex and changeable patterns. Therefore, many solutions based on ensemble learning have been proposed to address this problem. However, most previous studies have the main drawback that they overlook the similarity between the weak classifiers , which may degrade the detection performance. What is more, most existing works use offline and supervised algorithms, which means a large number of computing resources and reliable labels are necessary during the training period. In this paper, we propose ADSim , an online, unsupervised, and similarity-aware network anomaly detection algorithm based on ensemble learning. For a similarity-aware scheme, the target of ADSim can be intuitively described as recognizing the similar weak classifiers during the training phase and treat them as a whole. To achieve this, ADSim first incrementally maintains a distance matrix to record the similarity between the classifiers in the training phase and uses Hierarchy Clustering to group the similar classifiers. In the detecting phase, each cluster will be assigned a weight depending on the consistency of the detection results of the classifiers within it. Moreover, the working procedure of ADSim is online and unsupervised, which significantly improves its practicality. We test ADSim on two datasets, MAWILab and CIC-IDS-2017. The results show that ADSim outperforms the state-of-the-art ensemble learning methods and has ideal runtime performance.

Fine-grained attack detection is an important network security task. A large number of machine learning/deep learning( ML/DL) based algorithms have been proposed. However, attacks not present in the training set pose a challenge to the model (openset problem). Further, ML/DL based models face the problem of adversarial attacks. Despite the large amount of work attempting to address these problems, there are still some challenges as follows. First, the open-set problem in fine-grained attack detection is difficult to solve because there is no effective representation of the distribution of unknown attacks. Second, in the open set environment, how the fine-grained attack detection model resists the adversarial attack is a more difficult problem. For example, the presence of unknown attacks poses a challenge for adversarial defense. For these reasons, we propose the RFG-HELAD model, which consists of aKclassification model based on deep neural network (DNN) with contrastive learning (CL), and aK+ 1 classification model combining a generative adversarial networks (GAN) with two discriminators and deepk-nearest neighbors (Deep kNN). Among them, Deep kNN uses latent features from GAN and contrastive learning as input, which is essentially a distance-based out-of-distribution detection algorithm used to determine unknown attacks. The large category of unknown attacks has been added to theKclassification, so it is aK+ 1 classification. To further improve the robustness of the RFG-HELAD model, we perform Fourier transform as well as feature fusion on the features, and also conduct adversarial training on theKclassification model. Generative adversarial training of our GAN model can implicitly defend against adversarial attack. Experiments show that our model is superior to other state-of-the-art (SOTA) models in the presence of unknown attacks as well as under adversarial attacks. Especially, our model improves the accuracy by at least 18.7% over the corresponding SOTA model with adversarial defense. Further, we discuss the grounded deployment of the model and demonstrate its feasibility.

Unsupervised Deep Learning (DL) techniques have been widely used in various security-related anomaly detection applications, owing to the great promise of being able to detect unforeseen threats and superior performance provided by Deep Neural Networks (DNN). However, the lack of interpretability creates key barriers to the adoption of DL models in practice. Unfortunately, existing interpretation approaches are proposed for supervised learning models and/or non-security domains, which are unadaptable for unsupervised DL models and fail to satisfy special requirements in security domains.

With the increasing network security risks, network intrusion detection technology has become more important. At present, machine learning is applied in most advanced traffic anomaly detection algorithms, but these algorithms have three main shortcomings. First, algorithms using deep neural network are highly complex and not suitable for real-time online processing. Second, algorithms based on supervised learning require training on huge labeled data sets, which are limited and insufficient. Third, most algorithms have such poor generalization ability and portability that they are less suitable for real-world environments. Therefore, we propose a novel network anomaly detection model, STRAD. We use Word2vec and Damped Incremental Statistics algorithm for spatiotemporal features extraction, latent space compression (LSC) for feature vectors compression and an unsupervised one-class classifier for anomaly detection. Our evaluations show that STRAD has a better performance than other state of the art algorithms.

Machine learning (ML), especially deep learning (DL) techniques have been increasingly used in anomaly-based network intrusion detection systems (NIDS). However, ML/DL has shown to be extremely vulnerable to adversarial attacks, especially in such security-sensitive systems. Many adversarial attacks have been proposed to evaluate the robustness of ML-based NIDSs. Unfortunately, existing attacks mostly focused on feature-space and/or white-box attacks, which make impractical assumptions in real-world scenarios, leaving the study on practical gray/black-box attacks largely unexplored. To bridge this gap, we conduct the first systematic study of the gray/black-box traffic-space adversarial attacks to evaluate the robustness of ML-based NIDSs. Our work outperforms previous ones in the following aspects: (i) practical -the proposed attack can automatically mutate original traffic with extremely limited knowledge and affordable overhead while preserving its functionality; (ii) generic -the proposed attack is effective for evaluating the robustness of various NIDSs using diverse ML/DL models and non-payload-based features; (iii) explainable -we propose an explanation method for the fragile robustness of ML-based NIDSs. Based on this, we also propose a defense scheme against adversarial attacks to improve system robustness. We extensively evaluate the robustness of various NIDSs using diverse feature sets and ML/DL models. Experimental results show our attack is effective (e.g., >97% evasion rate in half cases for Kitsune, a state-of-the-art NIDS) with affordable execution cost and the proposed defense method can effectively mitigate such attacks (evasion rate is reduced by >50% in most cases).

In currently popular access control models, the security policies and regulations never change in the running system process once they are identified, which makes it possible for attackers to find the vulnerabilities in a system, resulting in the lack of ability to perceive the system security status and risks in a dynamic manner and exposing the system to such risks. By introducing the maximum entropy (MaxENT) models into the rule optimization for the Bell-LaPadula (BLP) model, this paper proposes an improved BLP model with the self-learning function: MaxENT-BLP. This model first formalizes the security properties, system states, transformational rules, and a constraint model based on the states transition of the MaxENT. After handling the historical system access logs as the original data sets, this model extracts the user requests, current states, and decisions to act as the feature vectors. Second, we use k -fold cross validation to divide all vectors into a training set and a testing set. In this paper, the model training process is based on the Broyden-Fletcher-Goldfarb-Shanno algorithm. And this model contains a strategy update algorithm to adjust the access control rules dynamically according to the access and decision records in a system. Third, we prove that MaxENT-BLP is secure through theoretical analysis. By estimating the precision, recall, and F1-score, the experiments show the availability and accuracy of this model. Finally, this paper provides the process of model training based on deep learning and discussions regarding adversarial samples from the malware classifiers. We demonstrate that MaxENT-BLP is an appropriate choice and has the ability to help running information systems to avoid more risks and losses.

Latent Dirichlet Allocation (LDA), has been recently used to automatically generate text corpora topics, and applied to sentences extraction based multi-document summarization algorithms. In this paper, we propose a novel approach to automatic generation of aspect-oriented summaries from multiple documents. Our approach is to combine the traditional summary generation algorithm and the the abstract generation algorithm based on deep learning.We employ the improved traditional summary generation algorithm to convert multiple documents into a single document, and then using the resulting single document with the deep learning method to extract the final summary. At first, we apply improved LDA model to cluster sentences in all documents. Second, We employ the extended LexRank algorithm to sort the sentences in each cluster. Third, we use extended Hedge Trimmer algorithm for sentence compression. Fourth, We apply Integer Linear Programming for sentence selection, and in this step ,we get the single document. Finally, We employ the textum on TensorFlow to get the final abstract. The experiments showed that the proposed algorithm achieved better performance compared the other state-of-the-art algorithms on DUC2005 and TAC2010 corpus.