Network anomaly detection provides a reliable and stable service to detect faults and prevent security attacks effectively. However, existing detection methods still encounter many challenges. The supervised learning method is unsuitable because the anomaly samples are seriously sparse and hard to label. Unsupervised learning, as a promising method, is widely used while the discriminative features are ignored when reconstructing from the normal feature space. This paper proposes a novel probability correlation learning based on autoencoder called PCDetect, a semi-supervised learning method. Since we assumed the anomaly samples deviate from the distribution of normal samples, approximating the distribution of original data is proposed as an efficient preprocessing methodology to capture the discriminative features. Moreover, an encoder-decoder neural network associated with the proposed loss function is designed to learn the low-dimensional feature representation from raw data and constrain the latent representation to follow different referenced distributions based on a few anomaly labels. In this way, The correlation of the referenced distribution and the reconstruction of latent representation will be used to quantify the probability of anomaly. Extensive experiments are conducted on two public real-world datasets, NSL-KDD and UNSW-NB15. Results show the proposed PCDetect can efficiently cope with the imbalance and high-dimensional issues compared with several popular supervised learning methods, significantly improve the accuracy, and reduce the false rate as a whole compared with unsupervised learning.

MOTIVATION: Low coverage sequencing provides an economic strategy for whole genome sequencing. When sequencing a set of individuals, genotype calling can be challenging due to low sequencing coverage. Linkage disequilibrium (LD) based refinement of genotyping calling is essential to improve the accuracy. Current LD-based methods use read counts or genotype likelihoods at individual potential polymorphic sites (PPSs). Reads that span multiple PPSs (jumping reads) can provide additional haplotype information overlooked by current methods. RESULTS: In this article, we introduce a new Hidden Markov Model (HMM)-based method that can take into account jumping reads information across adjacent PPSs and implement it in the HapSeq program. Our method extends the HMM in Thunder and explicitly models jumping reads information as emission probabilities conditional on the states of adjacent PPSs. Our simulation results show that, compared to Thunder, HapSeq reduces the genotyping error rate by 30%, from 0.86% to 0.60%. The results from the 1000 Genomes Project show that HapSeq reduces the genotyping error rate by 12 and 9%, from 2.24% and 2.76% to 1.97% and 2.50% for individuals with European and African ancestry, respectively. We expect our program can improve genotyping qualities of the large number of ongoing and planned whole genome sequencing projects. CONTACT: [email protected]; [email protected] AVAILABILITY: The software package HapSeq and its manual can be found and downloaded at www.ssg.uab.edu/hapseq/. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.