Robust principal component analysis (PCA) is a key technique for dynamical high-dimensional data analysis, including background subtraction for surveillance video. Typically, robust PCA requires all observations to be stored in memory before processing. The batch manner makes robust PCA inefficient for big data. In this paper, we develop an efficient online robust PCA method, namely, online moving window robust principal component analysis (OMWRPCA). Unlike the existing algorithms, OMWRPCA can successfully track not only slowly changing subspaces but also abruptly changing subspaces. Embedding hypothesis testing into the algorithm enables OMWRPCA to detect change points of the underlying subspaces. Extensive numerical experiments, including real-time background subtraction, demonstrate the superior performance of OMWRPCA compared with other state-of-the-art approaches.

Support vector data description (SVDD) is a machine learning technique that is used for single-class classification and outlier detection. The idea of SVDD is to find a set of support vectors that defines a boundary around data. When dealing with online or large data, existing batch SVDD methods have to be rerun in each iteration. We propose an incremental learning algorithm for SVDD that uses the Gaussian kernel. This algorithm builds on the observation that all support vectors on the boundary have the same distance to the center of sphere in a higher-dimensional feature space as mapped by the Gaussian kernel function. Each iteration involves only the existing support vectors and the new data point. Moreover, the algorithm is based solely on matrix manipulations; the support vectors and their corresponding Lagrange multiplier αi’s are automatically selected and determined in each iteration. It can be seen that the complexity of our algorithm in each iteration is only O(k2), where k is the number of support vectors. Experimental results on some real data sets indicate that FISVDD demonstrates significant gains in efficiency with almost no loss in either outlier detection accuracy or objective function value.

Local outlier factor (LOF) is a state-of-the-art unsupervised machine learning model for anomaly detection in the Internet of Things (IoT). In recent years, there have been many important applications of LOF in large IoT systems, such as wireless transmission device system in high-speed trains, smart city vehicular social networks, industrial conditional monitoring systems, and network infrastructures for cyber security. LOF aims to forecast the imminent faults of an appliance in the IoT system, whose predictive performance greatly depends on the selection of its hyperparameters. The major challenge of hyperparameter tuning for unsupervised machine learning models including LOF in the context of IoT systems is that the incoming data may contain new types of anomaly that are unseen in the training data. In this paper, we propose a novel, heuristic methodology to tune the hyperparameters in LOF that accounts for this challenge. A tuned LOF model that uses the proposed method shows good predictive performance in both simulations and real data applications.

Support vector data description (SVDD) provides a useful approach, with various practical applications, for constructing a description of multivariate data for single-class classification and outlier detection. The Gaussian kernel that is used in SVDD formulation allows a flexible data description defined by observations that are designated as support vectors. The data boundary of such a description is nonspherical and conforms to the geometric features of the data. By varying the Gaussian kernel bandwidth parameter, the SVDD-generated boundary can be made either smoother (more spherical, which might lead to underfitting), or tighter and more jagged (which might result in overfitting). Kakde et al. [1] proposed a peak criterion for selecting an optimal value of the kernel bandwidth to strike a balance between the data boundary smoothness and a method's ability to capture the general geometric shape of the data. The peak criterion approach involves training the SVDD at various values of the kernel bandwidth parameter. When training data sets are large, the time required to obtain the optimal value of the Gaussian kernel bandwidth parameter according to the peak method can become prohibitively large. This paper proposes an extension of the peak method for the case of large data. The proposed method produces good results when applied to several data sets. Two existing alternative methods of computing the Gaussian kernel bandwidth parameter (coefficient of variation and distance to the farthest neighbor) were modified in order to allow comparison with the proposed method on convergence. Empirical comparison demonstrates the advantage of the proposed method.