As deep learning (DL) continues to advance, effective feature extraction from large-scale data remains crucial for enhancing model performance. To leverage the advantages of the frequency domain, such as concentrated signal energy, prominent data features, and rich detailed characteristics, this paper proposes a novel frequency-domain feature extraction method. However, existing frequency component selection algorithms often struggle to adapt to diverse tasks, tend to yield only locally optimal solutions, and require prolonged processing times. To overcome these limitations, we introduce the Adaptive Fast Frequency Selection (AFFS) algorithm, which seamlessly integrates a frequency component selection factor layer into DL models to identify globally optimal frequency combinations suited to various downstream tasks. We further analyze the relationship between selected frequency components and model performance, providing theoretical guarantees regarding optimality, robustness, and generalization error bounds. Moreover, a fast selection procedure is developed to exploit the empirically observed rapid convergence of the selection-factor ranking, significantly accelerating the selection process. Extensive experiments on five datasets, ten DL models, and two subsequent tasks demonstrate that AFFS achieves superior performance: even when the input data size is reduced to only 10% of the original frequency features, model classification accuracy improves by approximately 1%, while the early stopping mechanism shortens the selection process by about 80%.

Mobile crowd sensing (MCS) has become a new paradigm for environment sensing. However, the sensing data often face the challenge of missing values, which can impact the performance of subsequent tasks. Although some deep learning-based imputation methods perform well, they still struggle with insufficient training data due to high missing rate and long-term missing data. To address these challenges, we propose a Historical Data-based Diffusion Imputation (HDDI) method. Unlike existing deep learning-based imputation methods, we design a historical data supplement module to match and fuse historical data to supplement the training data. Additionally, we propose a diffusion imputation module that utilizes the supplement training data to achieve high-accuracy imputation even under high missing rate and long-term missing scenarios. We conduct extensive experiments on four public datasets, the results show that our HDDI outperforms baseline methods across four datasets. Particularly, when the data missing rate is 90%, HDDI improves accuracy by 25.15% compared to the best baseline method in the random missing scenario, and by 13.64% in the long-term missing scenario.

Deep neural networks (DNNs) have made great breakthroughs and seen applications in many domains. However, the incomparable accuracy of DNNs is achieved with the cost of considerable memory consumption and high computational complexity, which restricts their deployment on conventional desktops and portable devices. To address this issue, low-rank factorization, which decomposes the neural network parameters into smaller sized matrices or tensors, has emerged as a promising technique for network compression. In this article, we propose leveraging the emerging tensor ring (TR) factorization to compress the neural network. We investigate the impact of both parameter tensor reshaping and TR decomposition (TRD) on the total number of compressed parameters. To achieve the maximal parameter compression, we propose an algorithm based on prime factorization that simultaneously identifies the optimal tensor reshaping and TRD. In addition, we discover that different execution orders of the core tensors result in varying computational complexities. To identify the optimal execution order, we construct a novel tree structure. Based on this structure, we propose a top-to-bottom splitting algorithm to schedule the execution of core tensors, thereby minimizing computational complexity. We have performed extensive experiments using three kinds of neural networks with three different datasets. The experimental results demonstrate that, compared with the three state-of-the-art algorithms for low-rank factorization, our algorithm can achieve better performance with much lower memory consumption and lower computational complexity.

Accurately detecting traffic anomalies becomes increasingly crucial in network management. Algorithms that model the traffic data as a matrix suffers from low detection accuracy, while the work using the tensor model often assumes the tensor is regular without considering that network nodes may dynamically join in or leave, which will fail in a practical network with the change of node set as a result of mobility and churn behaviors. We propose a novel Tensor Recovery scheme in a Dynamic Network (TRDN) with traffic data modeled as a practical irregular tensor for accurate anomaly detection. To take advantage of correlations among small tensors, each formed with a short time duration to capture more hidden information in the data for higher detection accuracy, we propose several novel techniques: 1) a new joint tensor factorization model to capture the characteristic shared by the common nodes of small tensors, 2) a tensor partition algorithm to identify the data that can be applied to train the shared parameters efficiently, and 3) a bar-based algorithm that partitions nodes into the minimum number of no-overlapping subsets to form the shared tensor model. Extensive experiments on two Internet traffic data sets, Abilene and GÈANT, demonstrate the effectiveness of the proposed TRDN.

Robust PCA is a popular anomaly detection technique and has been widely used in many applications. Although Robust PCA is promising, it is usually designed in a two-order matrix form, which is inferior to the tensor that can capture multilinearity features of data. Moreover, the detection accuracy under Robust PCA further suffers due to its sensitivity to the rank parameter which is hard to set in practice and the limitation of PCA method in capturing the non-linear feature in the data. To address the issues, we propose a Robust Tensor Convolutional Autoencoder (RTCAE) where the autoencoder instead of SVD is exploited to recover the normal data from the corrupted measurement tensor data. However, directly exploiting deep autoencoder may suffer from the problem of high memory consumption and computation overhead due to the large number of parameters used in autoencoder. To make our anomaly detection lightweight, we further design a Light Convolutional Autoencoder (LightCAE) which contains a compressed autoencoder by exploiting tensor factorization to largely compress the parameters while significantly reducing the computation complexity. We conduct extensive experiments on three real data traces to compare the performance of our proposed schemes (RTCAE and lightCAE) with that of seven baseline algorithms. The experiment results demonstrate that our proposed RTCAE achieves the highest anomaly detection accuracy. Moreover, our LightCAE requires over 60 times smaller memory storage than that required in RTCAE while achieving the similar anomaly detection accuracy.

In network distance measurement, how to estimate the whole network distance data from partially observed samples has attracted lots of attention because of its significance for network performance evaluation. Matrix completion becomes the most effective approach. However, the two-dimension matrix can only capture the spatial features in the network distance data while ignoring the temporal features. To conquer the problem, few recent studies begin to model the network distance data as a three-dimension tensor and propose tensor completion approaches for distance estimation. Although promising, existing tensor completion approaches still suffer the problem of low recovery accuracy and high measurement cost because they ignore the history priority information. To fully utilize both spatial and temporal features hidden in the distance data, this paper formulates a novel History Priority Enhanced Tensor Completion (HPETC) for distance estimation as a weighted tensor nuclear norm minimization problem where the weight is defined based on the history subspaces information. To solve the weighted tensor nuclear norm minimization problem, we firstly transform it into a factorization-based Frobenius norm minimization problem to avoid costly T-SVD computations, and then propose an iterative algorithm to solve the transformed problem. We further derive a theoretical sampling bound that is lower than the existing sampling bound, thus leads a lower measurement cost. We demonstrate the effectiveness of the proposed algorithm by conducting extensive experiments using two real network distance datasets. The result shows that the proposed algorithm can not only improve the estimation accuracy but also reduce the sampling complexity compared to the state-of-the-art approaches.

Detecting anomalous traffic is a crucial task for network management. Although many anomaly detection algorithms have been proposed recently, constrained by their matrix-based traffic data model, existing algorithms often suffer from low detection accuracy. To fully utilize the multi-dimensional information hidden in the traffic data, this paper uses the tensor model for more accurate Internet anomaly detection. Only considering the low-rank linearity features hidden in the data, current tensor factorization techniques would result in low anomaly detection accuracy. We propose a novel Graph-based Tensor Recovery model (Graph-TR) to well explore both low-rank linearity features as well as the non-linear proximity information hidden in the traffic data for better anomaly detection. We encode the non-linear proximity information of the traffic data by constructing nearest neighbor graphs and incorporate this information into the tensor factorization using the graph Laplacian. Moreover, to facilitate the quick building of neighbor graph, we propose a nearest neighbor searching algorithm with the simple locality-sensitive hashing (LSH). Besides only detecting random anomalies, our algorithm can also effectively detect structured anomalies that appear as bursts. We have conducted extensive experiments using Internet traffic trace data Abilene and GÈANT. Compared with the state of art algorithms on matrix-based anomaly detection and tensor recovery approach, our Graph-TR can achieve higher Accuracy and Recall.

Network measurements provide critical inputs for a wide range of network management. Existing network-wide monitoring methods face the challenge of incurring a high measurement cost. Some recent studies show that network-wide measurement data such as end-to-end latency and flow traffic, have hidden spatio-temporal correlations and thus low-rank features. Taking advantage of the low-rank feature, enlightened by tensor model's strong capability of information representation and extracting, this paper studies a novel sparse measurement scheduling problem which selects a proportion of Origin and Destination (OD) pairs to take measurements in the future time slots, while ensuring the data of the remaining un-measured OD pairs be accurately inferred through tensor completion. It is challenging to find the optimal sampling points (OD pairs) without knowing the structure of the future data and also infer the un-measured data in the presence of noise in the measurement samples. To conquer the challenges, we propose several techniques: a tripartite graph to illustrate the relationship between sample locations and tensor factorization, a graph-based sample selection algorithm, and a graph-based robust tensor completion algorithm. We have conducted extensive experiments based on two real network latency monitoring traces (PlanetLab and Harvard) and two other network monitoring traces (including a traffic trace Abilene and a throughput trace WS-Dream). Our results demonstrate that, even with a sampling ratio of less than 5%, our scheme can accurately obtain the complete network-wide monitoring data by inferring the missing ones based on the samples taken. To achieve similar recovery performance, the best peer tensor completion algorithm needs a significantly larger number of samples, with the sampling ratio up to 25-150 times ours.

Network-wide monitoring is important for many network functions. However, monitoring data are often incomplete due to the need of sampling to reduce high measurement cost, system failure, and unavoidable transmission loss under severe communication. Instead of only targeting to estimate all missing monitoring data entries with a small set of measurement samples, we study a new order-preserved monitoring data estimation problem to accurately estimate the missing data entries while preserving the data entries’ order in the dataset. We propose a novel order-preserved tensor completion model that integrates both the low rank property and the order information into a joint learning problem to estimate the missing data. With well designed non-convex function to directly approximate the tensor rank and order-preserved constraint under the linear self-recovery method, our model can not only more accurately capture the low-rank property of monitoring data to increase the estimation performance of missing data, but also can capture the order information in monitoring data to ensure the estimation accuracy. Extensive experiments using four real datasets demonstrate that compared with the state-of-the-art tensor completion algorithms, our proposed algorithm can provide more accurate estimation and keep the value order of recovered entries to more effectively retrieve top-k large entries.

Mobile crowdsensing (MCS) has become a new paradigm for the environment sensing. However, the sparse sensory data prevent the practical and large-scale deployment of MCS systems. Recent studies have demonstrated that the matrix factorization is an effective technique which can estimate the missing sensory data entries based on a small set of observed data entries. However, there could be multiple sensory data sets with each regarded as a different view on the environment. Applying current matrix factorization individually to each data set, the recovery performance will be low as some data sets do not have enough observed data entries thus enough information. By partitioning the parameters involved in matrix factorization, we design some novel regularizations to encode the similarities among different data sets and specific knowledge in the single data set. Based on the regularizations, we propose one basic multiview matrix factorization (MVMF) model and one neural MVMF (NMVMF) model to combine multiple sensory data sets to mutually reinforce the estimation of each single data set. The extensive experimental results demonstrate that, with the help of other data sets, our models can estimate the missing entries in the data set with a very low sampling ratio accurately while the other five baseline algorithms cannot.

The attacks, faults, and severe communication/system conditions in Mobile Crowd Sensing (MCS) make false data detection a critical problem. Observing the intrinsic low dimensionality of general monitoring data and the sparsity of false data, false data detection can be performed based on the separation of normal data and anomalies. Although the existing separation algorithm based on Direct Robust Matrix Factorization (DRMF) is proven to be effective, requiring iteratively performing Singular Value Decomposition (SVD) for low-rank matrix approximation would result in a prohibitively high accumulated computation cost when the data matrix is large. In this work, we observe the quick false data location feature from our empirical study of DRMF, based on which we propose an intelligent Light weight Low Rank and False Matrix Separation algorithm (LightLRFMS) that can reuse the previous result of the matrix decomposition to deduce the one for the current iteration step. Depending on the type of data corruption, random or successive/mass, we design two versions of LightLRFMS. From a theoretical perspective, we validate that LightLRFMS only requires one round of SVD computation and thus has very low computation cost. We have done extensive experiments using a PM 2.5 air condition trace and a road traffic trace. Our results demonstrate that LightLRFMS can achieve very good false data detection performance with the same highest detection accuracy as DRMF but with up to 20 times faster speed thanks to its lower computation cost.

Traffic anomaly detection is critical for advanced Internet management. Existing detection algorithms usually work off-line and cannot timely detect anomalies. They also suffer from high cost for storage and computation. Although online and accurate traffic anomaly detection is very important, it very difficult to achieve. We propose to utilize tensor model to well exploit the multi-dimensional information hidden in the traffic data for more accurate online Internet anomaly detection. We decouple the tensor recovery problem to iteratively solve two sub problems, a tensor factorization sub-problem and an anomaly detection sub-problem. To reduce the high cost for computation and storage involved in tensor factorization, we propose two lightweight techniques to effectively derive factor matrices of tensor in the current window and iteration, taking advantage of tensor decomposition results of the previous window and iteration. We have done extensive experiments using two real traffic traces to compare with three tensor based algorithms and three matrix based algorithms. The experiment results demonstrate that our online anomaly detection algorithm can achieve the same anomaly detection accuracy as that of the best offline tensor based algorithm, but at 6100 times faster speed and with very low storage cost.

With the proliferation of smartphones, a novel sensing paradigm called Mobile Crowd Sensing (MCS) has emerged very recently. However, the attacks and faults in MCS cause a serious false data problem. Observing the intrinsic low dimensionality of general monitoring data and the sparsity of false data, false data detection can be performed based on the separation of normal data and anomalies. Although the existing separation algorithm based on Direct Robust Matrix Factorization (DRMF) is proven to be effective, requiring iteratively performing Singular Value Decomposition (SVD) for low-rank matrix approximation would result in a prohibitively high accumulated computation cost when the data matrix is large. In this work, we observe the quick false data location feature from our empirical study of DRMF, based on which we propose an intelligent Light weight Low Rank and False Matrix Separation algorithm (LightLRFMS) that can reuse the previous result of the matrix decomposition to deduce the one for the current iteration step. Our algorithm can largely speed up the whole iteration process. From a theoretical perspective, we validate that LightLRFMS only requires one round of SVD computation and thus has very low computation cost. We have done extensive experiments using a PM 2.5 air condition trace and a road traffic trace. Our results demonstrate that LightLRFMS can achieve very good false data detection performance with the same highest detection accuracy as DRMF but with up to 10 times faster speed thanks to its lower computation cost.

A major factor that prevents the large scale deployment of Mobile Crowd Sensing (MCS) is its sensing and communication cost. Given the spatio-temporal correlation among the environment monitoring data, matrix completion (MC) can be exploited to only monitor a small part of locations and time, and infer the remaining data. Rather than only taking random measurements following the basic MC theory, to further reduce the cost of MCS while ensuring the quality of missing data inference, we propose an Active Sparse MCS (AS-MCS) scheme which includes a bipartite-graph-based sensing scheduling scheme to actively determine the sampling positions in each upcoming time slot, and a bipartite-graph-based matrix completion algorithm to robustly and accurately recover the un-sampled data in the presence of sensing and communications errors. We also incorporate the sensing cost into the bipartite-graph to facilitate low cost sample selection and consider the incentives for MCS. We have conducted extensive performance studies using the data sets from the monitoring of PM 2.5 air condition and road traffic speed, respectively. Our results demonstrate that our AS-MCS scheme can recover the missing data at very high accuracy with the sampling ratio only around $11%$, while the peer matrix completion algorithms with similar recovery performance requires up to 4-9 times the number of samples of ours for both the data sets.

A prime question for plasma physicists is how a fraction of charged particles is accelerated to very high energy.To answer this question, physicists simulate trillions of particles with detailed dynamics and analyze their trajectories. This process requires a range of data analysis tasks with high diversity. In this paper, we present a use case of formulating various analysis tasks on terabyte-scale particle data with a novel data analysis framework called ArrayUDF. The flexibility of ArrayUDF allows it to compose a wide range of particle data operations. We also present optimization strategies to avoid frequent global reduction and to take full advantage of the data locality. Tests show that our optimization methods could accelerate these particle data analysis operations by up to 1,600 times.

Detecting anomalous traffic is a crucial task of managing networks. Many anomaly detection algorithms have been proposed recently. However, constrained by their matrix-based traffic data model, existing algorithms often suffer from low detection accuracy. To fully utilize the multi-dimensional information hidden in the traffic data, this paper takes an initiative to investigate the potential and methodologies of performing tensor factorization for more accurate Internet anomaly detection. Only considering the low-rank linearity features hidden in the data, current tensor factorization techniques would result in low anomaly detection accuracy. We propose a novel Graph-based Tensor Recovery model (Graph-TR) to well explore both low rank linearity features as well as the non-linear proximity information hidden in the traffic data for better anomaly detection. We encode the non-linear proximity information of the traffic data by constructing nearest neighbor graphs and incorporate this information into the tensor factorization using the graph Laplacian. Moreover, to facilitate the quick building of neighbor graph, we propose a nearest neighbor searching algorithm with the simple locality-sensitive hashing (LSH). We have conducted extensive experiments using Internet traffic trace data Abilene and GEANT. Compared with the state of art algorithms on matrix-based anomaly detection and tensor recovery approach, our Graph-Trcan achieve significantly lower False Positive Rate and higher True Positive Rate.

Detecting anomalous traffic is a critical task for advanced Internet management. Many anomaly detection algorithms have been proposed recently. However, constrained by their matrix-based traffic data model, existing algorithms often suffer from low accuracy in anomaly detection. To fully utilize the multi-dimensional information hidden in the traffic data, this paper takes the initiative to investigate the potential and methodologies of performing tensor factorization for more accurate Internet anomaly detection. More specifically, we model the traffic data as a three-way tensor and formulate the anomaly detection problem as a robust tensor recovery problem with the constraints on the rank of the tensor and the cardinality of the anomaly set. These constraints, however, make the problem extremely hard to solve. Rather than resorting to the convex relaxation at the cost of low detection performance, we propose TensorDet to solve the problem directly and efficiently. To improve the anomaly detection accuracy and tensor factorization speed, TensorDet exploits the factorization structure with two novel techniques, sequential tensor truncation and two-phase anomaly detection. We have conducted extensive experiments using Internet traffic trace data Abilene and GÈANT. Compared with the state of art algorithms for tensor recovery and matrix-based anomaly detection, TensorDet can achieve significantly lower false positive rate and higher true positive rate. Particularly, benefiting from our well designed algorithm to reduce the computation cost of tensor factorization, the tensor factorization process in TensorDet is 5 (Abilene) and 13 (GÈANT) times faster than that of the traditional Tucker decomposition solution.