Anomaly detection plays a crucial role in various real-world applications, including healthcare and finance systems. Owing to the limited number of anomaly labels in these complex systems, unsupervised anomaly detection methods have attracted great attention in recent years. Two major challenges faced by the existing unsupervised methods are as follows: 1) distinguishing between normal and abnormal data when they are highly mixed together and 2) defining an effective metric to maximize the gap between normal and abnormal data in a hypothesis space, which is built by a representation learner. To that end, this work proposes a novel scoring network with a score-guided regularization to learn and enlarge the anomaly score disparities between normal and abnormal data, enhancing the capability of anomaly detection. With such score-guided strategy, the representation learner can gradually learn more informative representation during the model training stage, especially for the samples in the transition field. Moreover, the scoring network can be incorporated into most of the deep unsupervised representation learning (URL)-based anomaly detection models and enhances them as a plug-in component. We next integrate the scoring network into an autoencoder (AE) and four state-of-the-art models to demonstrate the effectiveness and transferability of the design. These score-guided models are collectively called SG-Models. Extensive experiments on both synthetic and real-world datasets confirm the state-of-the-art performance of SG-Models.

Anomalies are ubiquitous in real-world time-series data which call for effective and timely detection, especially in an unsupervised setting for labeling cost saving. In this paper, we develop an unsupervised density reconstruction model for multi-dimensional time-series anomaly detection. In particular, it directly handles an important realistic setting that the detection is achieved towards raw time-series contaminated with noise for training, in contrast to most existing anomaly detection works that assume the training data is in general clean i.e. not contaminated with anomaly. It extends recent advancements in deep generative models and state space models to achieve robust anomaly detection. Our approach comprises of a novel state space based generative model, a filtering based inference model, together with a carefully-designated emission model based on robust statistics theory. Extensive experimental results are conducted to show that our approach can adapt to complex patterns even given severely contaminated training data. We also develop visualization techniques to help better understand the behavior of the anomaly detection models. Empirical results show that our method outperforms state-of-the-arts on both synthetic and real-world datasets.

Time-series and event sequences are widely collected data types in real-world applications. Modeling and forecasting of such temporal data play an important role in an informed decision-making process. A major limitation of previous methods is that they either focus on time-series or events, rather than the combination of the two worlds. In fact, the two types of data often provide complementary information, emphasizing the necessity of jointly modeling the both. In this paper, we propose the RNN-ODE collaborative model for joint modeling and forecasting of heterogeneous time-series and event sequence data, which combines several useful techniques from both Bayesian and deep learning for its interpretability. Specifically, we devise a tailored encoder to combine the advances in deep temporal point processes models and variational recurrent neural networks. To predict the probability of event occurrence over an arbitrary continuous-time horizon, we base our model on the mathematical foundation of Neural Ordinary Differential Equations (NODE). Extensive experimental results on simulations and real data sets show that compared with existing methods, our integrated approach can achieve more competitive forecasting performance of both time-series and event sequences.

Time-series is ubiquitous across applications, such as transportation, finance and healthcare. Time-series is often influenced by external factors, especially in the form of asynchronous events, making forecasting difficult. However, existing models are mainly designated for either synchronous time-series or asynchronous event sequence, and can hardly provide a synthetic way to capture the relation between them. We propose Variational Synergetic Multi-Horizon Network (VSMHN), a novel deep conditional generative model. To learn complex correlations across heterogeneous sequences, a tailored encoder is devised to combine the advances in deep point processes models and variational recurrent neural networks. In addition, an aligned time coding and an auxiliary transition scheme are carefully devised for batched training on unaligned sequences. Our model can be trained effectively using stochastic variational inference and generates probabilistic predictions with Monte-Carlo simulation. Furthermore, our model produces accurate, sharp and more realistic probabilistic forecasts. We also show that modeling asynchronous event sequences is crucial for multi-horizon time-series forecasting.

Learning the distribution of future trajectories conditioned on the past is a crucial problem for understanding multi-agent systems. This is challenging because humans make decisions based on complex social relations and personal intents, resulting in highly complex uncertainties over trajectories. To address this problem, we propose a conditional deep generative model that combines advances in graph neural networks. The prior and recognition model encodes two types of latent codes for each agent: an inter-agent latent code to represent social relations and an intra-agent latent code to represent agent intentions. The decoder is carefully devised to leverage the codes in a disentangled way to predict multi-modal future trajectory distribution. Specifically, a graph attention network built upon inter-agent latent code is used to learn continuous pair-wise relations, and an agent's motion is controlled by its latent intents and its observations of all other agents. Through experiments on both synthetic and real-world datasets, we show that our model outperforms previous work in multiple performance metrics. We also show that our model generates realistic multi-modal trajectories.

Deep generative models have demonstrated their effectiveness in learning latent representation and modeling complex dependencies of time series. In this article, we present a smoothness-inducing sequential variational auto-encoder (VAE) (SISVAE) model for the robust estimation and anomaly detection of multidimensional time series. Our model is based on VAE, and its backbone is fulfilled by a recurrent neural network to capture latent temporal structures of time series for both the generative model and the inference model. Specifically, our model parameterizes mean and variance for each time-stamp with flexible neural networks, resulting in a nonstationary model that can work without the assumption of constant noise as commonly made by existing Markov models. However, such flexibility may cause the model fragile to anomalies. To achieve robust density estimation which can also benefit detection tasks, we propose a smoothness-inducing prior over possible estimations. The proposed prior works as a regularizer that places penalty at nonsmooth reconstructions. Our model is learned efficiently with a novel stochastic gradient variational Bayes estimator. In particular, we study two decision criteria for anomaly detection: reconstruction probability and reconstruction error. We show the effectiveness of our model on both synthetic data sets and public real-world benchmarks.

Probabilistic time series forecasting involves estimating the distribution of future based on its history, which is essential for risk management in downstream decision-making. We propose a deep state space model for probabilistic time series forecasting whereby the non-linear emission model and transition model are parameterized by networks and the dependency is modeled by recurrent neural nets. We take the automatic relevance determination (ARD) view and devise a network to exploit the exogenous variables in addition to time series. In particular, our ARD network can incorporate the uncertainty of the exogenous variables and eventually helps identify useful exogenous variables and suppress those irrelevant for forecasting. The distribution of multi-step ahead forecasts are approximated by Monte Carlo simulation. We show in experiments that our model produces accurate and sharp probabilistic forecasts. The estimated uncertainty of our forecasting also realistically increases over time, in a spontaneous manner.