Robust continual learning (CL) poses fundamental challenges and is essential for developing reliable and adaptable intelligent systems. Learning models must sustain a robust performance as they adapt to dynamically evolving environments through sequential learning, effectively overcoming the catastrophic forgetting problem. This paper proposes a novel, trustworthy CL framework based on the Bayesian variational uncertainty learned during training on each task. We integrate the Bayesian inference and propagate the first two moments of the variational posterior distribution over the probabilistic model’s parameters. The variational moments (mean and covariance matrix) are learned simultaneously during training on each task and then used to estimate the predictive distribution. The covariance matrix of the variational posterior distribution captures the variational uncertainty in the learned parameters (particularly critical in the context of sequential learning within dynamic real-world settings). We develop an adaptive evidence lower bound (ELBO) loss function that supports managing the stability-elasticity dilemma. The variational continual optimization minimizes the expected log-likelihood of the data given the model’s parameters and the Kullback-Leibler (KL) divergence between the variational distributions learned from the current and previous tasks weighted by an uncertainty-based metric. Moreover, we advance an architecture-based CL technique that masks important network parameters learned from each task based on their variational uncertainty. The proposed Bayesian regularization and architecture-based CL approaches prevent significant changes in the parameters of the learning models to preserve representations of previous tasks. The experiments on benchmark datasets demonstrate the robustness of the proposed framework when learning in continual scenarios compared to the stat-of-the-art CL homologs.

The massive time-series production through the Internet of Things and digital healthcare requires novel data modeling and prediction. Recurrent neural networks (RNNs) are extensively used for analyzing time-series data. However, these models are unable to assess prediction uncertainty, which is particularly critical in heterogeneous and noisy environments. Bayesian inference allows reasoning about predictive uncertainty by estimating the posterior distribution of the parameters. The challenge remains in propagating the high-dimensional distribution through the sequential, non-linear layers of RNNs, resulting in mode collapse leading to erroneous uncertainty estimation and exacerbating the gradient explosion problem. This paper proposes a TRustworthy Uncertainty propagation for Sequential Time-series analysis (TRUST) in RNNs by introducing a Gaussian prior over network parameters and estimating the first two moments of the Gaussian variational distribution using the evidence lower bound. We propagate the variational moments through the sequential, non-linear layers of RNNs using the first-order Taylor approximation. The propagated covariance of the predictive distribution captures uncertainty in the output decision. The extensive experiments using ECG5000 and PeMS-SF classification and weather and power consumption prediction tasks demonstrate 1) significant robustness of TRUST-RNNs against noise and adversarial attacks and 2) self-assessment through the uncertainty that increases significantly with increasing noise.