Robust continual learning (CL) poses fundamental challenges and is essential for developing reliable and adaptable intelligent systems. Learning models must sustain a robust performance as they adapt to dynamically evolving environments through sequential learning, effectively overcoming the catastrophic forgetting problem. This paper proposes a novel, trustworthy CL framework based on the Bayesian variational uncertainty learned during training on each task. We integrate the Bayesian inference and propagate the first two moments of the variational posterior distribution over the probabilistic model’s parameters. The variational moments (mean and covariance matrix) are learned simultaneously during training on each task and then used to estimate the predictive distribution. The covariance matrix of the variational posterior distribution captures the variational uncertainty in the learned parameters (particularly critical in the context of sequential learning within dynamic real-world settings). We develop an adaptive evidence lower bound (ELBO) loss function that supports managing the stability-elasticity dilemma. The variational continual optimization minimizes the expected log-likelihood of the data given the model’s parameters and the Kullback-Leibler (KL) divergence between the variational distributions learned from the current and previous tasks weighted by an uncertainty-based metric. Moreover, we advance an architecture-based CL technique that masks important network parameters learned from each task based on their variational uncertainty. The proposed Bayesian regularization and architecture-based CL approaches prevent significant changes in the parameters of the learning models to preserve representations of previous tasks. The experiments on benchmark datasets demonstrate the robustness of the proposed framework when learning in continual scenarios compared to the stat-of-the-art CL homologs.

The massive time-series production through the Internet of Things and digital healthcare requires novel data modeling and prediction. Recurrent neural networks (RNNs) are extensively used for analyzing time-series data. However, these models are unable to assess prediction uncertainty, which is particularly critical in heterogeneous and noisy environments. Bayesian inference allows reasoning about predictive uncertainty by estimating the posterior distribution of the parameters. The challenge remains in propagating the high-dimensional distribution through the sequential, non-linear layers of RNNs, resulting in mode collapse leading to erroneous uncertainty estimation and exacerbating the gradient explosion problem. This paper proposes a TRustworthy Uncertainty propagation for Sequential Time-series analysis (TRUST) in RNNs by introducing a Gaussian prior over network parameters and estimating the first two moments of the Gaussian variational distribution using the evidence lower bound. We propagate the variational moments through the sequential, non-linear layers of RNNs using the first-order Taylor approximation. The propagated covariance of the predictive distribution captures uncertainty in the output decision. The extensive experiments using ECG5000 and PeMS-SF classification and weather and power consumption prediction tasks demonstrate 1) significant robustness of TRUST-RNNs against noise and adversarial attacks and 2) self-assessment through the uncertainty that increases significantly with increasing noise.

Data plenitude is the bottleneck for data-driven approaches, including neural networks. In particular, Convolutional Neural Networks (CNNs) require an abundant database of training images to achieve a desired high accuracy. Current techniques employed for boosting small datasets are data augmentation and synthetic data generation, which suffer from computational complexity and imprecision compared to original datasets. In this paper, we intercalate prior knowledge based on spatial relation between images in the third dimension by computing the gradient of subsequent images in the dataset to remove extraneous information and highlight subtle variations between images. The approach is coined “Inverted Cone” because the volume of brain images below the level of the eyes is ordered to form an inverted cone geometry. The application explored in this work is deboning, or brain extraction, in brain magnetic resonance imaging (MRI) scans. The difficulty of obtaining ground truth for this application prevents the ability of obtaining a large quantity of training images to train the CNN. We considered a limited dataset of 23 patients with and without malignant glioblastoma. Deboning was performed by employing an optimized CNN architecture with and without the Inverted Cone processing. The classic CNN without prior knowledge achieved a validation accuracy of 77%, while the Inverted Cone CNN model achieved a validation accuracy of 86% in a dataset of 451 brain MRI slices.

We propose a new non-parametric level set model for automatic image clustering and segmentation based on non-negative matrix factorization (NMF). We show that NMF: (i) clusters the image into distinct homogeneous regions and (ii) provides the local spatial distribution of each region within the image. Furthermore, NMF has a controllable resolution and can discover homogeneous regions as small as one pixel. Coupled with the level-set approach, NMF is an efficient method for image segmentation. The proposed model is unsupervised and relies on local histogram modeling to define an energy functional, whose optimization leads to the final segmentation. A unique and desirable feature of the proposed method is that it does not incorporate any spurious model parameters; hence, the optimization is performed only w.r.t level set functions. We apply the proposed Non-parametrIc Unsupervised SegmentatioN approach (geNIUS) to synthetic and real images and compare it to three state-of-the-art parametric and non-parametric level set approaches: the localized Gaussian distribution fitting model (LGDF) [1], the local histogram fitting (LHF) model [2], and our recent work: NMF-LSM in [3]. The proposed geNIUS model results in a superior accuracy and more efficient implementation, which is a result of its free-model parameter feature.

This paper presents a new level set method for image segmentation by integrating the level set formulation and the non-negative matrix factorization (NMF). The proposed model characterizes the histogram of the image by dividing the image into blocks and computing the histograms of the blocks as nonnegative combinations of basic histograms. This is achieved by using the NMF algorithm. The basic histograms form a clustering of the image into distinct regions. Our model also takes into account the intensity inhomogeneity or the bias field that usually corrupts medical images. In a level set formulation, this clustering criterion defines an energy in terms of the level set functions that represent a partition of the image domain. The image segmentation is achieved by minimizing this energy with respect to the level set functions and the bias field. Our method is compared, using synthetic and real images, to other state-of-the-art level set approaches that are based on localized clustering and local Gaussian distribution fitting. It is shown that the proposed approach is more robust to noise in the image and intensity inhomogeneity. These advantages stem from the fact that the proposed model i) depends on the distribution of pixels intensities (the histogram) rather than the direct intensity values and ii) does not introduce additional model parameters to be simultaneously estimated with the bias field and the level set functions.