This paper introduces a novel structure for deep Gaussian processes (DGPs) and a method for determining their depths. The proposed framework enables faster convergence of their parameters and reduces computational cost to optimize them while maintaining performance comparable to that of conventional DGP models. Furthermore, our approach presents a feasible solution to reduce the risk that the model becomes trapped in local minima during the simultaneous training of multiple layers, which ensures more efficient and reliable model training. Through experimental evaluation, we demonstrate the effectiveness of these models and the advantages of integrating transfer learning to reduce retraining costs.

Random feature latent variable models (RFLVMs) are state-of-the-art tools for uncovering structure in high-dimensional, non-Gaussian data. However, their reliance on Monte Carlo sampling significantly limits scalability, posing challenges for large-scale applications. To overcome these limitations, we develop a scalable RFLVM framework based on variational Bayesian inference (VBI), a deterministic and optimization-based alternative to sampling methods. Applying VBI to RFLVMs is nontrivial due to two key challenges: (i) the lack of an explicit probability density function (PDF) for Dirichlet process (DP) mixing weights, and (ii) the inefficiency of existing VBI approaches when handling the high-dimensional variational parameters of RFLVMs. To address these issues, we adopt the stick-breaking construction for the DP, which provides an explicit and tractable PDF over mixing weights, and propose a novel inference algorithm, block coordinate descent variational inference (BCD-VI), which partitions variational parameters into blocks and applies tailored solvers to optimize them efficiently. The resulting scalable model, referred to as SRFLVM, supports various likelihoods; we demonstrate its effectiveness under Gaussian and logistic settings. Extensive experiments on diverse benchmark datasets show that SRFLVM achieves superior scalability, computational efficiency, and performance in latent representation learning and missing data imputation, consistently outperforming state-of-the-art latent variable models, including deep generative approaches.

In many science and engineering problems, we observe high-dimensional data acquired sequentially. At each time instant, these data correspond to one of a predefined number of classes. The sequence of classes follows a certain pattern, with the transition probabilities of the classes being unknown. Our hypothesized generative model of the observed data involves two latent processes. The first is a root process representing the sequence of classes, while the second is a low-dimensional process generated as a Markovian process, depending on the current class and the previous value of the low-dimensional process. The observed high-dimensional process is generated from the low-dimensional state process. Our objective is to infer the posterior distributions of the classes as they evolve over time based on the observed data and the adopted model. To achieve this, we propose a method for estimating the latent processes. We demonstrate the effectiveness of our approach on synthesized data.

The inherent adaptability and flexibility of Gaussian processes lie in the capability of their kernel functions to capture diverse data characteristics. Thus, selecting an appropriate kernel function is crucial because an improper choice can detrimentally affect the model’s performance. One way to enhance the capability of Gaussian processes in capturing intricate data features is by stacking multiple Gaussian processes, thus constructing deep Gaussian processes. In this paper, we propose a novel structure for deep Gaussian processes that uses an ensemble of diverse kernels. We demonstrate that this structure leads to superior results compared to those of a single-kernel deep Gaussian processes.

In this article, we propose novel Gaussian process-gated hierarchical mixtures of experts (GPHMEs). Unlike other mixtures of experts with gating models linear in the input, our model employs gating functions built with Gaussian processes (GPs). These processes are based on random features that are non-linear functions of the inputs. Furthermore, the experts in our model are also constructed with GPs. The optimization of the GPHMEs is performed by variational inference. The proposed GPHMEs have several advantages. They outperform tree-based HME benchmarks that partition the data in the input space, and they achieve good performance with reduced complexity. Another advantage is the interpretability they provide for deep GPs, and more generally, for deep Bayesian neural networks. Our GPHMEs demonstrate excellent performance for large-scale data sets, even with quite modest sizes.

In science and engineering, we often deal with signals that are acquired from time-varying systems represented by dynamic graphs. We observe these signals, and the interest is in finding the time-varying topology of the graphs. We propose two Bayesian methods for estimating these topologies without assuming any specific functional relationships among the signals on the graphs. The two methods exploit Gaussian processes, where the first method uses the length scale of the kernel and relies on variational inference for optimization, and the second method is based on derivatives of the functions and Monte Carlo sampling. Both methods estimate the time-varying topologies of the graphs sequentially. We provide numerical tests that show the performance of the methods in two settings.

Determining the number of latent variables, or the dimensions of latent states, is a ubiquitous problem in dimension reduction. In this paper, we introduce a novel sequential method that relies on the Bayesian approach to estimate the dimension of a latent space of a Gaussian process latent variable model. The proposed method also considers settings where the number of latent variables varies with time. To evaluate our methodology, we compared the estimated dimensions with the true dimensions as they vary with time. Results on synthetic data demonstrate that our method has a very good performance.