Deep kernel learning (DKL) marries the uncertainty quantification of Gaussian processes (GPs) and the representational power of deep neural networks. However, training DKL is challenging and often leads to overfitting. Most notably, DKL often learns “non-local” kernels — incurring spurious correlations. To remedy this issue, we propose using amortized inducing points and a parameter-sharing scheme, which ties together the amortization and DKL networks. This design imposes an explicit dependency between the ELBO’s model fit and capacity terms. In turn, this prevents the former from dominating the optimization procedure and incurring the aforementioned spurious correlations. Extensive experiments show that our resulting method, amortized varitional DKL (AVDKL), i) consistently outperforms DKL and standard GPs for tabular data; ii) achieves significantly higher accuracy than DKL in node classification tasks; and iii) leads to substantially better accuracy and negative log-likelihood than DKL on CIFAR100.

In this paper, a new Fuzzy ART-based architecture called SBFA is introduced. The SBFA is composed of the modules ART and Target (T), and a third novel module called SC2that links these two modules. The new module SC2formulates the multiresponse classification problem as a linear system computed from the input data, the ART categories, and the targets labels in the module T. The module SC2then solves the linear system through the multiresponse Sparse Bayesian Learning (M-SBL) algorithm. The solution of such a linear system is the prediction field W. Since we use the M-SBL algorithm, the prediction field is sparse and, thus, the number of categories also decreases. From the achievements, our proposal is in general equivalent to state-of-art models and outperforms other ARTMAP-based models. Also, the SBFA successfully incorporates the M-SBL in the learning process.