Accurate uncertainty quantification is critical for robust and trustworthy predictions in many real-world applications. Bayesian Neural Networks (BNNs) provide a principled approach for modeling uncertainty but are often limited by the computational complexity of Bayesian inference. In this paper, we introduce a statistical linearization approach for multilayer feedforward BNNs. We demonstrate that this statistical linearization is equivalent to the Kalman Bayesian Neural Networks (KBNN) framework. This equivalence unifies these methodologies, providing a theoretical foundation for understanding the relationship between different BNN training approaches.

Bayesian machine learning models-especially Bayesian neural networks (BNNs)-offer powerful black-box approaches for prediction and uncertainty quantification. However, these models frequently exhibit inconsistent prediction quality across input regions, and conventional global metrics (e.g., the mean squared error (MSE)) are inadequate for capturing such local discrepancies. To overcome this limitation, we introduce a novel kernel-based framework for local calibration testing that assesses how well predicted distributions reflect both the function to be learned and inherent uncertainties. In our approach, spherical input-space kernels are used to define relevant subsets in the neighborhood of a point to be tested. This enables the online assessment of these localized regions using calibration metrics or statistical tests. By aggregating results across multiple kernel widths, our method yields both robust binary decisions and a continuous analysis over arbitrary inputs. Numerical experiments on single- and multi-dimensional regression tasks demonstrate the efficiency and scalability of our approach, underscoring its potential for real-time and large-scale applications.

Bayesian Neural Networks (BNNs) offer a sophisticated framework for extending classical neural network point estimates to encompass predictive distributions. Despite the high potential of BNNs, established BNN training methods such as Variational Inference (VI) and Markov Chain Monte Carlo (MCMC) grapple with issues such as scalability and hyperparameter dependence. In addressing these issues, our research focuses on the fundamental elements of BNNs, in particular perceptrons and their predictive capabilities. We introduce a new perspective on the closed-form solution for backward-pass computation for the Bayesian perceptron and prove that the state-of-the-art solution is equivalent to statistical linearization. To assess the efficacy of Bayesian perceptrons and provide insights into their performance in distinct input space regions, a novel methodology utilizing k-d trees as a space partitioning method is introduced to evaluate prediction quality within specific input space regions.