The Bayesian Cramér-Rao bound (CRB) provides a lower bound on the mean square error of any Bayesian estimator under mild regularity conditions. It can be used to benchmark the performance of statistical estimators, and provides a principled metric for system design and optimization. However, the Bayesian CRB depends on the underlying prior distribution, which is often unknown for many problems of interest. This work introduces a new data-driven estimator for the Bayesian CRB using score matching, i.e., a statistical estimation technique that models the gradient of a probability distribution from a given set of training data. The performance of the proposed estimator is analyzed in both the classical parametric modeling regime and the neural network modeling regime. In both settings, we develop novel non-asymptotic bounds on the score matching error and our Bayesian CRB estimator based on the results from empirical process theory, including classical bounds and recently introduced techniques for characterizing neural networks. We illustrate the performance of the proposed estimator with two application examples: a signal denoising problem and a dynamic phase offset estimation problem with applications in communication systems.

Score matching is a statistical approach for estimating the score (the gradient of the log-density) of a probability distribution from samples. It has found a number of applications, including in generative modeling, where it serves as a key component of the state-of-the-art diffusion modeling framework. The goal of this work is to provide non-asymptotic bounds on the score matching risk in the setting where the score model is a deep neural network. Here key challenges include the fact that the score model is vector-valued and that the score matching loss depends on the Jacobian of the score model. Our approach integrates results from empirical process theory, including classical bounds and recently introduced techniques for bounding covering numbers of neural network models, with novel covering results to address these challenges. The resulting bound has logarithmic dependence on the network width, allowing the network size to grow exponentially with the number of training samples without compromising the bound.

The Bayesian Cramér-Rao bound (CRB) provides a lower bound on the mean square error of any estimator in Bayesian inference under mild regularity conditions. It benchmarks the performance of statistical estimation and can also serve as a principled metric for system design and optimization. However, it is difficult to calculate the Bayesian CRB without explicit knowledge of the prior distribution. In this paper, we introduce a novel data-driven method for Bayesian CRB estimation, leveraging state-of-the-art score estimation and deep generative modeling techniques. We show that the proposed estimator is consistent and illustrate its performance in a denoising problem.