Many modern imaging applications can be modeled as compressed sensing linear inverse problems. When the measurement operator involved in the inverse problem is sufficiently random, denoising Scalable Message Passing (SMP) algorithms have a potential to demonstrate high efficiency in recovering compressed data. One of the key components enabling SMP to achieve fast convergence, stability and predictable dynamics is the Onsager correction that must be updated at each iteration of the algorithm. This correction involves the denoiser’s divergence that is traditionally estimated via the Black-Box Monte Carlo (BB-MC) method. While the BB-MC method demonstrates satisfying accuracy of estimation, it requires heuristic tuning and executing the denoiser additional times at each iteration and might lead to a substantial increase in computational cost of the SMP algorithms. In this work we develop two Large System Limit models of the Onsager correction for denoisers operating within SMP algorithms and use these models to propose practical black-box methods for divergence estimation that require no additional executions of the denoiser and demonstrate similar correction compared to the BB-MC method.

Vector Approximate Message Passing (VAMP) provides the means of solving a linear inverse problem in a Bayes-optimal way assuming the measurement operator is sufficiently random. However, VAMP requires implementing the linear minimum mean squared error (LMMSE) estimator at every iteration, which makes the algorithm intractable for large-scale problems. In this work, we present a class of warm-started (WS) methods that provides a scalable approximation of LMMSE within VAMP. We show that a Message Passing (MP) algorithm equipped with a method from this class can converge to the fixed point of VAMP while having a per-iteration computational complexity proportional to that of AMP. Additionally, we provide the Onsager correction and a multi-dimensional State Evolution for MP utilizing one of the WS methods. Lastly, we show that the approximation approach used in the recently proposed Memory AMP (MAMP) algorithm is a special case of the developed class of WS methods.

The Recently proposed Vector Approximate Message Passing (VAMP) algorithm demonstrates a great reconstruction potential at solving compressed sensing related linear inverse problems. VAMP provides high per-iteration improvement, can utilize powerful denoisers like BM3D, has rigorously defined dynamics and is able to recover signals measured by highly undersampled and ill-conditioned linear operators. Yet, its applicability is limited to relatively small problem sizes due to the necessity to compute the expensive LMMSE estimator at each iteration. In this work we consider the problem of upscaling VAMP by utilizing Conjugate Gradient (CG) to approximate the intractable LMMSE estimator. We propose a rigorous method for correcting and tuning CG withing CG-VAMP to achieve a stable and efficient reconstruction. To further improve the performance of CG-VAMP, we design a warm-starting scheme for CG and develop theoretical models for the Onsager correction and the State Evolution of Warm-Started CG-VAMP (WS-CG-VAMP). Additionally, we develop robust and accurate methods for implementing the WS-CG-VAMP algorithm. The numerical experiments on large-scale image reconstruction problems demonstrate that WS-CG-VAMP requires much fewer CG iterations compared to CG-VAMP to achieve the same or superior level of reconstruction.

In this paper we consider the problem of recovering a signal x of size N from noisy and compressed measurements y = Ax + w of size M, where the measurement matrix A is right-orthogonally invariant (ROI). Vector Approximate Message Passing (VAMP) demonstrates great reconstruction results for even highly ill-conditioned matrices A in relatively few iterations. However, performing each iteration is challenging due to either computational or memory point of view. On the other hand, a recently proposed Conjugate Gradient (CG) Expectation Propagation (CG-EP) framework is able to sacrifice some performance for efficiency, but requires access to exact singular spectrum of A. In this work we develop a CG-VAMP algorithm that does not require such information, is feasible to implement and converges to the neighborhood of the original VAMP.