Theoretical understanding of Plug-and-Play (PnP) algorithms, where an off-the-shelf denoiser is used for image regularization , is an active research topic. In this work, we study the problems of exact and stable signal recovery from compressively sensed (CS) measurements using PnP algorithms. We focus on a class of linear denoisers for which it is possible to associate a convex regularizer Φ . We consider the CS problem of minimizing Φ ⁡ ( 𝒙 ) subject to 𝐀 ⁢ 𝒙 = 𝐀 ⁢ 𝛏 , where 𝐀 is the random sensing matrix and 𝛏 is the ground truth. We prove that if 𝐀 is Gaussian and 𝛏 lies in the range of the associated denoiser 𝐖 , then the minimizer is almost surely 𝛏 if 𝑟 𝑎 𝑛 𝑘 ⁡ ( 𝐖 ) is less than the number of measurements and almost never otherwise. We extend the result to subgaussian matrices, except that we can guarantee exact recovery only with high probability. For noisy measurements, we consider a robust analogue of the recovery problem and prove that the error between the recovered and the ground-truth signal is bounded by the noise strength . In particular, we derive the sample complexity of CS as a function of reconstruction error and success rate. We perform numerical experiments to validate our theoretical findings.

We consider the problem of signal reconstruction from linearly corrupted data using plug-and-play (PnP) regularization. As opposed to traditional sparsity-promoting regularizers, PnP uses an off-the-shelf denoiser within a proximal algorithm such as ISTA or ADMM for image reconstruction. Although PnP has become popular in the imaging community, its regularization capacity is not fully understood. For example, it is not known if PnP can in theory recover a signal from few noiseless measurements as in classical compressed sensing and if the recovery is robust. We explore these questions in this work and present some theoretical and experimental results. In particular, we prove that if the denoiser in question has low rank and if the ground- truth lies in the range of the denoiser, then it can be recovered exactly from noiseless measurements. To the best of knowledge, this is first such result. Furthermore, we show using numerical simulations that even if the aforementioned conditions are violated, PnP recovery is robust in practice. We formulate a theorem regarding the recovery error based on these observations.

Multiband fusion involves combining an image having high spatial and low spectral resolution with another image having low spatial and high spectral resolution—resulting in a single multiband image with high spatial and spectral resolutions. In classical variational techniques, this problem is formulated as the minimization of an objective function consisting of two quadratic data-fidelity terms and an edge-preserving regularizer; the former account for blur, resolution mismatch and additive noise. In this work, we explore a constrained formulation of this problem where the regularization function is minimized subject to hard constraints on the data fidelity. Unlike the penalty approach, the advantage is that the user has direct control on the data fidelity of the reconstruction. We come up with an efficient ADMM solver for this constrained optimization problem. Moreover, for convex regularizers, we prove that that the ADMM iterates converge to an optimal solution (this is somewhat standard but requires the verification of certain technical conditions). To our knowledge, the use of constrained optimization for image fusion is novel. The proposed framework is shown to model the observations well and its fusion quality is competitive with state-of-the-art methods.

In plug-and-play (PnP) regularization, the knowledge of the forward model is combined with a powerful denoiser to obtain state-of-the-art image reconstructions. This is typically done by taking a proximal algorithm such as FISTA or ADMM, and formally replacing the proximal map associated with a regularizer by nonlocal means, BM3D or a CNN denoiser. Each iterate of the resulting PnP algorithm involves some kind of inversion of the forward model followed by denoiser-induced regularization. A natural question in this regard is that of optimality, namely, do the PnP iterations minimize some f+g , where f is a loss function associated with the forward model and g is a regularizer? This has a straightforward solution if the denoiser can be expressed as a proximal map, as was shown to be the case for a class of linear symmetric denoisers. However, this result excludes kernel denoisers such as nonlocal means that are inherently non-symmetric. In this paper, we prove that a broader class of linear denoisers (including symmetric denoisers and kernel denoisers) can be expressed as a proximal map of some convex regularizer g . An algorithmic implication of this result for non-symmetric denoisers is that it necessitates appropriate modifications in the PnP updates to ensure convergence to a minimum of f+g . Apart from the convergence guarantee, the modified PnP algorithms are shown to produce good restorations.

We propose a fast algorithm for an adaptive variant of the classical bilateral filter, where the range kernel is allowed to vary from pixel to pixel. Several fast and accurate algorithms have been proposed for bilateral filtering, but they assume that the same range kernel is used at each pixel and hence cannot be used for adaptive bilateral filtering (ABF). Only recently, it was shown that fast algorithms for ABF can be developed by approximating the local histogram around each pixel using polynomials. The present algorithm is derived using an entirely different approximation, namely, the range kernels across all pixels are jointly approximated (compressed) using singular value decomposition (SVD). The SVD involves a very large matrix and cannot be computed exactly; however, we are able to get a sufficiently accurate approximation using the Nyström method (without populating/storing the entire matrix). We show that this SVD-type decomposition allows us to approximate the adaptive bilateral filter using fast convolutions. To demonstrate the speed and accuracy of the proposed algorithm in relation to existing algorithms, we use it for texture filtering, JPEG deblocking, and detail enhancement.

Plug-and-play (PnP) method is a recent paradigm for image regularization, where the proximal operator (associated with some given regularizer) in an iterative algorithm is replaced with a powerful denoiser. Algorithmically, this involves repeated inversion (of the forward model) and denoising until convergence. Remarkably, PnP regularization produces promising results for several restoration applications. However, a fundamental question in this regard is the theoretical convergence of the PnP iterations, since the algorithm is not strictly derived from an optimization framework. This question has been investigated in recent works, but there are still many unresolved problems. For example, it is not known if convergence can be guaranteed if we use generic kernel denoisers (e.g. nonlocal means) within the ISTA framework (PnP-ISTA). We prove that, under reasonable assumptions, fixed-point convergence of PnP-ISTA is indeed guaranteed for linear inverse problems such as deblurring, inpainting and superresolution (the assumptions are verifiable for inpainting). We compare our theoretical findings with existing results, validate them numerically, and explain their practical relevance.

In the classical bilateral filter, a range kernel is used together with a spatial kernel for smoothing out fine details while simultaneously preserving edges. More recently, it has been demonstrated that even coarse textures can be smoothed using joint bilateral filtering. In this paper, we demonstrate that the superior texture filtering results can be obtained by adapting the spatial kernel at each pixel. To the best of our knowledge, spatial adaptation (of the bilateral filter) has not been explored for texture smoothing. The rationale behind adapting the spatial kernel is that one cannot smooth beyond a certain level using a fixed spatial kernel, no matter how we manipulate the range kernel. In fact, we should simply aggregate more pixels using a sufficiently wide spatial kernel to locally enhance the smoothing. Based on this reasoning, we propose to use the classical bilateral filter for texture smoothing, where we adapt the width of the spatial kernel at each pixel. We describe a simple and efficient gradient-based rule for the latter task. The attractive aspect is that we are able to develop a fast algorithm that can accelerate the computations by an order without visibly compromising the filtering quality. We demonstrate that our method outperforms classical bilateral filtering, joint bilateral filtering, and other filtering methods, and is competitive with the optimization methods. We also present some applications of texture smoothing using the proposed method.

The bilateral filter is popularly used for image enhancement. By using a range kernel along with a spatial kernel, the filter is able to smooth images without excessive blurring of edges. It has been shown that the enhancement capacity of the filter can be boosted by adapting the width of the range kernel at each pixel. A fast algorithm for grayscale images was recently proposed for this so-called adaptive bilateral filter, which is otherwise computationally expensive. This can trivially be extended for color filtering using channelwise processing. However, developing an efficient algorithm that can exploit correlations between color channels is not straightforward. We show that such a fast algorithm can be developed by first expressing the filtering in terms of the local histogram and then approximating the latter by an uniform distribution. The distribution in question is along the direction of maximum variance in the RGB space, which we compute from the local covariance (this is done efficiently using power iterations). The local covariances in turn are computed using fast convolutions. To demonstrate the effectiveness of our fast algorithm, we apply it for sharpening, detail enhancement, and deblocking.

In most state-of-the-art image restoration methods, the sum of a data-fidelity and a regularization term is optimized using an iterative algorithm such as ADMM (alternating direction method of multipliers). In recent years, the possibility of using denoisers for regularization has been explored in several works. A popular approach is to formally replace the proximal operator within the ADMM framework with some powerful denoiser. However, since most state-of-the-art denoisers cannot be posed as a proximal operator, one cannot guarantee the convergence of these so-called plug-and-play (PnP) algorithms. In fact, the theoretical convergence of PnP algorithms is an active research topic. In this letter, we consider the result of Chan et al. (IEEE TCI, 2017), where fixed-point convergence of an ADMM-based PnP algorithm was established for a class of denoisers. We argue that the original proof is incomplete, since convergence is not analyzed for one of the three possible cases outlined in the letter. Moreover, we explain why the argument for the other cases does not apply in this case. We give a different analysis to fill this gap, which firmly establishes the original convergence theorem.

In the classical bilateral filter, a fixed Gaussian range kernel is used along with a spatial kernel for edge-preserving smoothing. We consider a generalization of this filter, the so-called adaptive bilateral filter, where the center and width of the Gaussian range kernel are allowed to change from pixel to pixel. Though this variant was originally proposed for sharpening and noise removal, it can also be used for other applications, such as artifact removal and texture filtering. Similar to the bilateral filter, the brute-force implementation of its adaptive counterpart requires intense computations. While several fast algorithms have been proposed in the literature for bilateral filtering, most of them work only with a fixed range kernel. In this paper, we propose a fast algorithm for adaptive bilateral filtering, whose complexity does not scale with the spatial filter width. This is based on the observation that the concerned filtering can be performed purely in range space using an appropriately defined local histogram. We show that by replacing the histogram with a polynomial and the finite range-space sum with an integral, we can approximate the filter using analytic functions. In particular, an efficient algorithm is derived using the following innovations: the polynomial is fitted by matching its moments to those of the target histogram (this is done using fast convolutions), and the analytic functions are recursively computed using integration-by-parts. Our algorithm can accelerate the brute-force implementation by at least , without perceptible distortions in the visual quality. We demonstrate the effectiveness of our algorithm for sharpening, JPEG deblocking, and texture filtering.