Consistent improvement of image priors over the years has led to the development of better inverse problem solvers. Diffusion models are the newcomers to this arena, posing the strongest known prior to date. Recently, such models operating in a latent space have become increasingly predominant due to their efficiency. In recent works, these models have been applied to solve inverse problems. Working in the latent space typically requires multiple applications of an Autoencoder during the restoration process, which leads to both computational and restoration quality challenges. In this work, we propose a new approach for handling inverse problems with latent diffusion models, where a learned degradation function operates within the latent space, emulating a known image space degradation. Usage of the learned operator reduces the dependency on the Autoencoder to only the initial and final steps of the restoration process, facilitating faster sampling and superior restoration quality. We demonstrate the effectiveness of our method on a variety of image restoration tasks and datasets, achieving significant improvements over prior art.

Blind Face Restoration (BFR) aims to recover face images suffering from unknown degradations. A recent approach to solve BFR is via plug-and-play methods for image restoration, which combine a likelihood function with pre-trained diffusion models as priors. However, as the likelihood is inherently unknown in BFR, existing methods rely instead on heuristic constraints. This leads to suboptimal distortion and identity preservation metrics. We introduce Expectation-based Likelihood Approximation with Diffusion prior (ELAD), a novel plug-and-play approach that explicitly models the likelihood function for BFR. ELAD estimates the first and second moments of the likelihood distribution by employing a Degradation Estimator to predict the degradation sequence from the input. This enables principled Bayesian inference without requiring end-to-end training. Our method achieves state-of-the-art distortion and identity preservation results compared to existing plug-and-play BFR techniques, while maintaining competitive perceptual quality. As we show, while being plug-and-play, our method still rivals end-to-end trained BFR models.