This article introduces a variational color image decomposition and restoration model. The aim is to recover an image from its degraded version, while simultaneously decomposing the image into its cartoon and texture components. The energy involves adaptive higher-order regularizers, incorporated with an edge indicator function. This not only helps cartoon and texture decomposition, but also provides higher quality image restoration by ameliorating the staircasing effect that arises in total variation regularization methods. To realize the proposed models, we present fast and efficient iterative algorithms based on a variable splitting scheme and an augmented Lagrangian method. A convergence analysis of the proposed algorithms is also presented under certain conditions. Numerical results and comparisons demonstrate that the proposed model is more effective than state-of-the-art methods for both image decomposition and restoration.

This article introduces a novel class of active contour models for image segmentation. It makes use of nonlocal comparisons between pairs of patches within each region to be segmented. The corresponding variational segmentation problem is implemented using a level set formulation that can handle an arbitrary number of regions. The pairwise interaction of features constrains only the local homogeneity of image features, which is crucial in capturing regions with smoothly spatially varying features. This segmentation method is generic and can be adapted to various segmentation problems by designing an appropriate metric between patches. We instantiate this framework using several classes of features and metrics. Piecewise smooth grayscale and color images are handled using $L^2$ distance between image patches. We show examples of efficient segmentation of natural color images. Locally oriented textures are segmented using the $L^2$ distance between patches of Gabor coefficients. We use a Wasserstein distance between local empirical distributions for locally homogeneous random textures. A correlation metric between local motion signatures is able to segment piecewise smooth optical flows.

This article introduces a new variational image segmentation method that makes use of non-local comparisons between pairs of patches in the image and is robust to missing data (e.g. damaged pixels or large missing regions). The resulting segmentation is at the heart of a novel inpainting algorithm that also uses a non-local regularization. This segmentation and inpainting approach only requires a local homogeneity of the features inside and outside the region to be segmented. In contrast to existing region-based segmentation methods, it allows us to segment regions with smoothly varying intensity as well as multiple objects with different intensities. This comparison principle is also less sensitive to initialization than edge-based approaches.

We propose here a class of restoration algorithms for color images, based upon the Mumford-Shah (MS) model and nonlocal image information. The Ambrosio-Tortorelli and Shah elliptic approximations are defined to work in a small local neighborhood, which are sufficient to denoise smooth regions with sharp boundaries. However, texture is nonlocal in nature and requires semilocal/non-local information for efficient image denoising and restoration. Inspired from recent works (nonlocal means of Buades, Coll, Morel, and nonlocal total variation of Gilboa, Osher), we extend the local Ambrosio-Tortorelli and Shah approximations to MS functional (MS) to novel nonlocal formulations, for better restoration of fine structures and texture. We present several applications of the proposed nonlocal MS regularizers in image processing such as color image denoising, color image deblurring in the presence of Gaussian or impulse noise, color image inpainting, color image super-resolution, and color filter array demosaicing. In all the applications, the proposed nonlocal regularizers produce superior results over the local ones, especially in image inpainting with large missing regions. We also prove several characterizations of minimizers based upon dual norm formulations.

We wish to recover an original image u from several blurry-noisy versions fk, called frames. We assume a more severe degradation model, in which the image u has been blurred by a noisy (stochastic) point spread function. We consider the problem of restoring the degraded image in a variational framework. Since the recovery of u from one single frame f is a highly ill-posed problem, we formulate two minimization problems based on the multiframe approach proposed for image super-resolution by Marquina-Osher [2008]. Several experimental results for image restoration are shown, illustrating that the proposed models give visually satisfactory results.