Abstract We introduce a new class of data-fitting energies that couple image segmentation with image restoration. These functionals model the image intensity using the statistical framework of generalized linear models. By duality, we establish an information-theoretic interpretation using Bregman divergences. We demonstrate how this formulation couples in a principled way image restoration tasks such as denoising, deblurring (deconvolution), and inpainting with segmentation. We present an alternating minimization algorithm to solve the resulting composite photometric/geometric inverse problem. We use Fisher scoring to solve the photometric problem and to provide asymptotic uncertainty estimates. We derive the shape gradient of our data-fitting energy and investigate convex relaxation for the geometric problem. We introduce a new alternating split-Bregman strategy to solve the resulting convex problem and present experiments and comparisons on both synthetic and real-world images.

We present a discrete, unsupervised multi-region competition algorithm for image segmentation over different energy functionals. The number of regions present in an image does not need to be known a priori, nor their photometric properties. The algorithm jointly estimates the number of regions, their photometries, and their contours. The required regularization is provided by defining a region as a connected set of pixels. The evolving contours in the image are represented by computational particles that move as driven by an energy-minimization algorithm. We present an efficient discrete algorithm that allows minimizing a range of well-known energy functionals under the topological constraint of regions being connected components. The presented framework and algorithms are implemented in the open-source Insight Toolkit (ITK) image-processing library.