Abstract. Practical image segmentation tasks concern images which must be reconstructed from noisy, distorted, and/or incomplete observations. A recent approach for solving such tasks is to perform this reconstruction jointly with the segmentation, using each to guide the other. However, this work has so far employed relatively simple segmentation methods, such as the Chan–Vese algorithm. In this paper, we present a method for joint reconstruction-segmentation using graph-based segmentation methods, which have been seeing increasing recent interest. Complications arise due to the large size of the matrices involved, and we show how these complications can be managed. We then analyze the convergence properties of our scheme. Finally, we apply this scheme to distorted versions of “two cows” images familiar from previous graph-based segmentation literature, first to a highly noised version and second to a blurred version, achieving highly accurate segmentations in both cases. We compare these results to those obtained by sequential reconstruction-segmentation approaches, finding that our method competes with, or even outperforms, those approaches in terms of reconstruction and segmentation accuracy.

We introduce a class of higher-order anisotropic total variation regularizers, which are defined for possibly inhomogeneous, smooth elliptic anisotropies, that extends the total generalized variation regularizer and its variants. We propose a primal-dual hybrid gradient approach to approximating numerically the associated gradient flow. This choice of regularizers allows us to preserve and enhance intrinsic anisotropic features in images. This is illustrated on various examples from different imaging applications: image denoising, wavelet-based image zooming, and reconstruction of surfaces from scattered height measurements.

We analyze a new notion of total anisotropic higher-order variation which, differently from total generalized variation in [K. Bredies, K. Kunisch, and T. Pock, SIAM J. Imaging Sci., 3 (2010), pp. 492--526], quantifies for possibly nonsymmetric tensor fields their variations at arbitrary order weighted by possibly inhomogeneous, smooth elliptic anisotropies. We prove some properties of this total variation and of the associated spaces of tensors with finite variations. We show the existence of solutions to a related regularity-fidelity optimization problem. We also prove a decomposition formula which appears to be helpful for the design of numerical schemes, as shown in a companion paper, where several applications to image processing are studied.

We propose a new variational model for non-linear image fusion. Our approach is based on the use of an osmosis energy term related to the one studied in Vogel et al. [44] and Weickert et al. [45]. The minimization of the proposed non-convex energy realizes visually plausible image data fusion, invariant to multiplicative brightness changes. On the practical side, it requires minimal supervision and parameter tuning and can encode prior information on the structure of the images to be fused. For the numerical solution of the proposed model, we develop a primal-dual algorithm and we apply the resulting minimization scheme to solve multi-modal face fusion, color transfer and cultural heritage conservation problems. Visual and quantitative comparisons to state-of-the-art approaches prove the out-performance and the flexibility of our method.

In Cultural Heritage (CH) imaging, data acquired within different spectral regions are often used to inspect surface and sub-surface features. Due to the experimental setup, these images may suffer from intensity inhomogeneities, which may prevent conservators from distinguishing the physical properties of the object under restoration. Furthermore, in multi-modal imaging, the transfer of information between one modality to another is often used to integrate image contents. In this paper, we apply the image osmosis model proposed in [ 4 , 10 , 12 ] to solve correct these problems arising when diagnostic CH imaging techniques based on reflectance, emission and fluorescence mode in the optical and thermal range are used. For an efficient computation, we use stable operator splitting techniques to solve the discretised model. We test our methods on real artwork datasets: the thermal measurements of the mural painting “Monocromo” by Leonardo Da Vinci, the UV-VIS-IR imaging of an ancient Russian icon and the Archimedes Palimpsest dataset.