Most superpixel algorithms compute a trade-off between spatial and color features at the pixel level. Hence, they may need fine parameter tuning to balance the two measures, and highly fail to group pixels with similar local texture properties. In this paper, we address these issues with a new Texture-Aware SuperPixel (TASP) method. To accurately segment textured and smooth areas, TASP automatically adjusts its spatial constraint according to the local feature variance. Then, to ensure texture homogeneity within superpixels, a new pixel to super-pixel patch-based distance is proposed. TASP outperforms the segmentation accuracy of the state-of-the-art methods on texture and also natural color image datasets.

In this work, we propose a fast superpixel-based color transfer method (SCT) between two images. Superpixels enable to decrease the image dimension and to extract a reduced set of color candidates. We propose to use a fast approximate nearest neighbor matching algorithm in which we enforce the match diversity by limiting the selection of the same superpixels. A fusion framework is designed to transfer the matched colors, and we demonstrate the improvement obtained over exact matching results. Finally, we show that SCT is visually competitive compared to state-of-the-art methods.

Regular decompositions are necessary for most superpixel-based object recognition or tracking applications. So far in the literature, the regularity or compactness of a superpixel shape is mainly measured by its circularity. In this work, we first demonstrate that such measure is not adapted for super-pixel evaluation, since it does not directly express regularity but circular appearance. Then, we propose a new metric that considers several shape regularity aspects: convexity, balanced repartition, and contour smoothness. Finally, we demonstrate that our measure is robust to scale and noise and enables to more relevantly compare superpixel methods.

This paper deals with the difficult problem of video colorization. Methods in the literature are generally based on spatio-temporal video blocks or on frame-to-frame color propagation algorithms, each technique having its own advantages and drawbacks. In this paper, we present both a novel automatic frame-to-frame propagation approach and an interactive correction method within a variational framework. The proposed method propagates colors from an initial colorized frame to the whole grayscale video sequence. The automatic propagation results may be visually unsuitable in some cases. To overcome this limitation, a spatio-temporal functional with a user-guided correction is introduced. Two fast primal-dual algorithms are designed to solve the proposed variational models. Numerical results show the efficiency and the potentiality of the proposed approach in comparison with state-of-the-art methods.

Superpixels have become very popular in many computer vision applications. Nevertheless, they remain under-exploited, since the superpixel decomposition may produce irregular and nonstable segmentation results due to the dependency to the image content. In this paper, we first introduce a novel structure, a superpixel-based patch, called SuperPatch. The proposed structure, based on superpixel neighborhood, leads to a robust descriptor, since spatial information is naturally included. The generalization of the PatchMatch method to SuperPatches, named SuperPatchMatch, is introduced. Finally, we propose a framework to perform fast segmentation and labeling from an image database, and demonstrate the potential of our approach, since we outperform, in terms of computational cost and accuracy, the results of state-of-the-art methods on both face labeling and medical image segmentation.

This paper proposes a novel model for contrast enhancement of RGB images. The average local contrast measure is increased within a variational framework which preserves the hue of the original image by coupling the channels. The user is enabled to intuitively control the level of the contrast as well as the scale of the enhanced details. Moreover, our model avoids large modifications of the original image histogram and thereby preserves the global illumination of the scene. The minimizer of the proposed functional is computed by a hybrid primal-dual algorithm. Numerical experiments show the reliability of the proposed approach in comparison with state-of-the-art methods.

Superpixel decomposition methods are generally used as a pre-processing step to speed up image processing tasks. They group the pixels of an image into homogeneous regions while trying to respect existing contours. For all state-of-the-art superpixel decomposition methods, a trade-off is made between 1) computational time, 2) adherence to image contours and 3) regularity and compactness of the decomposition. In this paper, we propose a fast method to compute Superpixels with Contour Adherence using Linear Path (SCALP) in an iterative clustering framework. The distance computed when trying to associate a pixel to a superpixel during the clustering is enhanced by considering the linear path to the superpixel barycenter. The proposed framework produces regular and compact superpixels that adhere to the image contours. We provide a detailed evaluation of SCALP on the standard Berkeley Segmentation Dataset. The obtained results outperform state-of-the-art methods in terms of standard superpixel and contour detection metrics.

This paper addresses an application that has not been much explored, the de-identification of faces with expressions preservation in images. With the huge amount of images and videos shared on the Internet, protecting the identity of people in the data becomes crucial. Removing the identity information is often referred as de-identification. In this paper, we propose a novel de-identification process that preserves the important clues on the face for further behavior or emotions analysis. It is a difficult problem because obtaining the anonymity implies deteriorating the main face components. At the opposite, analyzing the expressions requires keeping enough information on the face such as, for instance, the gaze or the corners of the lips. Our approach relies on face and key points detection, followed by a variational adaptive filtering. Experimental results show the potential of the proposed method and open new insights for image dissemination or video broadcasting.

This paper provides a new method to colorize gray-scale images. While the computation of the luminance channel is directly performed by a linear transformation, the colorization process is an ill-posed problem that requires some priors. In the literature two classes of approach exist. The first class includes manual methods that need the user to manually add colors on the image to colorize. The second class includes exemplar-based approaches where a color image, with a similar semantic content, is provided as input to the method. These two types of priors have their own advantages and drawbacks. In this paper, a new variational framework for exemplar-based colorization is proposed. A nonlocal approach is used to find relevant color in the source image in order to suggest colors on the gray-scale image. The spatial coherency of the result as well as the final color selection is provided by a nonconvex variational framework based on a total variation. An efficient primal-dual algorithm is provided, and a proof of its convergence is proposed. In this work, we also extend the proposed exemplar-based approach to combine both exemplar-based and manual methods. It provides a single framework that unifies advantages of both approaches. Finally, experiments and comparisons with state-of-the-art methods illustrate the efficiency of our proposal.

This paper deals with the problem of image colorization. A model including total variation regularization is proposed. Our approach colorizes directly the three RGB channels, while most existing methods were only focusing on the two chrominance channels. By using the three channels, our approach is able to better preserve color consistency. Our model is non convex, but we propose an efficient primal-dual like algorithm to compute a local minimizer. Numerical examples illustrate the good behavior of our algorithm with respect to state-of-the-art methods.

In this paper, we address the problem of recovering a color image from a grayscale one. The input color data comes from a source image considered as a reference image. Reconstructing the missing color of a grayscale pixel is here viewed as the problem of automatically selecting the best color among a set of color candidates while simultaneously ensuring the local spatial coherency of the reconstructed color information. To solve this problem, we propose a variational approach where a specific energy is designed to model the color selection and the spatial constraint problems simultaneously. The contributions of this paper are twofold. First, we introduce a variational formulation modeling the color selection problem under spatial constraints and propose a minimization scheme, which computes a local minima of the defined nonconvex energy. Second, we combine different patch-based features and distances in order to construct a consistent set of possible color candidates. This set is used as input data and our energy minimization automatically selectsthe best color to transfer for each pixel of the grayscale image. Finally, the experiments illustrate the potentiality of our simple methodology and show that our results are very competitive with respect to the state-of-the-art methods.

We propose a transcription on graphs of recent continuous global active contours proposed for image segmentation to address the problem of binary partitioning of data represented by graphs. To do so, using the framework of Partial difference Equations (PdEs), we propose a family of nonlocal regularization functionals that verify the co-area formula on graphs. The gradients of a sub-graph are introduced and their properties studied. Relations, for the case of a sub-graph, between the introduced nonlocal regularization functionals and nonlocal discrete perimeters are exhibited and the co-area formula on graphs is introduced. Finally, nonlocal global minimizers can be considered on graphs with the associated energies. Experiments show the benefits of the approach for nonlocal image segmentation and high dimensional data clustering.

Mathematical morphology (MM) offers a wide range of operators to address various image processing problems. These operators can be defined in terms of algebraic (discrete) sets or as partial differential equations (PDEs). In this paper, we introduce a nonlocal PDEs-based morphological framework defined on weighted graphs. We present and analyze a set of operators that leads to a family of discretized morphological PDEs on weighted graphs. Our formulation introduces nonlocal patch-based configurations for image processing and extends PDEs-based approach to the processing of arbitrary data such as nonuniform high dimensional data. Finally, we show the potentialities of our methodology in order to process, segment and classify images and arbitrary data.

The main tools of Mathematical Morphology are a broad class of nonlinear image operators. They can be defined in terms of algebraic set operators or as Partial Differential Equations (PDEs). We propose a framework of partial difference equations on arbitrary graphs for introducing and analyzing morphological operators in local and non local configurations. The proposed framework unifies the classical local PDEs-based morphology for image processing, generalizes them for non local configurations and extends them to the processing of any discrete data living on graphs.

In this paper we present a method for impulse noise removal that makes use of spectral clustering and graph regularization. The image is modeled as a graph and local spectral analysis is performed to identify noisy and noise free pixels. On the set of noise free pixels, a topology adapted graph regularization is performed. Experimental results show the benefits of the proposed approach regarding the standard VMF when noise proportion is high.

In this paper we present a simple colorization method that relies on nonlocal graph regularization. We introduce nonlocal discrete differential operators and a family of weighted p-Laplace operators. Then, p-Laplace regularization on weighted graphs problem is presented and the associated filter family. Image colorization is then considered as a graph regularization problem for a function mapping vertices to chrominances. Several results illustrate our framework and demonstrate the benefits of nonlocal graph regularization for image colorization.

In this paper, a novel approach to mathematical morphology operations is proposed. Morphological operators based on partial differential equations (PDEs) are extended to weighted graphs of the arbitrary topologies by considering partial difference equations. We focus on a general class of morphological filters, the levelings; and propose a novel approach of such filters. Indeed, our methodology recovers classical local PDEs-based levelings in image processing, generalizes them to nonlocal configurations and extends them to process any discrete data that can be represented by a graph. Experimental results show applications and the potential of our levelings to textured image processing, region adjacency graph based multiscale leveling and unorganized data set filtering.