Abstract Digital terrains are a foundational element in the computer‐generated depiction of natural scenes. Given the variety and complexity of real‐world landforms, there is a need for authoring solutions that achieve perceptually realistic outcomes without sacrificing artistic control. In this paper, we propose setting aside the elevation domain in favour of modelling in the gradient domain. Such a slope‐based representation is height independent and allows a seamless blending of disparate landforms from procedural, simulation, and real‐world sources. For output, an elevation model can always be recovered using Poisson reconstruction, which can include Dirichlet conditions to constrain the elevation of points and curves. In terms of authoring our approach has numerous benefits. It provides artists with a complete toolbox, including: cut‐and‐paste operations that support warping as needed to fit the destination terrain, brushes to modify region characteristics, and sketching to provide point and curve constraints on both elevation and gradient. It is also a unifying representation that enables the inclusion of tools from the spectrum of existing procedural and simulation methods, such as painting localised high‐frequency noise or hydraulic erosion, without breaking the formalism. Finally, our constrained reconstruction is GPU optimized and executes in real‐time, which promotes productive cycles of iterative authoring.

We introduce a class of higher-order anisotropic total variation regularizers, which are defined for possibly inhomogeneous, smooth elliptic anisotropies, that extends the total generalized variation regularizer and its variants. We propose a primal-dual hybrid gradient approach to approximating numerically the associated gradient flow. This choice of regularizers allows us to preserve and enhance intrinsic anisotropic features in images. This is illustrated on various examples from different imaging applications: image denoising, wavelet-based image zooming, and reconstruction of surfaces from scattered height measurements.

We analyze a new notion of total anisotropic higher-order variation which, differently from total generalized variation in [K. Bredies, K. Kunisch, and T. Pock, SIAM J. Imaging Sci., 3 (2010), pp. 492--526], quantifies for possibly nonsymmetric tensor fields their variations at arbitrary order weighted by possibly inhomogeneous, smooth elliptic anisotropies. We prove some properties of this total variation and of the associated spaces of tensors with finite variations. We show the existence of solutions to a related regularity-fidelity optimization problem. We also prove a decomposition formula which appears to be helpful for the design of numerical schemes, as shown in a companion paper, where several applications to image processing are studied.

We present a system for the removal of objects from videos. As input, the system only needs a user to draw a few strokes on the first frame, roughly delimiting the objects to be removed. To the best of our knowledge, this is the first system allowing the semi-automatic removal of objects from videos with complex backgrounds. The key steps of our system are the following: after initialization, segmentation masks are first refined and then automatically propagated through the video. Missing regions are then synthesized using video inpainting techniques. Our system can deal with multiple, possibly crossing objects, with complex motions, and with dynamic textures. This results in a computational tool that can alleviate tedious manual operations for editing high-quality videos.

We propose a fast and automatic inpainting technique for high-definition videos which works under many challenging conditions such as a moving camera, a dynamic background or a long lasting occlusion. Built upon the previous work by Newson et al. [1] which optimizes a global patch-based function, our method makes a significant improvement, especially in motion preservation, by incorporating the optical flow in several stages of the algorithm. Moreover, code parallelization and a modification in the process of patches pairwise matching yield a significant reduction of computation time. Experimental results on both classical and challenging datasets show that our algorithm outperforms other state-of-the-art approaches.

This demonstration aims to show some resulting videos for our method presented in [1]. It is a fast and automatic inpainting technique for high-definition videos which works under many challenging conditions such as a moving camera, a dynamic background or a long-lasting occlusion. By incorporating optical flow in a global patch-based algorithm, our method provide improvements compared to the state-of-the-art, especially in motion preservation. The aim of the demonstration is to provide the audience with inpainted high resolution videos, in which some objects are removed with hardly any remaining trace of their former presence. As such, it provides an interesting complement to the conference paper.

We propose a variational approach for the reconstruction of a volume from slices. The reconstructed set is obtained as a minimizer of a geometric regularity criterion, either the perimeter or the Willmore energy, with inclusion-exclusion constraints associated with the cross sections. We propose a phase field approximation of this model, and we analyze it when the regularity criterion is the perimeter. We derive simple and accurate numerical schemes for both the perimeter-based and the Willmore-based formulations, and we illustrate with several numerical examples the performances of our approach, which proves to be effective for a large category of constraints.

Construction of a reliable graph capturing perceptual grouping cues of an image is fundamental for graph-cut based image segmentation methods. In this paper, we propose a novel sparse global/local affinity graph over superpixels of an input image to capture both short- and long-range grouping cues, and thereby enabling perceptual grouping laws, including proximity, similarity, continuity, and to enter in action through a suitable graph-cut algorithm. Moreover, we also evaluate three major visual features, namely, color, texture, and shape, for their effectiveness in perceptual segmentation and propose a simple graph fusion scheme to implement some recent findings from psychophysics, which suggest combining these visual features with different emphases for perceptual grouping. In particular, an input image is first oversegmented into superpixels at different scales. We postulate a gravitation law based on empirical observations and divide superpixels adaptively into small-, medium-, and large-sized sets. Global grouping is achieved using medium-sized superpixels through a sparse representation of superpixels' features by solving a ℓ0-minimization problem, and thereby enabling continuity or propagation of local smoothness over long-range connections. Small- and large-sized superpixels are then used to achieve local smoothness through an adjacent graph in a given feature space, and thus implementing perceptual laws, for example, similarity and proximity. Finally, a bipartite graph is also introduced to enable propagation of grouping cues between superpixels of different scales. Extensive experiments are carried out on the Berkeley segmentation database in comparison with several state-of-the-art graph constructions. The results show the effectiveness of the proposed approach, which outperforms state-of-the-art graphs using four different objective criteria, namely, the probabilistic rand index, the variation of information, the global consistency error, and the boundary displacement error.

In the context of lack of object-level annotation, we propose a model that enhances the weakly supervised deformable part model (DPM) by emphasizing the importance of size and aspect ratio of the initial class-specific root filter. For each image, to extract a reliable bounding box as this root filter estimate, we explore the generic objectness measurement to obtain a reference window based on the most salient region, and select a small set of candidate windows by adaptive thresholding and greedy Non-Maximum Suppression (NMS). The initial root filter estimate is decided by optimizing the score of overlap between the reference box and candidate boxes, as well as their corresponding objectness score. Then the derived window is treated as a positive training window for DPM training. Finally, we design a flexible enlarging-and-shrinking post-processing procedure to modify the output of DPM, which can effectively fit to the aspect ratio of the object and further improve the final accuracy. Experimental results on the challenging PASCAL VOC 2007 database demonstrate that our proposed framework is effective and competitive with the state-of-the-arts.

We propose a graph-cut based image segmentation method by constructing an affinity graph using ℓ0sparse representation. Computing first oversegmented images, we associate with all segments, that we call superpixels, a collection of features. We find the sparse representation of each set of features over the dictionary of all features by solving a ℓ0-minimization problem. Then, the connection information between superpixels is encoded as the non-zero representation coefficients, and the affinity of connected superpixels is derived by the corresponding representation error. This provides a ℓ0affinity graph that has interesting properties of long range and sparsity, and a suitable graph cut yields a segmentation. Experimental results on the BSD database demonstrate that our method provides perfectly semantic regions even with a constant segmentation number, but also that very competitive quantitative results are achieved.

Constructing a discriminative affinity graph plays an essential role in graph-based image segmentation, and feature directly influences the discriminative power of the affinity graph. In this paper, we propose a new method based on the weighted color patch to compute the weight of edges in an affinity graph. The proposed method intends to incorporate both color and neighborhood information by representing pixels with color patches. Furthermore, we assign both local and global weights adaptively for each pixel in a patch in order to alleviate the over-smooth effect of using patches. The normalized cut (NCut) algorithm is then applied on the resulting affinity graph to find partitions. We evaluate the proposed method on the Prague color texture image benchmark and the Berkeley image segmentation database. The extensive experiments show that our method is competitive compared to the other standard methods with multiple evaluation metrics.

Exemplar-based methods have proven their efficiency for the reconstruction of missing parts in a digital image. Texture as well as local geometry are often very well restored by such methods. Some applications, however, require the ability to reconstruct nonlocal geometric features, e.g., long edges. In order to do so, we propose to first compute a geometric sketch, which is then interpolated and used as a guide for the global reconstruction. In comparison with other related approaches, the originality of our work relies on the following points: (1) The geometric sketch computation is parameter-free and based on level lines, which provides a complete, reliable, and stable representation of the image. (2) The completion of the geometric sketch is fully automatic. It is done using a new—and interesting on its own—geometric inpainting approach that interpolates level lines with Euler spirals. Euler spirals are natural curves for shape completion and have been used already for edge completion and inpainting. It is the first time, however, that these curves are used for completing the whole level lines structure. (3) The general reconstruction is performed using a guided version of a classical exemplar-based method. However, we do not constrain the exemplar-based reconstruction to strictly follow the geometric guide. We actually use a new metric between blocks that consists of the sum of the classical ${{\mathrm L}^2}$ metric between any two blocks of the general image plus an ${{\mathrm L}^2}$ metric between the corresponding blocks in the completed geometric image. This is equivalent to a Lagrangian relaxation of a strictly guided reconstruction. We discuss in the paper the details of the method and some related mathematical issues, and we illustrate its efficiency on several examples.

We present the first ratio-based image segmentation method that allows imposing curvature regularity of the region boundary. Our approach is a generalization of the ratio framework pioneered by Jermyn and Ishikawa so as to allow penalty functions that take into account the local curvature of the curve. The key idea is to cast the segmentation problem as one of finding cyclic paths of minimal ratio in a graph where each graph node represents a line segment. Among ratios whose discrete counterparts can be globally minimized with our approach, we focus in particular on the elastic ratio [Formula: see text] that depends, given an image I, on the oriented boundary C of the segmented region candidate. Minimizing this ratio amounts to finding a curve, neither small nor too curvy, through which the brightness flux is maximal. We prove the existence of minimizers for this criterion among continuous curves with mild regularity assumptions. We also prove that the discrete minimizers provided by our graph-based algorithm converge, as the resolution increases, to continuous minimizers. In contrast to most existing segmentation methods with computable and meaningful, i.e., nondegenerate, global optima, the proposed approach is fully unsupervised in the sense that it does not require any kind of user input such as seed nodes. Numerical experiments demonstrate that curvature regularity allows substantial improvement of the quality of segmentations. Furthermore, our results allow drawing conclusions about global optima of a parameterization-independent version of the snakes functional: the proposed algorithm allows determining parameter values where the functional has a meaningful solution and simultaneously provides the corresponding global solution.

Blotches are very common, localized, and non persistent impairments in digitized film archive. Many methods have been proposed so far for detecting them and restoring the underlying regions. Most detection techniques rely on the hypothesis that blotches contradict a model of motion regularity and, up to a prior motion compensation, correspond to significant temporal variations of intensity with respect to a global threshold. In this paper, we propose a statistical approach to detect blotches in image sequences, which yields thresholds adapted to the local statistics of the frames, and which takes into account gray level differences in neighborhoods instead of isolated points. This approach is combined with a block-based motion estimation. The whole procedure is confronted with classical approaches on several sequences.

Object recognition, robot vision, image and film restoration may require the ability to perform disocclusion. We call disocclusion the recovery of occluded areas in a digital image by interpolation from their vicinity. It is shown in this paper how disocclusion can be performed by means of the level-lines structure, which offers a reliable, complete and contrast-invariant representation of images. Level-lines based disocclusion yields a solution that may have strong discontinuities. The proposed method is compatible with Kanizsa's amodal completion theory.