Text-to-image latent diffusion models (LDMs) have recently emerged as powerful generative models with great potential for solving inverse problems in imaging. However, leveraging such models in a Plug & Play (PnP), zero-shot manner remains challenging because it requires identifying a suitable text prompt for the unknown image of interest. Also, existing text-to-image PnP approaches are highly computationally expensive. We herein address these challenges by proposing a novel PnP inference paradigm specifically designed for embedding generative models within stochastic inverse solvers, with special attention to Latent Consistency Models (LCMs), which distill LDMs into fast generators. We leverage our framework to propose LAtent consisTency INverse sOlver (LATINO), the first zero-shot PnP framework to solve inverse problems with priors encoded by LCMs. Our conditioning mechanism avoids automatic differentiation and reaches SOTA quality in as little as 8 neural function evaluations. As a result, LATINO delivers remarkably accurate solutions and is significantly more memory and computationally efficient than previous approaches. We then embed LATINO within an empirical Bayesian framework that automatically calibrates the text prompt from the observed measurements by marginal maximum likelihood estimation. Extensive experiments show that prompt self-calibration greatly improves estimation, allowing LATINO with PRompt Optimization to define new SOTAs in image reconstruction quality and computational efficiency. The code is available at https://latino-pro.github.io

Abstract Video inpainting is the task of filling a region in a video in a visually convincing manner It is very challenging due to the high dimensionality of the data and the temporal consistency required for obtaining convincing results. Recently, diffusion models have shown impressive results in modeling complex data distributions, including images and videos. Such models remain nonetheless very expensive to train and to perform inference with, which strongly reduce their applicability to videos, and yields unreasonable computational loads. We show that in the case of video inpainting, thanks to the highly auto‐similar nature of videos, the training data of a diffusion model can be restricted to the input video and still produce very satisfying results. With this internal learning approach, where the training data is limited to a single video, our lightweight models perform very well with only half a million parameters, in contrast to the very large networks with billions of parameters typically found in the literature. We also introduce a new method for efficient training and inference of diffusion models in the context of internal learning, by splitting the diffusion process into different learning intervals corresponding to different noise levels of the diffusion process. We show qualitative and quantitative results, demonstrating that our method reaches or exceeds state of the art performance in the case of dynamic textures and complex dynamic backgrounds.

Posterior sampling has been shown to be a powerful Bayesian approach for solving imaging inverse problems. The recent plug-and-play unadjusted Langevin algorithm (PnP-ULA) has emerged as a promising method for Monte Carlo sampling and minimum mean squared error (MMSE) estimation by combining physical measurement models with deep-learning priors specified using image denoisers. However, the intricate relationship between the sampling distribution of PnP-ULA and the mismatched data-fidelity and denoiser has not been theoretically analyzed. We address this gap by proposing a posterior-$L_2$ pseudometric and using it to quantify an explicit error bound for PnP-ULA under mismatched posterior distribution. We numerically validate our theory on several inverse problems such as sampling from Gaussian mixture models and image deblurring. Our results suggest that the sensitivity of the sampling distribution of PnP-ULA to a mismatch in the measurement model and the denoiser can be precisely characterized.

Using diffusion models to solve inverse problems is a growing field of research. Current methods assume the degradation to be known and provide impressive results in terms of restoration quality and diversity. In this work, we leverage the efficiency of those models to jointly estimate the restored image and unknown parameters of the degradation model such as blur kernel. In particular, we designed an algorithm based on the well-known Expectation-Minimization (EM) estimation method and diffusion models. Our method alternates between approximating the expected log-likelihood of the inverse problem using samples drawn from a diffusion model and a maximization step to estimate unknown model parameters. For the maximization step, we also introduce a novel blur kernel regularization based on a Plug & Play denoiser. Diffusion models are long to run, thus we provide a fast version of our algorithm. Extensive experiments on blind image deblurring demonstrate the effectiveness of our method when compared to other state-of-the-art approaches. Our code is available at https://github.com/claroche-r/FastDiffusionEM.

Plug & Play methods combine proximal algorithms with denoiser priors to solve inverse problems. These methods rely on the computability of the proximal operator of the data fidelity term. In this paper, we propose a Plug & Play framework based on linearized ADMM that allows us to bypass the computation of intractable proximal operators. We demonstrate the convergence of the algorithm and provide results on restoration tasks such as super-resolution and deblurring with non-uniform blur.

In this paper, we propose to regularize ill-posed inverse problems using a deep hierarchical variational autoencoder (HVAE) as an image prior. The proposed method synthesizes the advantages of i) denoiser-based Plug & Play approaches and ii) generative model based approaches to inverse problems. First, we exploit VAE properties to design an efficient algorithm that benefits from convergence guarantees of Plug-and-Play (PnP) methods. Second, our approach is not restricted to specialized datasets and the proposed PnP-HVAE model is able to solve image restoration problems on natural images of any size. Our experiments show that the proposed PnP-HVAE method is competitive with both SOTA denoiser-based PnP approaches, and other SOTA restoration methods based on generative models. The code for this project is available at https://github.com/jprost76/PnP-HVAE.

We propose a state-of-the-art method for super-resolution with non-uniform blur. Single-image super-resolution methods seek to restore a high-resolution image from blurred, subsampled, and noisy measurements. Despite their impressive performance, existing techniques usually assume a uniform blur kernel. Hence, these techniques do not generalize well to the more general case of non-uniform blur. Instead, in this paper, we address the more realistic and computationally challenging case of spatially-varying blur. To this end, we first propose a fast deep plug-and-play algorithm, based on linearized ADMM splitting techniques, which can solve the super-resolution problem with spatially-varying blur. Second, we unfold our iterative algorithm into a single network and train it end-to-end. In this way, we overcome the intricacy of manually tuning the parameters involved in the optimization scheme. Our algorithm presents remarkable performance and generalizes well after a single training to a large family of spatially-varying blur kernels, noise levels and scale factors.

Image generation is the task of producing new samples from one or several example images. Until recently, this has been done using large image databases, in particular using Generative Adversarial Networks (GANs). However, Shaham et al. [1] recently proposed the SinGAN method, which achieves this generation using a single image example. At the same time, researchers are realizing that classical patch- based methods can replace certain neural networks, with no costly training. In this paper, we present a purely patch-based method, named Patches for Single image generation (PSin), which requires no training and generates samples in seconds. Our algorithm is based on the minimization of a global, patch- based energy functional, which ensures the visual fidelity of the result to the original image. We also ensure diversity of the results by carefully choosing the initialization of the algorithm. We propose two initialization variants. We compare our results to both the original SinGAN and another recent patch-based image generation approach, both qualitatively and quantitatively using multiple metrics.

In this work we address the problem of solving ill-posed inverse problems in imaging where the prior is a variational autoencoder (VAE). Specifically we consider the decoupled case where the prior is trained once and can be reused for many different log-concave degradation models without retraining. Whereas previous MAP-based approaches to this problem lead to highly nonconvex optimization algorithms, our approach computes the joint (space-latent) MAP that naturally leads to alternate optimization algorithms and to the use of a stochastic encoder to accelerate computations. The resulting technique (JPMAP) performs joint posterior maximization using an autoencoding prior. We show theoretical and experimental evidence that the proposed objective function is quite close to biconvex. Indeed it satisfies a weak biconvexity property which is sufficient to guarantee that our optimization scheme converges to a stationary point. We also highlight the importance of correctly training the VAE using a denoising criterion, in order to ensure that the encoder generalizes well to out-of-distribution images, without affecting the quality of the generative model. This simple modification is key to providing robustness to the whole procedure. Finally we show how our joint MAP methodology relates to more common MAP approaches, and we propose a continuation scheme that makes use of our JPMAP algorithm to provide more robust MAP estimates. Experimental results also show the higher quality of the solutions obtained by our JPMAP approach with respect to other nonconvex MAP approaches which more often get stuck in spurious local optima.

Since the seminal work of Venkatakrishnan et al. in 2013, Plug & Play (PnP) methods have become ubiquitous in Bayesian imaging. These methods derive Minimum Mean Square Error (MMSE) or Maximum A Posteriori (MAP) estimators for inverse problems in imaging by combining an explicit likelihood function with a prior that is implicitly defined by an image denoising algorithm. The PnP algorithms proposed in the literature mainly differ in the iterative schemes they use for optimisation or for sampling. In the case of optimisation schemes, some recent works guarantee the convergence to a fixed point, albeit not necessarily a MAP estimate. In the case of sampling schemes, to the best of our knowledge, there is no known proof of convergence. There also remain important open questions regarding whether the underlying Bayesian models and estimators are well defined, well-posed, and have the basic regularity properties required to support these numerical schemes. To address these limitations, this paper develops theory, methods, and provably convergent algorithms for performing Bayesian inference with PnP priors. We introduce two algorithms: 1) PnP-ULA (Unadjusted Langevin Algorithm) for Monte Carlo sampling and MMSE inference; and 2) PnP-SGD (Stochastic Gradient Descent) for MAP inference. Using recent results on the quantitative convergence of Markov chains, we establish detailed convergence guarantees for these two algorithms under realistic assumptions on the denoising operators used, with special attention to denoisers based on deep neural networks. We also show that these algorithms approximately target a decision-theoretically optimal Bayesian model that is well-posed. The proposed algorithms are demonstrated on several canonical problems such as image deblurring, inpainting, and denoising, where they are used for point estimation as well as for uncertainty visualisation and quantification.

Aerial or satellite imagery is a great source for land surface analysis, which might yield land-use maps or elevation models. In this letter, we present a neural network framework for learning semantics and local height together. We show how this joint multitask learning benefits to each task on the large data set of the 2018 Data Fusion Contest. Moreover, our framework also yields an uncertainty map that allows assessing the prediction of the model. Code is available at https://github.com/marcelampc/mtl_aerial_images.

Speckle reduction is a necessary step for many applications. Very effective methods have been developed in the recent years for single-image speckle reduction and multi-temporal speckle filtering. However, to reduce the presence of sidelobes around bright targets, SAR images are spectrally weighted and this processing impacts the speckle statistics by introducing spatial correlations. These correlations severely impact speckle reduction methods that require uncorrelated speckle as input. Thus, spatial down-sampling is typically applied to reduce the speckle spatial correlations prior to speckle filtering. To better preserve the spatial resolution, we describe how to correctly resample SAR images and extract bright targets in order to process full-resolution images with speckle-reduction methods.

We present a system for the removal of objects from videos. As input, the system only needs a user to draw a few strokes on the first frame, roughly delimiting the objects to be removed. To the best of our knowledge, this is the first system allowing the semi-automatic removal of objects from videos with complex backgrounds. The key steps of our system are the following: after initialization, segmentation masks are first refined and then automatically propagated through the video. Missing regions are then synthesized using video inpainting techniques. Our system can deal with multiple, possibly crossing objects, with complex motions, and with dynamic textures. This results in a computational tool that can alleviate tedious manual operations for editing high-quality videos.

Depth estimation from a single monocular image has reached great performances thanks to recent works based on deep networks. However, as various choices of losses, architectures and experimental conditions are proposed in the literature, it is difficult to establish their respective influence on the performances. In this paper we propose an in-depth study of various losses and experimental conditions for depth regression, on NYUv2 dataset. From this study we propose a new network for depth estimation combining an encoder-decoder architecture with an adversarial loss. This network reaches top scores in the competitive evaluation of NUYv2 dataset while being simpler to train in a single phase.

We propose a fast and automatic inpainting technique for high-definition videos which works under many challenging conditions such as a moving camera, a dynamic background or a long lasting occlusion. Built upon the previous work by Newson et al. [1] which optimizes a global patch-based function, our method makes a significant improvement, especially in motion preservation, by incorporating the optical flow in several stages of the algorithm. Moreover, code parallelization and a modification in the process of patches pairwise matching yield a significant reduction of computation time. Experimental results on both classical and challenging datasets show that our algorithm outperforms other state-of-the-art approaches.

This demonstration aims to show some resulting videos for our method presented in [1]. It is a fast and automatic inpainting technique for high-definition videos which works under many challenging conditions such as a moving camera, a dynamic background or a long-lasting occlusion. By incorporating optical flow in a global patch-based algorithm, our method provide improvements compared to the state-of-the-art, especially in motion preservation. The aim of the demonstration is to provide the audience with inpainted high resolution videos, in which some objects are removed with hardly any remaining trace of their former presence. As such, it provides an interesting complement to the conference paper.

JPEG and Wavelet compression artifacts leading to Gibbs effects and loss of texture are well known and many restoration solutions exist in the literature. So is denoising, which has occupied the image processing community for decades. However, when a noisy image is compressed, the noisy wavelet coefficients can be assigned to the “wrong” quantization interval, generating artifacts that can have dramatic consequences in products derived from satellite image pairs such as sub-pixel stereo vision and digital terrain elevation models. Despite the fact that the importance of such artifacts in very high resolution satellite imaging has recently been recognized, this restoration problem has been rarely addressed in the literature. In this work we present a thorough probabilistic analysis of the wavelet outliers phenomenon, and conclude that their probabilistic nature is characterized by a single parameter related to the ratio q/σ of the compression rate and the instrumental noise. This analysis provides the conditional probability for a Bayesian MAP estimator, whereas a patch-based local Gaussian prior model is learnt from the corrupted image iteratively, like in state-of-the-art patch-based denoising algorithms, albeit with the additional difficulty of dealing with non-Gaussian noise during the learning process. The resulting joint denoising and decompression algorithm is experimentally evaluated under realistic conditions. The results show its ability to simultaneously denoise, decompress and remove wavelet outliers better than the available alternatives, both from a quantitative and a qualitative point of view. As expected, the advantage of our method is more evident for large values of q/σ.

The Surface Moisture and Ocean Salinity (SMOS) mission senses ocean salinity and soil moisture by measuring Earth's brightness temperature using interferometry in the L-band. These interferometry measurements known as visibilities constitute the SMOS L1A data product. Despite the L-band being reserved for Earth observation, the presence of illegal emitters causes radio frequency interference (RFI) that masks the energy radiated from the Earth and strongly corrupts the acquired images. Therefore, the recovery of brightness temperature from corrupted data by image restoration techniques is of major interest. In this paper, we propose a variational model to recover superresolved, denoised brightness temperature maps by decomposing the images into two components: an image T that models the Earth's brightness temperature and an image O modeling the RFIs.

Gaussian Mixture Models have become one of the major tools in modern statistical image processing, and allowed performance breakthroughs in patch-based image denoising and restoration problems. Nevertheless, their adoption level was kept relatively low because of the computational cost associated to learning such models on large image databases. This work provides a flexible and generic tool for dealing with such models without the computational penalty or parameter tuning difficulties associated to a naïve implementation of GMM-based image restoration tasks. It does so by organising the data manifold in a hirerachical multiscale structure (the Covariance Tree) that can be queried at various scale levels around any point in feature-space. We start by explaining how to construct a Covariance Tree from a subset of the input data, how to enrich its statistics from a larger set in a streaming process, and how to query it efficiently, at any scale. We then demonstrate its usefulness on several applications, including non-local image filtering, data-driven denoising, reconstruction from random samples and surface modeling from unorganized 3D points sets.

Building high dynamic range (HDR) images by combining photographs captured with different exposure times present several drawbacks, such as the need for global alignment and motion estimation in order to avoid ghosting artifacts. The concept of spatially varying pixel exposures (SVE) proposed by Nayar et al. enables to capture in only one shot a very large range of exposures while avoiding these limitations. In this paper, we propose a novel approach to generate HDR images from a single shot acquired with spatially varying pixel exposures. The proposed method makes use of the assumption stating that the distribution of patches in an image is well represented by a Gaussian Mixture Model. Drawing on a precise modeling of the camera acquisition noise, we extend the piecewise linear estimation strategy developed by Yu et al. for image restoration. The proposed method permits to reconstruct an irradiance image by simultaneously estimating saturated and under-exposed pixels and denoising existing ones, showing significant improvements over existing approaches.

Data degradation by radio frequency interferences (RFI) is one of the major challenges that SMOS and other interferometers radiometers missions have to face. Although a great number of the illegal emitters were turned off since the mission was launched, not all of the sources were completely removed. Moreover, the data obtained previously is already corrupted by these RFI. Thus, the recovery of brightness temperature from corrupted data by image restoration techniques is of major interest. In this work we propose a variational approach to recover a super-resolved, denoised brightness temperature map based on two spatial components: an image u that models the brightness temperature and an image o modeling the RFI. The approach is totally new to our knowledge, in the sense that it is directly and exclusively based on the visibilities (L1a data), and thus can also be considered as an alternative to other brightness temperature recovery methods.

We propose an automatic video inpainting algorithm which relies on the optimization of a global, patch-based functional. Our algorithm is able to deal with a variety of challenging situations which naturally arise in video inpainting, such as the correct reconstruction of dynamic textures, multiple moving objects, and moving background. Furthermore, we achieve this in an order of magnitude less execution time with respect to the state-of-the-art. We are also able to achieve good quality results on high-definition videos. Finally, we provide specific algorithmic details to make implementation of our algorithm as easy as possible. The resulting algorithm requires no segmentation or manual input other than the definition of the inpainting mask and can deal with a wider variety of situations than is handled by previous work.

Line scratch detection in old films is a particularly challenging problem due to the variable spatiotemporal characteristics of this defect. Some of the main problems include sensitivity to noise and texture, and false detections due to thin vertical structures belonging to the scene. We propose a robust and automatic algorithm for frame-by-frame line scratch detection in old films, as well as a temporal algorithm for the filtering of false detections. In the frame-by-frame algorithm, we relax some of the hypotheses used in previous algorithms in order to detect a wider variety of scratches. This step's robustness and lack of external parameters is ensured by the combined use of an a contrario methodology and local statistical estimation. In this manner, over-detection in textured or cluttered areas is greatly reduced. The temporal filtering algorithm eliminates false detections due to thin vertical structures by exploiting the coherence of their motion with that of the underlying scene. Experiments demonstrate the ability of the resulting detection procedure to deal with difficult situations, in particular in the presence of noise, texture, and slanted or partial scratches. Comparisons show significant advantages over previous work.

The film defect known as the line scratch is difficult to restore automatically due to the large number of false alarms present in scratch detection algorithms. In this paper, an algorithm for dealing with these false alarms is proposed. Validating true scratches, which is the approach generally proposed in the literature, is a difficult task since scratch characteristics are hard to determine, making tracking these defects problematic. Instead, we eliminate false alarms by analysing their compatibility with a global motion estimation. We compare our algorithm with two other scratch detection methods from the literature. Experiments show that our algorithm outperforms these two, and that the proposed temporal filtering greatly improves precision while maintaining high recall.

Estimates of soil moisture and surface salinity are of significant importance to improve meteorological and climate prediction. The SMOS mission monitor these quantities, by measuring the brightness temperature by means of L-band aperture synthesis interferometry. Despite the L-band being reserved for Earth and space exploration, SMOS images reveal large number of strong outliers, produced by illegal antennas emitting in this band. In this work we propose a variational approach to recover a super-resolved, denoised brightness temperature map. The measurements are modeled as the superposition of three super-resolved components in the spatial domain: the target brightness temperature map u, an image o modeling the outliers, and Gaussian noise n. This decomposition allows to isolate each of its constituent parts, thanks to a sparsity operator that acts on o, and a bounded variation prior on u that extrapolates its spectrum promoting a non-oscillating behavior. The proposed model is interesting in itself, as it is general enough to be applied to other restoration problems. Experiments on real and synthetic data confirm the suitability of the proposed approach.

This paper introduces a statistical method to decide whether two blocks in a pair of images match reliably. The method ensures that the selected block matches are unlikely to have occurred "just by chance." The new approach is based on the definition of a simple but faithful statistical background model for image blocks learned from the image itself. A theorem guarantees that under this model, not more than a fixed number of wrong matches occurs (on average) for the whole image. This fixed number (the number of false alarms) is the only method parameter. Furthermore, the number of false alarms associated with each match measures its reliability. This a contrario block-matching method, however, cannot rule out false matches due to the presence of periodic objects in the images. But it is successfully complemented by a parameterless self-similarity threshold. Experimental evidence shows that the proposed method also detects occlusions and incoherent motions due to vehicles and pedestrians in nonsimultaneous stereo.

In most digital cameras, and even in high-end digital single lens reflex cameras, the acquired images are sampled at rates below the Nyquist critical rate, causing aliasing effects. This work introduces an algorithm for the subpixel estimation of the point spread function (PSF) of a digital camera from aliased photographs. The numerical procedure simply uses two fronto-parallel photographs of any planar textured scene at different distances. The mathematical theory developed herein proves that the camera PSF can be derived from these two images, under reasonable conditions. Mathematical proofs supplemented by experimental evidence show the well-posedness of the problem and the convergence of the proposed algorithm to the camera in-focus PSF. An experimental comparison of the resulting PSF estimates shows that the proposed algorithm reaches the accuracy levels of the best nonblind state-of-the-art methods.

This article explores the subpixel accuracy attainable for the disparity computed from a rectified stereo pair of images with small baseline. In this framework we consider translations as the local deformation model between patches in the images. A mathematical study first shows how discrete block-matching can be performed with arbitrary precision under Shannon–Whittaker conditions. This study leads to the specification of a block-matching algorithm which is able to refine disparities with subpixel accuracy. Moreover, a formula for the variance of the disparity error caused by the noise is introduced and proved. Several simulated and real experiments show a decent agreement between this theoretical error variance and the observed root mean squared error in stereo pairs with good signal-to-noise ratio and low baseline. A practical consequence is that under realistic sampling and noise conditions in optical imaging, the disparity map in stereo-rectified images can be computed for the majority of pixels (but only for those pixels with meaningful matches) with a $1/20$ pixel precision.

Restoring a regular image from irregular samples was shown feasible via quadratic regularization using Fourier and spline representations. When the image is also blurred and noisy (as is usually the case in satellite imaging) ℓ1regularizers (like TV) were shown most effective, but their Fourier-domain implementation has a prohibitive computational cost. We present here a new method that combines a spline representation (for speed) with TV regularization to obtain a more accurate and good-quality restored image. Extending this approach to the blurred case is not as trivial as in the Fourier representation. Indeed, in order to avoid the sampling operator to lose its sparse structure, a projection of the convolution operator on a spline space becomes necessary. Extensive experimental results with automatic regularization and stopping criteria show that our method achieves the accuracy of with much less computational cost, closer to.

We propose a new method for fast detection of planar patches in disparity maps. We first use a region growing algorithm on random seeds. This approach is similar to the one introduced in [1] for fast line segment detection in images. Then, the parameter-free criterion introduced in [2] is used to keep only the patches that are planar. The main advantage of our method is to be able to estimate the disparity map precision which is usually a critical parameter in other methods. This method is specially well suited to 3D reconstruction of urban environments from low-baseline aerial or satellite stereo pairs where a piecewise-planar model can be applied.

This paper proposes a statistical rejection rule, designed for small baseline stereo satellites. The method learns an a contrario model for image blocks and discards the casual matches between the images of the stereo pair. A formula estimating the expected number of false alarms under the background model is proved. Comparative experiments on quasi-simultaneous stereo in aerial imagery demonstrate the elimination of all incoherent motions.

The obtention of 3D information from two images requires the perfect control of a long chain of algorithms: internal and external calibration, stereo-rectification, correlation, and finally 3D reconstruction. In this paper we focus on the improvement of the correlation step for small baseline stereo. In that setting a very strong sub-pixel accuracy is possible. This accuracy is also necessary to obtain high resolution urban maps in geographic information systems. We show that if the images are carefully taken, then the disparity map in stereo-rectified images can be computed for a majority of image points to a 1/20 pixel precision under realistic noise conditions. Experiments on the Middlebury benchmark also stress the need for a methodology to create reliable ground truths.

Minimal surface regularization has been used in several applications ranging from stereo to image segmentation, sometimes hidden as a graph-cut discrete formulation, or as a strictly convex approximation to TV minimization. In this paper we consider a modified version of minimal surface regularization coupled with a robust data fitting term for interpolation purposes, where the corresponding evolution equation is constrained to diffuse only along the isophotes of a given image u and we design a convergent numerical scheme to accomplish this. To illustrate the usefulness of our approach, we apply this framework to the digital elevation model interpolation and to constrained vector probability diffusion. 1

Traditionally, discrete images are assumed to be sampled on a square grid and from a special kind of band-limited continuous image, namely one whose Fourier spectrum is contained within the rectangular "reciprocal cell" associated with the sampling grid. With such a simplistic model, resolution is just given by the distance between sample points. Whereas this model matches to some extent the characteristics of traditional acquisition systems, it doesn't explain aliasing problems, and it is no longer valid for certain modern ones, where the sensors may show a heavily anisotropic transfer function, and may be located on a non-square (in most cases hexagonal) grid. In this work we first summarize the generalizations of Fourier theory and of Shannon's sampling theorem, that are needed for such acquisition devices. Then we explore its consequences: (i) A new way of measuring the effective resolution of an image acquisition system; (ii) A more accurate way of restoring the original image which is represented by the samples. We show on a series of synthetic and real images, how the proposed methods make a better use of the information present in the samples, since they may drastically reduce the amount of aliasing with respect to traditional methods. Finally we show how in combination with Total Variation minimization, the proposed methods can be used to extrapolate the Fourier spectrum in a reasonable manner, visually increasing image resolution.

Even though vanishing points in digital images result from parallel lines in the 3D scene, most of the proposed detection algorithms are forced to rely heavily either on additional properties (like orthogonality or coplanarity and equal distance) of the underlying 3D lines, or on knowledge of the camera calibration parameters, in order to avoid spurious responses. In this work, we develop a new detection algorithm that relies on the Helmoltz principle recently proposed for computer vision by Desolneux et al (2001; 2003), both at the line detection and line grouping stages. This leads to a vanishing point detector with a low false alarms rate and a high precision level, which does not rely on any a priori information on the image or calibration parameters, and does not require any parameter tuning.

Interpolation of digital elevation models becomes necessary in many situations, for instance, when constructing them from contour lines (available e.g., from nondigital cartography), or from disparity maps based on pairs of stereoscopic views, which often leaves large areas where point correspondences cannot be found reliably. The absolutely minimizing Lipschitz extension (AMLE) model is singled out as the simplest interpolation method satisfying a set of natural requirements. In particular, a maximum principle is proven, which guarantees not to introduce unnatural oscillations which is a major problem with many classical methods. The authors then discuss the links between the AMLE and other existing methods. In particular, they show its relation with geodesic distance transformation. They also relate the AMLE to the thin-plate method, that can be obtained by a prolongation of the axiomatic arguments leading to the AMLE, and addresses the major disadvantage of the AMLE model, namely its inability to interpolate slopes as it does for values. Nevertheless, in order to interpolate slopes, they have to give up the maximum principle and authorize the appearance of oscillations. They also discuss the possible link between the AMLE and Kriging methods that are the most widely used in the geoscience literature.

A common approach in fingerprint matching algorithms consists of minimizing a similarity measure between feature vectors of both images, over a set of linear transformations of one image to the other. In this work we propose the thin-plate spline as a more accurate model for the geometric transformations that arise in fingerprint images. In addition we show how such a model can be integrated into a matching algorithm by means of a two-step iterative minimization with auxiliary variables. Such a method allows to correct many of the false pairings of minutiae commonly found by matching algorithms based on linear transforms.

This work presents two mechanisms for processing fingerprint images; shape-adapted smoothing based on second moment descriptors and automatic scale selection based on normalized derivatives. The shape adaptation procedure adapts the smoothing operation to the local ridge structures, which allows interrupted ridges to be joined without destroying essential singularities such as branching points and enforces continuity of their directional fields. The scale selection procedure estimates local ridge width and adapts the amount of smoothing to the local amount of noise. In addition, a ridgeness measure is defined, which reflects how well the local image structure agrees with a qualitative ridge model, and is used for spreading the results of shape adaptation into noisy areas. The combined approach makes it possible to resolve fine scale structures in clear areas while reducing the risk of enhancing noise in blurred or fragmented areas. The result is a reliable and adaptively detailed estimate of the ridge orientation field and ridge width, as well as a smoothed grey-level version of the input image. We propose that these general techniques should be of interest to developers of automatic fingerprint identification systems as well as in other applications of processing related types of imagery.