Even though data annotation is extremely important for interpretability, research, and development of artificial intelligence solutions, annotating data remains costly. Research efforts such as active learning or few-shot learning alleviate the cost by increasing sample efficiency, yet the problem of annotating data more quickly has received comparatively little attention. Leveraging a predictor has been shown to reduce annotation cost in practice but has not been theoretically considered. We ask the following question: to annotate a binary classification dataset with N samples, can the annotator answer less than N yes/no questions? Framing this question-and-answer (Q&A) game as an optimal encoding problem, we find a positive answer given by the Huffman encoding of the possible labelings. Unfortunately, the algorithm is computationally intractable even for small dataset sizes. As a practical method, we propose to minimize a cost function a few steps ahead, similarly to lookahead minimization in optimal control. This solution is analyzed, compared with the optimal one, and evaluated using several synthetic and real-world datasets. The method allows a significant improvement (23-86%) in the annotation efficiency of real-world datasets.

Applying style transfer to a full 3D environment is a challenging task that has seen many developments since the advent of neural rendering. 3D Gaussian splatting (3DGS) has recently pushed further many limits of neural rendering in terms of training speed and reconstruction quality. This work introduces SGSST: Scaling Gaussian Splatting Style Transfer, an optimization-based method to apply style transfer to pretrained 3DGS scenes. We demonstrate that a new multiscale loss based on global neural statistics, that we name SOS for Simultaneously Optimized Scales, enables style transfer to ultra-high resolution 3D scenes. Not only SGSST pioneers 3D scene style transfer at such high image resolutions, it also produces superior visual quality as assessed by thorough qualitative, quantitative and perceptual comparisons.

Palms are ecologically and economically indicators of tropical forest health, biodiversity, and human impact that support local economies and global forest product supply chains. While palm detection in plantations is well-studied, efforts to map naturally occurring palms in dense forests remain limited by overlapping crowns, uneven shading, and heterogeneous landscapes. We develop PRISM (Processing, Inference, Segmentation, and Mapping), a flexible pipeline for detecting and localizing palms in dense tropical forests using large orthomosaic images. Orthomosaics are created from thousands of aerial images and spanning several to hundreds of gigabytes. Our contributions are threefold. First, we construct a large UAV-derived orthomosaic dataset collected across 21 ecologically diverse sites in western Ecuador, annotated with 8,830 bounding boxes and 5,026 palm center points. Second, we evaluate multiple state-of-the-art object detectors based on efficiency and performance, integrating zero-shot SAM~2 as the segmentation backbone, and refining the results for precise geographic mapping. Third, we apply calibration methods to align confidence scores with IoU and explore saliency maps for feature explainability. Though optimized for palms, PRISM is adaptable for identifying other natural objects, such as eastern white pines. Future work will explore transfer learning for lower-resolution datasets (0.5–1m). Data and code can be found at github.com/Zippppo/PRISM.

Abstract. Automatic anomaly detection (AD) in a series of images of industrial parts is a key component of industrial production and an exemplary problem for machine learning. Since it can only realistically function with minimal supervision, unsupervised methods dominate the field. Their principle is that the “normal aspect” of objects is learned from recently observed samples, so that anomalies can be detected as outliers. In this paper, we start by reviewing recent AD methods and their performance-based ranking on recent benchmark datasets. The recent progress of such methods is such that they learn from a few hundred normal samples only. However, we argue that the current method evaluation based on static datasets is limited and biased. Indeed, a main feature of industrial production is that the aspect of objects evolves over time, due to changes in production and acquisition conditions, thus leading to significant probability distribution shifts. By introducing artificial but realistic deviations into the classic MVTec benchmark we show that the smallest deviation is sufficient to make these stationary models collapse. We argue that some of these models, especially the stochastic ones, can be easily adapted to cope with distribution shifts. The Global-to-Local Anomaly Detector (GLAD) is such an example of a method that uses Gaussian Mixture Models to model the distribution of regular objects. Using the stochastic approximation of expectation maximization, we design Online-GLAD, an improved GLAD that can update and adapt online. In the experiments, we show that Online-GLAD is able to maintain good performance even in the presence of multiple progressive deviations, and with constant complexity compatible with real-time implementation.

Abstract. Image vectorization converts raster images into vector graphics composed of regions separated by curves. Typical vectorization methods first define the regions by grouping similar colored regions by color quantization, then approximate their boundaries by Bézier curves. In that way, the raster input is converted into an SVG format parameterizing the regions’ colors and the Bézier control points. This compact representation has many graphical applications thanks to its universality and resolution-independence. In this paper, we remark that image vectorization is nothing but an image segmentation, and that it can be built by fine to coarse region merging. Our analysis of the problem leads us to propose a vectorization method that alternates region merging and curve smoothing. We formalize the method by alternate operations on the dual and primal graph induced by any domain partition. In that way, we address a limitation of current vectorization methods, which separate the update of regional information from curve approximation. We formalize region merging methods by associating them with various gain functionals, including the classic Beaulieu–Goldberg and Mumford–Shah functionals. More generally, we introduce and compare region merging criteria that involve the number of regions, the scale, the area, and the internal standard deviation of each region. We also show that the curve smoothing, implicit in all vectorization methods, can be performed by the shape-preserving affine scale-space. We extend this flow to a network of curves and give a sufficient condition for the topological preservation of the segmentation. The general vectorization method that follows from this analysis shows explainable behaviors, explicitly controlled by a few intuitive parameters. It is experimentally compared to state-of-the-art software and proved to have comparable or superior fidelity and cost efficiency.

Understanding the spatial distribution of palms in tropical forests is essential for ecological monitoring, conservation strategies, and the sustainable integration of natural forest products into local and global supply chains. However, the analysis of remotely sensed data are challenged by overlapping palm and tree crowns, uneven shading across the canopy surface, and the heterogeneous nature of the forest landscapes, which often affect the performance of palm detection and segmentation algorithms. To overcome these issues, we introduce PalmDSNet, a deep learning framework for efficient detection, segmentation, and counting of canopy palms. To model spatial patterns, we introduce a bimodal reproduction algorithm that simulates palm propagation based on PalmDSNet outputs. We used UAV-captured imagery to create orthomosaics from 21 sites across western Ecuadorian tropical forests, covering a gradient from the everwet Chocó forests near Colombia to the drier forests of southwestern Ecuador. Expert annotations were used to create a comprehensive dataset, including 7,356 bounding boxes on image patches and 7,603 palm centers across five orthomosaics, encompassing a total area of 449 hectares. By integrating detection and spatial modeling, we effectively simulate the spatial distribution of palms in diverse and dense tropical environments, validating its utility for advanced applications in tropical forest monitoring and remote sensing analysis. The dataset can be accessed at 10.5281/zenodo.13822508, and the code to replicate the study is available at github.com/ckn3/palm-ds-sp.

Reducing methane emissions is essential to tackle climate change. Here, we address the problem of detecting automatically point source methane leaks using high resolution hyperspectral images from the PRISMA satellite. We propose an improvement of the classical matched filter method by using an adjustment coefficient. We introduce this new method under the name: Model Adjusted Matched Filter (MAMF). We show that the MAMF method reduces the fraction of false detections compared to the Matched Filter (MF) and the Adaptive Cosine Estimator (ACE) without preventing the detection of plumes. To validate the method, we use a dataset of manually annotated plumes on PRISMA images. We then show that our method outperforms the matched filter and the adaptive cosine estimator in terms of F1 score.

Abstract Neural style transfer (NST) is a deep learning technique that produces an unprecedentedly rich style transfer from a style image to a content image. It is particularly impressive when it comes to transferring style from a painting to an image. NST was originally achieved by solving an optimization problem to match the global statistics of the style image while preserving the local geometric features of the content image. The two main drawbacks of this original approach is that it is computationally expensive and that the resolution of the output images is limited by high GPU memory requirements. Many solutions have been proposed to both accelerate NST and produce images with larger size. However, our investigation shows that these accelerated methods all compromise the quality of the produced images in the context of painting style transfer. Indeed, transferring the style of a painting is a complex task involving features at different scales, from the color palette and compositional style to the fine brushstrokes and texture of the canvas. This paper provides a solution to solve the original global optimization for ultra‐high resolution (UHR) images, enabling multiscale NST at unprecedented image sizes. This is achieved by spatially localizing the computation of each forward and backward passes through the VGG network. Extensive qualitative and quantitative comparisons, as well as a perceptual study, show that our method produces style transfer of unmatched quality for such high‐resolution painting styles. By a careful comparison, we show that state‐of‐the‐art fast methods are still prone to artifacts, thus suggesting that fast painting style transfer remains an open problem.

In this article, we propose a new planar shape representation, the medial branch graph representation (MBGR) which allows to easily generate, vary and edit all-new sorts of parametric shapes. Each MBGR shape is described by a collection of connected control circles organized as a graph. Each pair of connected circles generates a branch of the shape. The boundaries of the branches are designed using cubic Bézier curves which enable shape representation in a compact SVG format. Each circle is associated with a regularity parameter locally controlling the smoothness of the branch connections. In this way, we can manage shapes having both sharp corners and smooth boundaries. To illustrate the potential of the MBGR representation, we have created an online facility ( https://mbgrg.github.io/mbgrg/ ) where the user can create/edit MBGR shapes automatically or interactively.

Sparse keypoints based methods allow to match two images in an efficient manner. However, even though they are sparse, not all generated keypoints are necessary. This uselessly increases the computational cost during the matching step and can even add uncertainty when these keypoints are not discriminatory enough, thus leading to imprecise, or even wrong, alignment. In this paper, we address the important case where the alignment deals with the same scene or the same type of object. This enables a preliminary learning of optimal keypoints, in terms of efficiency and robustness. Our fully unsupervised selection method is based on a statistical a contrario test on a small set of training images to build without any supervision a dictionary of the most relevant points for the alignment. We show the usefulness of the proposed method on two applications, the stabilization of video surveillance sequences and the fast alignment of industrial objects containing repeated patterns. Our experiments demonstrate an acceleration of the method by 20 factor and significant accuracy gain.

We propose a novel variational image vectorization algorithm (VIVA) which alternatively smooths contours by affine shortening flow and eliminates spurious regions by minimizing a Mumford-Shah-type functional. We introduce dual-primal graphs representing domain partitions which allows for effective iterative computation. The method provides varying levels of simplicity on the topology of the resulted vector graphics while effectively removing pixelization. It compares favorably to the state-of-the-art (SOTA) vectorization methods.

Video manipulation detection plays a vital role in modern multimedia forensics. In particular, double compression detection provides significant clues leading to the video edition history and hinting at potential malevolent manipulation. While such an analysis is well-understood on images, the research on this subject remains lacking in videos and existing methods are not yet able to reliably detect double-compressed videos. This work presents a novel method for identifying double compression in H.264 codec videos. Our technique exploits the periodicity of frame residuals caused by fixed Group of Pictures in the initial compression, and employs an a contrario framework to minimize and control false detections. The proposed method can reliably detect double compression in videos. It does not require threshold tuning, thus enabling automatic detection. The code is available at https://github.com/li-yanhao/gop_detection.

In this article, we examine the possibility of using the interferometric phase on some fixed corners of storage tanks to infer the tank fill ratio. For the study, we use floating roof tanks for which the fill ratio can be inferred from the floating roof position. We observe a correlation between the phase double difference taken at the fixed roof of neighboring tanks and the fill ratio double difference. We highlight some challenges that require further investigation and the development of adapted InSAR techniques.

We explore an iterative annotation strategy adapted to recurrent multispectral imagery provided by constellations such as Sentinel-2 and applied to the monitoring of events that develop over time. Our key example is the tracking of the progress in the installation of solar power plants. This problem has four difficulties that seem hard to tackle with automatic methods: an unknown and variable spectrum for the solar panels, a variable background, a variable orientation of the panels causing variable cover of the ground, and a variability of lighting and atmosphere transparency due to cloud shadows and water vapor density. We found that each site is different and that only the interactive annotation of the time series can give an acceptable segmentation of the panels. At this point, we describe a weakly monitored interactive annotation tool that predicts the annotation of new paneled zones from an annotation at a previous date. In that way, the human operator intervention is aided and limited to a few corrections from date to date.

We propose a statistically based method to detect hot sources in night-time data from the Visible Infrared Imaging Radiometer Suite (VIIRS). This instrument is aboard three satellites and collects nearly global night-time imagery every day. Our method looks for bright pixels in at least two spectrally adjacent bands, and for groups of bright pixels close on the ground. Four near- to short-wave infrared bands are used, as well as a middle-wave infrared band after background subtraction. Thresholds are set automatically using the a-contrario framework. The algorithm has thus only one parameter with a clear signification: the expected number of false detections that can be tolerated in one image. A comparison to detections reported in VIIRS Nightfire shows that the proposed technique enables the detection of some supplementary points with weak signals.

Reducing methane emissions is essential to tackle climate change. Here, we address the problem of detecting automatically point source methane leaks using high resolution hyperspectral images from the PRISMA satellite. We use a variation of the Matched Filter (MF) called the Adjusted Spectral Matched Filter (ASMF) to detect methane plumes in satellite images. To remove false positives, the detected plumes are confirmed by comparing their orientation to the wind direction extracted from the standard meteorological reanalysis product ERA5. The ASMF reduces the fraction of false detections compared to the MF and without preventing the detection of plumes. To validate the method, we use a recently proposed dataset of manually annotated plumes on PRISMA images. We also compare our detection rate to the detection rate of methods using deep learning or the standard matched filter. We then show that our method outperforms those methods in terms of F1 score.

Learning to detect automatic anomalies in production plants remains a machine learning challenge. Since anomalies by definition cannot be learned, their detection must rely on a very accurate "normality model". To this aim, we introduce here a global-to-local Gaussian model for neural network features, learned from a set of normal images. This probabilistic model enables unsupervised anomaly detection. A global Gaussian mixture model of the features is first learned using all available features from normal data. This global Gaussian mixture model is then localized by an adaptation of the K-MLE algorithm, which learns a spatial weight map for each Gaussian. These weights are then used instead of the mixture weights to detect anomalies. This method enables precise modeling of complex data, even with limited data. Applied on WideResnet50-2 features, our approach outperforms the previous state of the art on the MVTec dataset, particularly on the object category. It is robust to perturbations that are frequent in production lines, such as imperfect alignment, and is on par in terms of memory and computation time with the previous state of the art.

The COVID-19 pandemy has created a radically new situation where most countries provide raw measurements of their daily incidence and disclose them in real time. This enables new machine learning forecast strategies where the prediction might no longer be based just on the past values of the current incidence curve, but could take advantage of observations in many countries. We present such a simple global machine learning procedure using all past daily incidence trend curves. Each of the 27,418 COVID-19 incidence trend curves in our database contains the values of 56 consecutive days extracted from observed incidence curves across 61 world regions and countries. Given a current incidence trend curve observed over the past four weeks, its forecast in the next four weeks is computed by matching it with the first four weeks of all samples, and ranking them by their similarity to the query curve. Then the 28 days forecast is obtained by a statistical estimation combining the values of the 28 last observed days in those similar samples. Using comparison performed by the European Covid-19 Forecast Hub with the current state of the art forecast methods, we verify that the proposed global learning method, EpiLearn, compares favorably to methods forecasting from a single past curve.

The most common reported epidemic time series in epidemiological surveillance are the daily or weekly incidence of new cases, the hospital admission count, the ICU admission count, and the death toll, which played such a prominent role in the struggle to monitor the Covid-19 pandemic. We show that pairs of such curves are related to each other by a generalized renewal equation depending on a smooth time varying delay and a smooth ratio generalizing the reproduction number. Such a functional relation is also explored for pairs of simultaneous curves measuring the same indicator in two neighboring countries. Given two such simultaneous time series, we develop, based on a signal processing approach, an efficient numerical method for computing their time varying delay and ratio curves, and we verify that its results are consistent. Indeed, they experimentally verify symmetry and transitivity requirements and we also show, using realistic simulated data, that the method faithfully recovers time delays and ratios. We discuss several real examples where the method seems to display interpretable time delays and ratios. The proposed method generalizes and unifies many recent related attempts to take advantage of the plurality of these health data across regions or countries and time, providing a better understanding of the relationship between them. An implementation of the method is publicly available at the EpiInvert CRAN package.

We propose a novel algorithm for converting quantized raster color images to resolution-independent scalable vector graphics (SVG). Starting from the discontinuity set of the input image, the algorithm connects the pieces of curves separating two constant regions to reconstruct the apparent contours of objects and interpret T-junctions and saddle points. This structure is depixelized by curve affine shortening, which requires maintaining the topology of the discontinuity set during filtering. The resulting Hierarchical Curve-based Vectorization (HCV) algorithm compares favorably to several state-of-art vectorization algorithms and software for color-quantized photos and pixel art1.

We propose a block-based signal-dependent noise estimation method on videos, that leverages inter-frame redundancy to separate noise from signal. Block matching is applied to find block pairs between two consecutive frames with similar signal. Then Ponomarenko’s method is extended by sorting pairs by their low-frequency energy and estimating noise in the high frequencies. Experiments on three datasets show that this method improves on the state of the art.

Since the commissioning phase of Sentinel-1A, several calibration studies have improved the geolocation and geometric modeling of the data. The implementation of the corrections presented in these studies is left to the user. The issues found might be confusing when working with bursts in the interferometric wide swath mode, because the geometric shifts present in the data are not usually the same at the burst boundaries. This might introduce small inconsistencies in a mosaic product if not properly handled, which is especially inconve-nient in high precision applications. This paper proposes a method to account for this effect by resampling the bursts before stitching. The method is validated with experiments on real Sentinel-1 data.

Methane emissions monitoring is essential to control methane pollution. In this paper, we propose an automatic practical methodology using time series to estimate the quantity of methane in a given plume using a multispectral satellite like Sentinel-2. Sentinel-2 proposes a low revisit time, a good spatial resolution and a low acquisition cost. Contrary to previous methods, the proposed approach does not require a manual selection of an optimal reference image. We compared its performance on an oil-and-gas site in Kazakhstan. This is the first step toward an automatic global monitoring system for methane plume detection and quantification with these satellites.

Shape from shading (SfS) enables 3D reconstruction of stockpiles from a single image. However, this method requires proper boundary conditions to work properly. Obtaining such Dirichlet and Neumann conditions is equivalent to a segmentation of the heaps. To get a fast and accurate 3D reconstruction, we propose a simple and interactive segmentation method. SfS is then applied on 0.5-meter resolution Synthetic Aperture Radar (SAR) images with more precise boundary conditions. The results show that prior segmentation is preferable to no segmentation for the stockpiles volume estimation problem. Furthermore, we show that the proposed interactive segmentation method reduces the annotation time needed for such a prior segmentation

International audience

We propose 4Point (Forensics with Positional Internal Training), an unsupervised neural network trained to assess the consistency of the image colour mosaic to find forgeries. Positional learning trains the model to learn the modulo-2 position of pixels, leveraging the translation-invariance of CNN to replicate the underlying mosaic and its potential inconsistencies. Internal learning on a single potentially forged image improves adaption and robustness to varied post-processing and counter-forensics measures. This solution beats existing mosaic detection methods, is more robust to various post-processing and counter-forensic artefacts such as JPEG compression, and can exploit traces to which state-of-the-art generic neural networks are blind. Check qbammey.github.io/4point for the code.

We propose a new method to evaluate image forensics tools, that characterizes what image cues are being used by each detector. Our method enables effortless creation of an arbitrarily large dataset of carefully tampered images in which controlled detection cues are present. Starting with raw images, we alter aspects of the image formation pipeline inside a mask, while leaving the rest of the image intact. This does not change the image’s interpretation; we thus call such alterations "non-semantic", as they yield no semantic inconsistencies. This method avoids the painful and often biased creation of convincing semantics. All aspects of image formation (noise, CFA, compression pattern and quality, etc.) can vary independently in both the authentic and tampered parts of the image. Alteration of a specific cue enables precise evaluation of the many forgery detectors that rely on this cue, and of the sensitivity of more generic forensic tools to each specific trace of forgery, and can be used to guide the combination of different methods. Based on this methodology, we create a database and conduct an evaluation of the main state-of-the-art image forensics tools, where we characterize the performance of each method with respect to each detection cue. Check qbammey.github.io/trace for the database and code.

Binary shapes, or silhouettes, are essential in human communication. They include, for example, all fonts and many logos. They can be extracted from images in raster form but require a vectorization for resolution independent editing. In this paper, we propose a mathematically founded silhouette vectorization algorithm, which converts a raster 2D shape to a Scalable Vector Graphics (SVG) format whose control points are geometrically stable under affine transformations. The proposed method can also be used as a reliable feature point detector for silhouettes. Compared to state-of-the-art graphics software, our algorithm shows a superior reduction in the number of control points for an equal or better accuracy.

Anomaly detectors are necessary to automatize industrial quality control. However, crafting such detectors is difficult due to the complexity and variability of the object even when working only with rigid objects. We show that adding a deep learning normalization step as a preprocessing step to model based detectors allows for better and more robust detections. This self-supervised normalization neural network is trained on non-anomalous data only. The proposed preprocessing method, followed by an automatic detector, achieves state-of-the-art results on rigid objects from the MvTec dataset.

The definition of an ”object” through the presentation of several of its instances is certainly one of the most efficient ways for humans and machines to learn. An object can be ”learned” from a single image, just because it is repeating. In this paper, we explore a three step algorithm to detect repeated objects in images. Starting from a graph of auto-correspondences inside an image, we first extract subgraphs composed of repetitions of unbreakable pieces of objects, that we call atoms. Then, these graphs of atoms are grouped into initial propositions of object instances. Finally, geometry inconsistencies are filtered out to end up with the final repeated object. The meaningfulness of object repetitions is measured by their Number of False Alarms (NFA), which provides a natural order among repeated objects in images; a very low NFA being a strong proof of existence of the discovered object. Source codes are available at https://rdguez-mariano.github.io/pages/autosim.

In this study, we compare the different spectral centering methods (referred to as deramping) for the Sentinel-1 images acquired with the TOPSAR method, in the context of interferometry. The deramping is a necessary step prior to image interpolation. We show the analogy between two different approaches in the literature and propose our own improvements. The proposed deramping method approximately re-centers the spectrum, i.e. a small residual spectral shift remains. We validate our improvements with experiments on Sentinel-1 data, and show that interpolating the images containing this residual spectral shift should not induce considerable errors.

The Rational Polynomial Camera (RPC) model can be used to describe a variety of image acquisition systems in remote sensing, notably optical and Synthetic Aperture Radar (SAR) sensors. RPC functions relate 3D to 2D coordinates and vice versa, regardless of physical sensor specificities, which has made them an essential tool to harness satellite images in a generic way. This article describes a terrain-independent algorithm to accurately derive a RPC model from a set of 3D-2D point correspondences based on a regularized least squares fit. The performance of the method is assessed by varying the point correspondences and the size of the area that they cover. We test the algorithm on SAR and optical data, to derive RPCs from physical sensor models or from other RPC models after composition with corrective functions.

The recent proliferation of constellations of recurrent satellites enables the constitution of temporally dense times series of registered images. We therefore propose in this paper a more in depth detection and analysis of observable changes. This approach is intended to be generic and independent of the type of satellite used, whether band limited or multispectral. It is based on a global analysis of the sequence. The detection stage is based on the definition of a residual sequence calculated from the novelty filter. A statistical approach based on the NFA test is then employed to detect significant changes. We then use these detections to classify the changes according to their nature: unique or lasting. To establish the efficiency of the method, we created an open dataset of 28 sequences of 20 images acquired by Sentinel-2 in different regions of the world. We obtain satisfactory results which are consistent with the visual observations of experts.

Image registration is a fundamental tool of remote sensing. The recent proliferatio of earth observation satellites has opened the way to the analysis of long image time series with denser temporal repetition. Given this wealth of images, it is crucial to design automatic tools to process them. We thus propose a method for the global registration of satellite image time series, that leverages their redundancy to improve in precision and robustness. By computing the relative displacement for all possible pairs of images, we are able to discard outliers and minimize the number of misaligned images. Experiments on synthetic data show that longer image series are registered with a higher precision.

Reducing methane emissions is essential to tackle climate change. Here, we address the problem of detecting large methane leaks by using hyperspectral data from the satellite Sentinel-5P. By sampling Sentinel-5P spectral data at fine scale, we detect methane absorption features in the shortwave infrared wavelength range (SWIR). Our method involves two separate steps: i) background subtraction and ii) detection of local maxima in the negative logarithmic spectrum of each pixel. In the first step, we remove the impact of the albedo using albedo maps and the impact of the atmosphere by using a principal component analysis (PCA) over a time series of past observations. In the second step, we count for each pixel the number of local maxima that correspond to a subset of local maxima in the methane absorption spectrum. This counting method allows us to set up a statistical a contrario test that controls the false alarm rate of our detections.

Cloud detection is a crucial step for automatic satellite image analysis. Some cloud detection methods exploit specially designed spectral bands, other base the detection on time series, or on the inter-band delay in push-broom satellites. Nevertheless many use cases occur where these methods do not apply. This paper describes a convolutional neural network for cloud detection in panchromatic and single-frame images. Only a per-image annotation is required, indicating which images contain clouds and which are cloud-free. Our experiments show that, in spite of using less information, the proposed method produces competitive results.

Using optical satellite images requires detecting accurately all clouds in any image. For many applications, automatic cloud detection methods are not accurate enough. We describe here a fast machine learning based annotation system and demonstrate on Sentinel-2 images its efficacy to reach in four clicks or less a more than 95% accurate cloud detector. To obtain these statistics, we constructed an eclectic database of partially cloudy images and its ground truth, and evaluated its accuracy to be larger than 98%. We then show that our fast supervised annotation is far more accurate than recent sophisticated cloud detectors.

With the widespread application of convolutional neural networks (CNNs), the traditional model based denoising algorithms are now outperformed. However, CNNs face two problems. First, they are computationally demanding, which makes their deployment especially difficult for mobile terminals. Second, experimental evidence shows that CNNs often over-smooth regular textures present in images, in contrast to traditional non-local models. In this letter, we propose a solution to both issues by combining a nonlocal algorithm with a lightweight residual CNN. s solution gives full latitude to the advantages of both models. We apply this framework to two GPU implementations of classic nonlocal algorithms (NLM and BM3D) and observe a substantial gain in both cases, performing better than the state-of-the-art with low computational requirements. Our solution is between 10 and 20 times faster than CNNs with equivalent performance and attains higher PSNR. In addition the final method shows a notable gain on images containing complex textures like the ones of the MIT Moiré dataset.

Automatically finding suspicious regions in a potentially forged image by splicing, inpainting or copy-move remains a widely open problem. Blind detection neural networks trained on benchmark data are flourishing. Yet, these methods do not provide an explanation of their detections. The more traditional methods try to provide such evidence by pointing out local inconsistencies in the image noise, JPEG compression, chromatic aberration, or in the mosaic. In this paper we develop a blind method that can train directly on unlabelled and potentially forged images to point out local mosaic inconsistencies. To this aim we designed a CNN structure inspired from demosaicing algorithms and directed at classifying image blocks by their position in the image modulo (2 × 2). Creating a diversified benchmark database using varied demosaicing methods, we explore the efficiency of the method and its ability to adapt quickly to any new data.

The classic approach to image matching consists in the detection, description and matching of keypoints. In the description, the local information surrounding the keypoint is encoded. This locality enables affine invariant methods. Indeed, smooth deformations caused by viewpoint changes are well approximated by affine maps. Despite numerous efforts, affine invariant descriptors have remained elusive. This has led to the development of IMAS (Image Matching by Affine Simulation) methods that simulate viewpoint changes to attain the desired invariance. Yet, recent CNN-based methods seem to provide a way to learn affine invariant descriptors. Still, as a first contribution, we show that current CNN-based methods are far from the state-of-the-art performance provided by IMAS. This confirms that there is still room for improvement for learned methods. Second, we show that recent advances in affine patch normalization can be used to create adaptive IMAS methods that select their affine simulations depending on query and target images. The proposed methods are shown to attain a good compromise: on the one hand, they reach the performance of state-of-the-art IMAS methods but are faster; on the other hand, they perform significantly better than non-simulating methods, including recent ones. Source codes are available at https://rdguez-mariano.github.io/pages/adimas.

The European Space Agency (ESA) Soil Moisture and Ocean Salinity (SMOS) has been providing the longest consistent data record of passive L-band (1.4 GHz) observations for more than ten years. SMOS, as well as the NASA missions SMAP and Aquarius have demonstrated the interest of L-band observations for land, ocean and cryosphere studies. The continuity of L-band observations must be assured taking into account that the spatial resolution (~ 40 km) of SMOS and SMAP is too coarse for some applications. Disaggregation strategies can be implemented but using airborne data, we show that the quality of the downscaled data cannot match that of an instrument with higher native resolution. The goal of the SMOS-HR (High Resolution) mission is to ensure the continuity of L-band observations while increasing the native resolution to 10 km. SMOS-HR will carry an array of ~ 230 antennas to perform aperture synthesis. The antenna distribution has been optimized to reduce the aliasing in the reconstructed images and SMOS-HR will incorporate advanced on-board Radio Frequency Interferences (RFI) mitigation techniques.

Exposure Fusion is a high dynamic range imaging technique fusing a bracketed exposure sequence into a high quality image. In this paper, we provide a refined version resolving its out-of-range artifact and its low-frequency halo. It improves on the original Exposure Fusion by augmenting contrast in all image parts. Furthermore, we extend this algorithm to single exposure images, thereby turning it into a competitive contrast enhancement operator. To do so, bracketed images are first simulated from a single input image and then fused by the new version of Exposure Fusion. The resulting algorithm competes with state of the art image enhancement methods.

Modeling the processing chain that has produced a video is a difficult reverse engineering task, even when the camera is available. This makes model based video processing a still more complex task. In this paper we propose a fully blind video denoising method, with two versions off-line and on-line. This is achieved by fine-tuning a pre-trained AWGN denoising network to the video with a novel frame-to-frame training strategy. Our denoiser can be used without knowledge of the origin of the video or burst and the post-processing steps applied from the camera sensor. The on-line process only requires a couple of frames before achieving visually pleasing results for a wide range of perturbations. It nonetheless reaches state-of-the-art performance for standard Gaussian noise, and can be used off-line with still better performance.

We consider the problem of detecting small targets in videos where the textured background is also possibly moving. The proposed method is based on a two steps statistical framework. In a first step, the optical flow is computed using a pyramidal scheme incorporating statistical tests for a result with reliability guarantees. In the second step, the detection of targets is changed into a problem of anomaly detection in noise, and statistical testing ensures a control of the number of false detections.

Non-local patch-based methods were until recently state-of-the-art for image denoising but are now outperformed by convolutional neural networks (CNNs). Yet they are still the best ones for video denoising, as video redundancy is a key factor to attain high denoising performance. In this work we propose a novel video denoising CNN. Non-local self-similarity is incorporated into the network via a first non-trainable layer which finds for each patch in the input image its most similar patches in a 3D spatio-temporal search region centered at the target patch. The central values of these patches are then gathered in a feature vector which is assigned to each image pixel. This information is presented to a CNN which is trained to predict a clean image. The proposed architecture achieves state-of-the-art results. To the best of our knowledge, this is the first successful application of CNNs to video denoising.

The classic approach to image matching consists in the detection, description and matching of keypoints. The descriptor encodes the local information around the keypoint. An advantage of local approaches is that viewpoint deformations are well approximated by affine maps. This motivated the quest for affine invariant local descriptors. Despite numerous efforts, such descriptors remained elusive, ultimately resulting in the compromise of using viewpoint simulations to attain affine invariance. In this work we propose a CNN-based patch descriptor which captures affine invariance without the need for viewpoint simulations. This is achieved by training a neural network to associate similar vectorial representations to patches related by affine transformations. During matching, these vectors are compared very efficiently. The invariance to translation, rotation and scale is still obtained by the first stages of SIFT, which produce the keypoints. The proposed descriptor outperforms the state-of-the-art in retaining affine invariant properties.

The Soil Moisture and Ocean Salinity (SMOS) satellite has provided, for the very first time, systematic passive L-band (1420−1427 MHz) measurements from space with a spatial resolution of ~50 Km. This contribution presents preliminary results of studies conducted for a High Resolution (HR) follow-on mission. The SMOS-HR project is currently undergoing a Phase 0 study by the French space agency. The goal is to ensure continuity of L-band measurements while increasing the spatial resolution to ~10 Km without degrading the radiometric sensitivity and keeping the revisit time of 3 days unchanged.

This article addresses the problem of estimating scene visibility in time series of satellite images. We are especially focused on satellites with few spectral bands and high revisit frequency. Our approach exploits the redundancy of information acquired during these revisits. It is based on an unsupervised algorithm that tracks local ground textures across time and detects ruptures caused by opaque clouds, haze, cirrus and shadows. Experiments have been carried out on 18 PlanetScope time series covering various locations. These time series come with hand-made labeled ground truth. We compare our results with those of the PlanetScope algorithm and demonstrate the effectiveness of the proposed method : success rates of 94% and 84% are reached for the visible and occulted regions classification.

The European Space Agency (ESA) Soil Moisture and Ocean Salinity (SMOS) satellite has provided, for the first time, systematic passive L-band (1.4 GHz) measurements from space. This new data set, with a spatial resolution of ~40 km, has allowed a number of outstanding results over land (soil moisture, vegetation properties, frozen soils, ...), ocean (salinity, meso-scale phenomena, river plumes, high winds, ...) and cryosphere. SMOS, together with the NASA missions SMAP and Aquarius, have demonstrated the interest of the continuity of L-band observations. However, higher spatial resolution (1-10 km) is needed for applications related to water resources management and food security, for instance. Over the ocean as well as in coastal areas, higher resolution will bring the possibility to study in detail meso-scale processes and salinity (and density) variations closer to the coast. Over ice, higher spatial resolution will allow to monitor melting events in the coastal regions of Antarctica, for instance. In order to ensure the continuity of Earth observations in the L-band, while improving the resolution of the current generation of radiometers, new mission concepts are needed. We present the SMOS-HR (High-Resolution) project, which is currently in Phase 0 at CNES (Centre National d'Etudes Spatiales).

This keynote presentation reviews the algorithms able to analyse a digital image and able to retrieve part of its processing history.This problem is relevant because more and more images happen to have lost their native EXIF metadata. In this presentation is described several tools gathering information about an image's compression, resampling, cropping, its gamma correction and its demosaicing process. This information may be used to detect images manipulations and sometimes its tampering. A common denominator of all detection tools is that they need a false alarms control. I'll illustrate how a false alarm rate can be rigorously associated to each detection.

Anomaly detectors address the difficult problem of detecting automatically exceptions in an arbitrary background image. Detection methods have been proposed by the thousands because each problem requires a different background model. By analyzing the existing approaches, we show that the problem can be reduced to detecting anomalies in residual images (extracted from the target image) in which noise and anomalies prevail. Hence, the general and impossible background modeling problem is replaced by simpler noise modeling, and allows the calculation of rigorous thresholds based on the a contrario detection theory. Our approach is therefore unsupervised and works on arbitrary images.

In this article we propose a new recursive video denoising method with high performance. The method is recursive and uses only the current frame and the previous denoised one. It considers the video as a set of overlapping temporal patch trajectories. Following a Bayesian approach each trajectory is modeled as linear dynamic Gaussian model and denoised by a Kalman filter. To estimate its parameters, similar patches are grouped and their trajectories are considered as sharing the same model parameters. The filtering is mainly temporal; non-local spatial similarity is only used to estimate the parameters. This temporally causal method obtains results comparable (in terms of PSNR and SSIM) to state-of-the-art methods using several frames per frame denoised, but with a higher temporal consistency.

The osmosis model is a parabolic equation reconstructing a composite image from an input generally given by the drift fields extracted from one or several images. This global model is sometimes a valid alternative to Poisson editing. It is particularly adapted to tasks where the input images' contrast vary wildly, as is the case for the application to shadow removal. In this paper we prove that the osmosis global parabolic equation can be advantageously be replaced by a stationary local elliptic equation. We state its existence and uniqueness result and give it a consistent numerical scheme. We stress three advantages of our numerical model: it yields fast local solvers applied on the regions of interest only. It gives a new flexibility for the boundary conditions, that can be mixed and therefore distinguish in the restoration cast shadows from shaded zones. Finally it maintains intact the target image outside its modified regions, which is not possible with the global model.

We propose a mathematical method to analyze the numerous algorithms performing image matching by affine simulation (IMAS). To become affine invariant they apply a discrete set of affine transforms to the images, prior to the comparison of all images by a scale invariant image matching (SIIM), like SIFT (scale invariant feature transform). Obviously this multiplication of images to be compared increases the image matching complexity. Three questions arise: (a) what is the best set of affine transforms to apply to each image to gain full practical affine invariance? (b) what is the lowest attainable complexity for the resulting method? (c) how is the underlying SIIM method chosen? We provide an explicit answer and a mathematical proof of quasi-optimality of the solution to the first question. As an answer to (b) we find that the near-optimal complexity ratio between full affine matching and scale invariant matching is more than halved, compared to the current IMAS methods. This means that the number of key points necessary for affine matching can be halved, and that the matching complexity is divided by four for exactly the same performance. This also means that an affine invariant set of descriptors can be associated with any image. The price to pay for full affine invariance is that the cardinality of this set is around 6.4 times larger than for a SIIM.

Video stabilization aims at removing the undesirable effects of camera motion by estimating its shake and applying a smoothing compensation. This paper proposes a unified mathematical analysis and classification of existing smoothing strategies. We assume that the apparent velocity induced by the camera is estimated as a set of global parametric models, typically those of a homography. We classify the existing smoothing strategies into compositional and additive methods and discuss their technical issues, particularly the definition of the boundary conditions. Our discussion of the various alternatives leads to clear-cut conclusions. It rules out the global compositional methods in favor of local linear methods and finds the adequate boundary conditions. We also show that the best smoothing strategy yields a scale-space analysis of the camera ego-motion parameters. Analyzing this scale-space on examples, we show how it is highly characteristic of the camera path, permitting us to compute ego-motion frequencies and to detect periodic ego-motions like walking or running.

In this paper we reconsider the class of patch based denoising algorithms and observe that they underperform at lower image frequencies. We solve this problem by operating them within a multiscale structure. Our main observation is that denoising algorithms cannot be trusted with the restoration of high frequency details in the image. Indeed, since denoising algorithms must impose their image prior, the fine details are either smoothed or sharpened in the result. In any case the high frequency properties of the images are altered. This realization has a profound implication on the multiscale approaches, which assume that coarse scale restorations are better denoised and hence are replaced in the finer resolutions. This leads to frequency cut-off artifacts as the coarse restorations are pasted at higher resolutions. We start by studying this phenomenon on a simple Discrete Cosine Transform (DCT) pyramid, for which the artifacts resulting from this process are evident. We propose a simple solution consisting of a “conservative recomposition” of the scales that only retains the lower frequencies of each scale, with the obvious exception of the scale at the highest resolution. This soft fusion eliminates the ringing artifacts and attenuates staircasing artifacts and low frequency bumps. An added benefit of the DCT pyramid is that it allows one to maintain the white noise at the lower resolutions, hence it can be combined with any denoising algorithm without adaptation. This soft fusion recipe can be generalized to any other pyramid structure. We apply it to a Laplacian pyramid as an example. Our proposal merges and operates any denoising algorithm into a multiscale method, with improvements both in visual quality and Peak Signal to Noise Ratio (PSNR), and with little additional complexity. The method is demonstrated on several classic or state-of-the-art denoising algorithms.

This paper addresses the question of identifying the right camera direct or inverse distortion model, permitting a high subpixel precision to fit to real camera distortion. Five classic camera distortion models are reviewed and their precision is compared for direct or inverse distortion. By definition, the three radially symmetric models can only model a distortion radially symmetric around some distortion center. They can be extended to deal with non-radially symmetric distortions by adding tangential distortion components, but still may be too simple for very accurate modeling of real cameras. The polynomial and the rational models instead miss a physical or optical interpretation, but can cope equally with radially and non-radially symmetric distortions. Indeed, they do not require the evaluation of a distortion center. When requiring high precisions, we found that the distortion modeling must also be evaluated primarily as a numerical problem. Indeed, all models except the polynomial involve a non-linear minimization, which increases the numerical risk. The estimation of a polynomial distortion model leads instead to a linear problem, which is secure and much faster. We concluded by extensive numerical experiments that, although high degree polynomials were required to reach a high precision of 1/100 pixels, such polynomials were easily estimated and produced a precise distortion modeling without overfitting. Our conclusion is validated by three independent experimental setups: the models were compared first on the lens distortion database of the Lensfun library by their distortion simulation and inversion power; second by fitting real camera distortions estimated by a non parametric algorithm; and finally by the absolute correction measurement provided by the photographs of tightly stretched strings, warranting a high straightness.

The growing size of digital images and their increasing information content in terms of bits per pixel (or signal-to-noise ratio (SNR)) lead us to ask to what extent the known samples permit restoration of the underlying continuous image. In the context of band-limited data the Shannon--Whittaker theory gives an adequate theoretical answer provided that infinitely many samples are measured. Yet, we show that the current accuracy of digital images will be limited in the future by the truncation error. Indeed, with eight-bit images this error was small compared to other perturbations such as quantization or aliasing. With 16-bit images, it is no longer negligible. To do so, we propose a method to estimate the truncation error. All of our results are expressed in terms of root mean squared error (RMSE) under the common hypothesis of band-limited weakly stationary random processes. As a first contribution, we present a general expression of the truncation RMSE involving the spectral content of the image. We then derive a simple and generic scheme to evaluate bounds on the truncation error. The actual computation of error bounds is conducted for two standard interpolation schemes, namely the Shannon--Whittaker and the DFT interpolators. These theoretical bounds reveal a specific decay of the truncation error as a function of the distance from the sample to the image boundary. The tight estimates obtained and validated on a set of experiments confirm that the truncation error can become the main error term in high dynamic range (HDR) images. In classic eight-bit images it is bound by the quantization error at a moderate distance from the image boundary but still requires large images to become manageable.

Flutter shutter (coded exposure) is a new paradigm for cameras that allows for an arbitrary increase of the exposure time when the relative camera/scene motion is uniform. The photon flux is interrupted according to a flutter shutter code. For arbitrarily severe uniform motion blur a well chosen code guarantees an invertible blur kernel. Yet, when the relative camera/scene velocity is a known constant, a flutter shutter cannot gain more than a 1.17 factor in terms of root mean-squared error compared to the optimal snapshot. In this paper, we prove that this optimality bound can be relaxed under the realistic assumption that a random model for the velocities is available. We give analytical formulae for the optimal flutter shutter code and the optimal snapshots associated with a random velocity distribution. Conversely we also prove formulae that reveal the velocity distribution underlying a given flutter shutter code.

Enhancement algorithms are absolutely necessary for the visualization of both shadowed and bright image regions. Defining algorithms that permit to visualize them simultaneously without altering the image content is therefore extremely relevant for remote sensing applications. In this paper, we present the results of two successive benchmarks which tested the performance of the state-of-the-art contrast enhancement and tone-mapping algorithms applied to satellite images. Experts from the French Space Agency Centre National d'Etudes Spatiales (CNES), Service Régional de Traitement d'Image et de Télédétection (SERTIT), and two European universities assessed the quality and fidelity of the results of several state-of-the-art enhancement algorithms on the excerpts from seven images (five Pleiades and two simulated 30-cm images). The first benchmark permitted to tighten the procedure and the selection of the test images for the second one, and to make a first selection of concurrent algorithms. The second benchmark not only included the best algorithms selected by the first benchmark but also added even more competitors in the tone-mapping class. The results of both benchmarks were coherent. They point a particular retinex-based algorithm as the best compromise between the competitive requirements of a contrast enhancement in dark regions and a preservation of detail in bright parts.

This paper presents DA3D (Data Adaptive Dual Domain De-noising), a “last step denoising” method that takes as input a noisy image and as a guide the result of any state-of-the-art denoising algorithm. The method performs frequency domain shrinkage on shape and data-adaptive patches. Unlike other dual denoising methods, DA3D doesn't process all the image samples, which allows it to use large patches (64 × 64 pixels). The shape and data-adaptive patches are dynamically selected, effectively concentrating the computations on areas with more details, thus accelerating the process considerably. DA3D also reduces the staircasing artifacts sometimes present in smooth parts of the guide images. The effectiveness of DA3D is confirmed by extensive experimentation. DA3D improves the result of almost all state-of-the-art methods, and this improvement requires little additional computation time.

Most computer vision application rely on algorithms finding local correspondences between different images. These algorithms detect and compare stable local invariant descriptors centered at scale-invariant keypoints. Because of the importance of the problem, new keypoint detectors and descriptors are constantly being proposed, each one claiming to perform better than the preceding ones. This raises the question of a fair comparison between very diverse methods. This evaluation has been mainly based on a repeatability criterion of the keypoints under a series of image perturbations (blur, illumination, noise, rotations, homotheties, homographies, etc). In this paper, we argue that the classic repeatability criterion is biased favoring algorithms producing redundant overlapped detections. We propose a sound variant of the criterion taking into account the descriptor overlap that seems to invalidate some of the community's claims of the last ten years.

In spite of many interesting attempts, the problem of automatically finding alignments in a 2D set of points seems to be still open. The difficulty of the problem is illustrated here by very simple examples. We then propose an elaborate solution. We show that a correct alignment detection depends on not less than four interlaced criteria, namely the amount of masking in texture, the relative bilateral local density of the alignment, its internal regularity, and finally a redundancy reduction step. Extending tools of the a contrario detection theory, we show that all of these detection criteria can be naturally embedded in a single probabilistic a contrario model with a single user parameter, the number of false alarms. Our contribution to the a contrario theory is the use of sophisticated conditional events on random point sets, for which expectation we nevertheless find easy bounds. By these bounds the mathematical consistency of our detection model receives a simple proof. Our final algorithm also includes a new formulation of the exclusion principle in Gestalt theory to avoid redundant detections. Aiming at reproducibility, a source code and an online demo open to any data point set are provided. The method is carefully compared to three state-of-the-art algorithms and an application to real data is discussed. Limitations of the final method are also illustrated and explained.

The camera calibration parameters and the image processing chain which generated a given image are generally not available to the receiver. This happens for example with scanned photographs and for most JPEG images. These images have undergone various nonlinear contrast changes and also linear and nonlinear filters. To deal with remnant noise in such images, we introduce a general nonparametric intensity and frequency-dependent noise model. We demonstrate by simulated and experiments with real images that this model, which requires the estimation of more than 1000 parameters, performs an efficient noise estimation. The proposed noise model is a patch model. Its estimation can therefore be used as a preliminary step to any patch-based denoising method. Our noise estimation method introduces several new tools for performing this complex estimation. One of them is a new sparse patch distance function permitting to find noisy patches with similar underlying geometry. A validation of the noise model and of its estimation method is obtained by comparing its results to ground-truth noise curves for both raw and JPEG-encoded images, and by visual inspection of the denoising results of real images. A fair comparison with the state of the art is also performed.

Arguably several thousands papers are dedicated to image denoising. Most papers assume a fixed noise model, mainly white Gaussian or Poissonian. This assumption is only valid for raw images. Yet, in most images handled by the public and even by scientists, the noise model is imperfectly known or unknown. End users only dispose the result of a complex image processing chain effectuated by uncontrolled hardware and software (and sometimes by chemical means). For such images, recent progress in noise estimation permits to estimate from a single image a noise model, which is simultaneously signal and frequency dependent. We propose here a multiscale denoising algorithm adapted to this broad noise model. This leads to a blind denoising algorithm which we demonstrate on real JPEG images and on scans of old photographs for which the formation model is unknown. The consistency of this algorithm is also verified on simulated distorted images. This algorithm is finally compared with the unique state of the art previous blind denoising method.

We present a novel method for automatic vanishing point detection based on primal and dual point alignment detection. The very same point alignment detection algorithm is used twice: First in the image domain to group line segment endpoints into more precise lines. Second, it is used in the dual domain where converging lines become aligned points. The use of the recently introduced PClines dual spaces and a robust point alignment detector leads to a very accurate algorithm. Experimental results on two public standard datasets show that our method significantly advances the state-of-the-art in the Manhattan world scenario, while producing state-of-the-art performances in non-Manhattan scenes.

The a contrario theory is a mathematical formalization of the so-called non-accidentalness principle of perception, which states that an observed configuration is relevant only when it is unlikely to appear just by chance. It has been successfully applied to several image processing and computer vision problems. In this paper, human vision is compared to an a contrario based algorithm in a simple perceptual task. For this aim, a psychophysical experiment was set up, in which subjects and the algorithm were asked to detect alignments of Gabor patches. We found that the proposed algorithm predicted accurately the subjects' responses, therefore providing an interpretation to perceptual thresholds.

We present a non-parametric method estimating an intensity and frequency dependent noise from a single image. The noise model is estimated on image patches and can be used consequently in all patch-based denoising methods. The method applies to cases where no access is granted to the image noise model, in particular to scanned photographs and JPEG images. The general noise model and the method to evaluate it are validated by comparing the estimations with the corresponding ground-truth curves for raw and JPEG images. Denoising experiments on scanned photographs also support the efficiency of the estimation method.

Image stereo pairs obtained from pinhole cameras can be stereo-rectified, thus permitting to test and use the many standard stereo matching algorithms of the literature. Yet, it is well-known that pushbroom Earth observation satellites produce image pairs that are not stereo-rectifiable. Nevertheless, we show that by a new and adequate use of the satellite calibration data, one can perform a precise local stereo-rectification of large Earth images. Based on this we built a fully automatic 3D reconstruction chain for the new Pléiades Earth observation satellite. It produces 1/10 pixel accurate Earth image stereo pairs at a high resolution. Examples will be made available online to the computer vision community.

Most papers on denoising methods assume a white Gaussian noise model. Yet in most images handled by the public or by scientific users, the noise model is unknown and is not white, because of the various processes applied to the image before it reaches the user: scanning, demosaicing, compression, de-convolution, etc. To cope with this wide ranging problem, we propose a blind multiscale denoising algorithm working for noise which is simultaneously signal and frequency dependent. On noisy images coming from diverse sources (JPEG, scans of old photographs, ...) we show perceptually convincing results. This algorithm is compared to the state-of-the-art and it is also validated on images with white noise.

We will consider the problem of detecting configurations of points regularly spaced and lying on a smooth curve. This corresponds to the notion of good continuation introduced in the Gestalt theory. We present a robust algorithm for clustering points along such curves, whilst at the same time discarding noisy samples. Based on the a contrario methodology, the detector builds upon a simple, symmetric primitive for a triplet of points, and finds statistically meaningful chains of such triplets. An efficient implementation is proposed using the Floyd-Warshall algorithm. Experiments on synthetic and real data show that the method is able to identify the perceptually relevant configuration of points in good continuation.

In this work we propose to analyze the formal properties of the center/surround versions of Retinex. Our main goal is to clarify what the “best” surround should be. Two conditions are sound or necessary from an image theoretical viewpoint: scale invariance and integrability. Then, we present a new kernel, which finds an acceptable compromise between these two conditions. This new kernel is compared with different kernels obtained from some center-surround methods.

The current state-of-the-art non-local algorithms for image denoising have the tendency to remove many low contrast details. Frequency-based algorithms keep these details, but on the other hand many artifacts are introduced. Recently, the Dual Domain Image Denoising (DDID) method has been proposed to address this issue. While beating the state-of-the-art, this algorithm still causes strong frequency domain artifacts. This paper reviews DDID under a different light, allowing to understand their origin. The analysis leads to the development of NLDD, a new denoising algorithm that outperforms DDID, BM3D and other state-of-the-art algorithms. NLDD is also three times faster than DDID and easily parallelizable.

The main approaches to texture modeling are the statistical psychophysically inspired model and the patch-based model. In the first model the texture is characterized by a sophisticated statistical signature. The associated sampling algorithm estimates this signature from the example and produces a genuinely different texture. This texture nevertheless often loses accuracy. The second model boils down to a clever copy-paste procedure, which stitches verbatim copies of large regions of the example. We propose in this communication to involve a locally Gaussian texture model in the patch space. It permits to synthesize textures that are everywhere different from the original but with better quality than the purely statistical methods.

The most popular image matching algorithm SIFT, introduced by D. Lowe a decade ago, has proven to be sufficiently scale invariant to be used in numerous applications. In practice, however, scale invariance may be weakened by various sources of error. The density of the sampling of the Gaussian scale-space and the level of blur in the input image are two of these sources. This article presents an empirical analysis of their impact on the extracted keypoints stability. We prove that SIFT is really scale and translation invariant only if the scale-space is significantly oversampled. We also demonstrate that the threshold on the difference of Gaussians value is inefficient for eliminating aliasing perturbations.

We evaluate and improve the matching precision of the SIFT method [1], defined as the root mean square error (RMSE) under a ground truth geometric transform. We first argue that the matching precision reflects to some extent the average relative localization precision between two images. For scale invariant feature detectors like SIFT, we show that the matching precision decreases with the scale of the keypoints, and that this is caused by the scale space sub-sampling in SIFT. We verify that canceling this sub-sampling therefore improves drastically the matching precision. Yet, in case of scale change, this improvement is marginal due to the coarse scale quantization in the scale space. A more sophisticated method is therefore also proposed to improve the matching precision even in case of scale change. This incremented precision is a key ingredient in many important image processing tasks requiring the best precision, such as registration, stitching, and camera calibration.

Modern Earth observation satellites are calibrated in such a way that a point on the ground can be located with an error of just a few pixels in the image domain. For many applications this error can be ignored, but this is not the case for stereo reconstruction, that requires sub-pixel accuracy. In this article we propose a method to correct this error. The method works by estimating local corrections that compensate the error relative to a reference image. The proposed method does not rely on ground control points, but only on the relative consistency of the image contents. We validate our method with Pléiades and WorldView-1 images on a representative set of geographic sites.

A recently introduced set of deep neural networks designed for the image denoising task achieves state-of-the-art performance. However, they are specialized networks in that each of them can handle just one noise level fixed in their respective training process. In this letter, by investigating the distribution invariance of the natural image patches with respect to linear transforms, we show how to make a single existing deep neural network work well across all levels of Gaussian noise, thereby allowing to significantly reduce the training time for a general-purpose neural network powered denoising algorithm.

This article proposes a new multiscale filter accelerating Monte Carlo renderer. Each pixel in the image is characterized by the colors of the rays that reach its surface. The proposed filter uses a statistical distance to compare with each other the ray color distributions associated with different pixels, at each scale. Based on this distance, it decides whether two pixels can share their rays or not. This simple and easily reproducible algorithm provides apsnrgain of 10 to 15 decibels, or equivalently accelerates the rendering process by using 10 to 30 times fewer samples without observable bias. The algorithm is consistent, does not assume a particular noise model, and is immediately extendable to synthetic movies. Being based on the ray color values only, it can be combined with all rendering effects.

A method for removing impulse noise from audio signals by fusing multiple copies of the same recording is introduced in this paper. The proposed algorithm exploits the fact that while in general multiple copies of a given recording are available, all sharing the same master, most degradations in audio signals are record-dependent. Our method first seeks for the optimal non-rigid alignment of the signals that is robust to the presence of sparse outliers with arbitrary magnitude. Unlike previous approaches, we simultaneously find the optimal alignment of the signals and impulsive degradation. This is obtained via continuous dynamic time warping computed solving an Eikonal equation. We propose to use our approach in the derivative domain, reconstructing the signal by solving an inverse problem that resembles the Poisson image editing technique. The proposed framework is here illustrated and tested in the restoration of old gramophone recordings showing promising results; however, it can be used in other applications where different copies of the signal of interest are available and the degradations are copy-dependent.

Recent state-of-the-art image denoising methods use nonparametric estimation processes for $8 \times 8$ patches and obtain surprisingly good denoising results. The mathematical and experimental evidence of two recent articles suggests that we might even be close to the best attainable performance in image denoising ever. This suspicion is supported by a remarkable convergence of all analyzed methods. Still more interestingly, most patch-based image denoising methods can be summarized in one paradigm, which unites the transform thresholding method and a Markovian Bayesian estimation. As the present paper shows, this unification is complete when the patch space is assumed to be a Gaussian mixture. Each Gaussian distribution is associated with its orthonormal basis of patch eigenvectors. Thus, transform thresholding (or a Wiener filter) is made on these local orthogonal bases. In this paper a simple patch-based Bayesian method is proposed, which on the one hand keeps most interesting features of former methods, and on the other hand slightly improves the state of the art of color images.

Photography is the art of acquiring as many photons as possible of a given scene. In classic cameras, the aperture time is irremediably limited by the risk of a motion blur when the camera and the scene are in relative motion. Nevertheless, two recent camera concepts, the Agrawal et al. flutter shutter and the Levin et al. motion-invariant photography permit one to extend indefinitely the exposure time while guaranteeing an invertible motion blur. In this paper, a complete mathematical theory of these new technologies is proposed. Modeling the capture noise, the theory furnishes explicit formulas for the signal to noise ratio $(SNR)$ of the final image after deconvolution when the motion is uniform. It puts in evidence the existence of two variants, the analog flutter shutter and the numerical flutter shutter. The results of the resulting quantitative comparison are slightly paradoxical. First, it is shown that the best camera aperture strategies are always flutter shutters, even when the aperture time is a priori fixed. Second, it is shown that the $SNR$ increase obtained by using a flutter shutter in the presence of a known motion remains bounded, even with an infinite exposure time. Incidentally, the theory gives the formula of the optimal classic snapshot in the presence of motion and compares its performance to the optimal flutter shutter.

The Gaussian mixture is a patch prior that has enjoyed tremendous success in image processing. In this work, by using Gaussian factor modeling, its dedicated expectation maximization (EM) inference, and a statistical filter selection and algorithm stopping rule, we develop SURE (Stein's unbiased risk estimator) guided piecewise linear estimation (S-PLE), a patch-based prior learning algorithm capable of delivering state-of-the-art performance at image denoising. In light of this algorithm's features and its results, we also seek to address the number of components to be included when setting up a Gaussian mixture for image patch modeling. By juxtaposing both options, we show that a simple learned prior can perform as well as, if not better than, a much richer yet fixed prior.

The frames of a video sequence can be improved by a spatial deconvolution of any motion blur not exceeding two pixels per frame. Yet, this requires an accurate blur estimation and local deconvolution, which is problematic for multiple local motions. We introduce an optimal temporal blur deconvolution filter restoring blindly any nonuniform motion blur with an amplitude below one pixel per frame. The discrete filter has a very low complexity of about 20 operations per pixel. Experiments illustrate the method on simulated data, real movies and on sequences from the Middlebury database.

This paper introduces a statistical method to decide whether two blocks in a pair of images match reliably. The method ensures that the selected block matches are unlikely to have occurred "just by chance." The new approach is based on the definition of a simple but faithful statistical background model for image blocks learned from the image itself. A theorem guarantees that under this model, not more than a fixed number of wrong matches occurs (on average) for the whole image. This fixed number (the number of false alarms) is the only method parameter. Furthermore, the number of false alarms associated with each match measures its reliability. This a contrario block-matching method, however, cannot rule out false matches due to the presence of periodic objects in the images. But it is successfully complemented by a parameterless self-similarity threshold. Experimental evidence shows that the proposed method also detects occlusions and incoherent motions due to vehicles and pedestrians in nonsimultaneous stereo.

In most digital cameras, and even in high-end digital single lens reflex cameras, the acquired images are sampled at rates below the Nyquist critical rate, causing aliasing effects. This work introduces an algorithm for the subpixel estimation of the point spread function (PSF) of a digital camera from aliased photographs. The numerical procedure simply uses two fronto-parallel photographs of any planar textured scene at different distances. The mathematical theory developed herein proves that the camera PSF can be derived from these two images, under reasonable conditions. Mathematical proofs supplemented by experimental evidence show the well-posedness of the problem and the convergence of the proposed algorithm to the camera in-focus PSF. An experimental comparison of the resulting PSF estimates shows that the proposed algorithm reaches the accuracy levels of the best nonblind state-of-the-art methods.

This paper addresses the problem of obtaining a quality photograph of a painting by multi-image fusion methods. The problem is particularly challenging because of the uncontrolled illumination conditions and of the destructive reflection speckle present in most photographs of paintings. A fully automatic image processing chain is described that, starting from several bursts of a painting taken under different angles, permits one to obtain the best possible result by eliminating highlights and motion blur by robust statistics, reducing noise by fusion, and compensating optical distortion in the registration process. This image fusion method is applicable to photographs of a painting taken with a hand-held camera without any particular setup. It works under bad lighting conditions and eliminates motion blur, even when the painting is protected by a glass screen creating structured reflections of the room. The careful discussion of each step of the processing chain also permits one to review and discuss the efficiency of the image fusion tools recently proposed in the literature and insert several new ones in the chain.

Human visual system (HVS) can perceive constant color under varying illumination conditions while digital images record information of both reflectance (physical color) of objects and illumination. Retinex theory, formulated by Edwin H. Land, aimed to simulate and explain this feature of HVS. However, to recover the reflectance from a given image is in general an ill-posed problem. In this paper, we establish an L1-based variational model for Retinex theory that can be solved by a fast computational approach based on Bregman iteration. Compared with previous works, our L1-Retinex method is more accurate for recovering the reflectance, which is illustrated by examples and statistics. In medical images such as magnetic resonance imaging (MRI), intensity inhomogeneity is often encountered due to bias fields. This is a similar formulation to Retinex theory while the MRI has some specific properties. We then modify the L1-Retinex method and develop a new algorithm for MRI data. We demonstrate the performance of our method by comparison with previous work on simulated and real data.

Plumb line lens distortion correction methods permit to avoid numerical compensation between the camera internal and external parameters in global calibration method. Once the distortion has been corrected by a plumb line method, the cam era is ensured to transform, up to the distortion precision, 3D straight lines into 2D straight lines, and therefore becomes a pinhole camera. This paper introduces a plumb line method for correcting and evaluating camera lens distortion with high precision. The evaluation criterion is defined as the average standard deviation from straightness of a set of approximately equally spaced straight strings photographed uniformly in all directions by the camera, so that their image crosses the whole camera field. The method uses an easily built "calibration harp," namely a frame on which good quality strings have been tightly stretched to ensure a very high physical straight ness. Real experiments confirm that our method produces high precision corrections (less than 0.05 pixel), approximating the distortion with a large number of degrees of freedom given by a polynomial model of order eleven.

Abstract This paper develops a scale space strategy for orienting and meshing exactly and completely a raw point set. The scale space is based on the intrinsic heat equation, also called mean curvature motion (MCM). A simple iterative scheme implementing MCM directly on the raw point set is described, and a mathematical proof of its consistency with MCM is given. Points evolved by this MCM implementation can be trivially backtracked to their initial raw position. Therefore, both the orientation and mesh of the data point set obtained at a smooth scale can be transported back on the original. The gain in visual accuracy is demonstrated on archaeological objects by comparison with several state of the art meshing methods.

This article explores the subpixel accuracy attainable for the disparity computed from a rectified stereo pair of images with small baseline. In this framework we consider translations as the local deformation model between patches in the images. A mathematical study first shows how discrete block-matching can be performed with arbitrary precision under Shannon–Whittaker conditions. This study leads to the specification of a block-matching algorithm which is able to refine disparities with subpixel accuracy. Moreover, a formula for the variance of the disparity error caused by the noise is introduced and proved. Several simulated and real experiments show a decent agreement between this theoretical error variance and the observed root mean squared error in stereo pairs with good signal-to-noise ratio and low baseline. A practical consequence is that under realistic sampling and noise conditions in optical imaging, the disparity map in stereo-rectified images can be computed for the majority of pixels (but only for those pixels with meaningful matches) with a $1/20$ pixel precision.

This paper explores the mathematical and algorithmic properties of two sample-based texture models: random phase noise (RPN) and asymptotic discrete spot noise (ADSN). These models permit to synthesize random phase textures. They arguably derive from linearized versions of two early Julesz texture discrimination theories. The ensuing mathematical analysis shows that, contrarily to some statements in the literature, RPN and ADSN are different stochastic processes. Nevertheless, numerous experiments also suggest that the textures obtained by these algorithms from identical samples are perceptually similar. The relevance of this study is enhanced by three technical contributions providing solutions to obstacles that prevented the use of RPN or ADSN to emulate textures. First, RPN and ADSN algorithms are extended to color images. Second, a preprocessing is proposed to avoid artifacts due to the nonperiodicity of real-world texture samples. Finally, the method is extended to synthesize textures with arbitrary size from a given sample.

International audience

This paper presents an image processing algorithm simulating a sub-pixel evolution of an image by mean curvature motion or by affine curvature motion. The sub-pixel algorithm computes the image curvature directly on the smoothed level lines, and yields a microscopic visualization of the curvature map revealing many image details, and getting rid of aliasing effects. This “curvature microscope” showing curvatures in false colors runs on line on any image proposed by users at http://www.ipol.im/pub/algo/cmmm_image_curvature_microscope/.

This paper points out and attempts to remedy a serious discrepancy in results obtained by global calibration methods: The re-projection error can be rendered very small by these methods, but we show that the optical distortion correction is far less accurate. This discrepancy can only be explained by internal error compensations in the global methods that leave undetected the inadequacy of the distortion model. This fact led us to design a model-free distortion correction method where the distortion can be any image domain diffeomorphism. The obtained precision compares favorably to the distortion given by state of the art global calibration and reaches a RMSE of 0.08 pixels. Nonetheless, we also show that this accuracy can still be improved.

This paper proposes a statistical rejection rule, designed for small baseline stereo satellites. The method learns an a contrario model for image blocks and discards the casual matches between the images of the stereo pair. A formula estimating the expected number of false alarms under the background model is proved. Comparative experiments on quasi-simultaneous stereo in aerial imagery demonstrate the elimination of all incoherent motions.

The obtention of 3D information from two images requires the perfect control of a long chain of algorithms: internal and external calibration, stereo-rectification, correlation, and finally 3D reconstruction. In this paper we focus on the improvement of the correlation step for small baseline stereo. In that setting a very strong sub-pixel accuracy is possible. This accuracy is also necessary to obtain high resolution urban maps in geographic information systems. We show that if the images are carefully taken, then the disparity map in stereo-rectified images can be computed for a majority of image points to a 1/20 pixel precision under realistic noise conditions. Experiments on the Middlebury benchmark also stress the need for a methodology to create reliable ground truths.

Abstract For each scanned object 3D triangulation laser scanners deliver multiple sweeps corresponding to multiple laser motions and orientations. The problem of aligning these scans has been well solved by using rigid and, more recently, non‐rigid transformations. Nevertheless, there are always residual local offsets between scans which forbid a direct merging of the scans, and force to some preliminary smoothing. Indeed, the tiling and aliasing effects due to the tiniest normal displacements of the scans can be dramatic. This paper proposes a general method to tackle this problem. The algorithm decomposes each scan into its high and low frequency components and fuses the low frequencies while keeping intact the high frequency content. It produces a mesh with the highest attainable resolution, having for vertices all raw data points of all scans. This exhaustive fusion of scans maintains the finest texture details. The method is illustrated on several high resolution scans of archeological objects.

We propose a linear-time line segment detector that gives accurate results, a controlled number of false detections, and requires no parameter tuning. This algorithm is tested and compared to state-of-the-art algorithms on a wide set of natural images.

Can images be decomposed into the sum of a geometric part and a textural part? In a theoretical breakthrough, [Y. Meyer, Oscillating Patterns in Image Processing and Nonlinear Evolution Equations. Providence, RI: American Mathematical Society, 2001] proposed variational models that force the geometric part into the space of functions with bounded variation, and the textural part into a space of oscillatory distributions. Meyer's models are simple minimization problems extending the famous total variation model. However, their numerical solution has proved challenging. It is the object of a literature rich in variants and numerical attempts. This paper starts with the linear model, which reduces to a low-pass/high-pass filter pair. A simple conversion of the linear filter pair into a nonlinear filter pair involving the total variation is introduced. This new-proposed nonlinear filter pair retains both the essential features of Meyer's models and the simplicity and rapidity of the linear model. It depends upon only one transparent parameter: the texture scale, measured in pixel mesh. Comparative experiments show a better and faster separation of cartoon from texture. One application is illustrated: edge detection.

In 1964 Edwin H. Land formulated the Retinex theory, the first attempt to simulate and explain how the human visual system perceives color. His theory and an extension, the "reset Retinex" were further formalized by Land and McCann. Several Retinex algorithms have been developed ever since. These color constancy algorithms modify the RGB values at each pixel to give an estimate of the color sensation without a priori information on the illumination. Unfortunately, the Retinex Land-McCann original algorithm is both complex and not fully specified. Indeed, this algorithm computes at each pixel an average of a very large set of paths on the image. For this reason, Retinex has received several interpretations and implementations which, among other aims, attempt to tune down its excessive complexity. In this paper, it is proved that if the paths are assumed to be symmetric random walks, the Retinex solutions satisfy a discrete screened Poisson equation. This formalization yields an exact and fast implementation using only two FFTs. Several experiments on color images illustrate the effectiveness of the Retinex original theory.

A fully affine invariant image comparison method, Affine-SIFT (ASIFT) is introduced. While SIFT is fully invariant with respect to only four parameters namely zoom, rotation and translation, the new method treats the two left over parameters : the angles defining the camera axis orientation. Against any prognosis, simulating all views depending on these two parameters is feasible. The method permits to reliably identify features that have undergone very large affine distortions measured by a new parameter, the transition tilt. State-of-the-art methods hardly exceed transition tilts of 2 (SIFT), 2.5 (Harris-Affine and Hessian-Affine) and 10 (MSER). ASIFT can handle transition tilts up 36 and higher (see Fig. 1).

If a physical object has a smooth or piecewise smooth boundary, its images obtained by cameras in varying positions undergo smooth apparent deformations. These deformations are locally well approximated by affine transforms of the image plane. In consequence the solid object recognition problem has often been led back to the computation of affine invariant image local features. Such invariant features could be obtained by normalization methods, but no fully affine normalization method exists for the time being. Even scale invariance is dealt with rigorously only by the scale-invariant feature transform (SIFT) method. By simulating zooms out and normalizing translation and rotation, SIFT is invariant to four out of the six parameters of an affine transform. The method proposed in this paper, affine-SIFT (ASIFT), simulates all image views obtainable by varying the two camera axis orientation parameters, namely, the latitude and the longitude angles, left over by the SIFT method. Then it covers the other four parameters by using the SIFT method itself. The resulting method will be mathematically proved to be fully affine invariant. Against any prognosis, simulating all views depending on the two camera orientation parameters is feasible with no dramatic computational load. A two-resolution scheme further reduces the ASIFT complexity to about twice that of SIFT. A new notion, the transition tilt, measuring the amount of distortion from one view to another, is introduced. While an absolute tilt from a frontal to a slanted view exceeding 6 is rare, much higher transition tilts are common when two slanted views of an object are compared (see Figure hightransitiontiltsillustration). The attainable transition tilt is measured for each affine image comparison method. The new method permits one to reliably identify features that have undergone transition tilts of large magnitude, up to 36 and higher. This fact is substantiated by many experiments which show that ASIFT significantly outperforms the state-of-the-art methods SIFT, maximally stable extremal region (MSER), Harris-affine, and Hessian-affine.

Demosaicking is the process by which from a matrix of colored pixels measuring only one color component per pixel, red, green, or blue, one can infer a whole color information at each pixel. This inference requires a deep understanding of the interaction between colors, and the involvement of image local geometry. Although quite successful in making such inferences with very small relative error, state-of-the-art demosaicking methods fail when the local geometry cannot be inferred from the neighboring pixels. In such a case, which occurs when thin structures or fine periodic patterns were present in the original, state-of-the-art methods can create disturbing artifacts, known as zipper effect, blur, and color spots. The aim of this paper is to show that these artifacts can be avoided by involving the image self-similarity to infer missing colors. Detailed experiments show that a satisfactory solution can be found, even for the most critical cases. Extensive comparisons with state-of-the-art algorithms will be performed on two different classic image databases.

One of the central problems of medical imaging is the three-dimensional (3D) visualization of body parts. The 3D volume can be viewed in slices, but the extraction of a part requires a segmentation process. Inasmuch as body parts are distinguishable by their various densities, a widely accepted method for extracting an organ is to extract isodensity surfaces by a simple threshold. Unfortunately, the density of organs, arteries, etc. varies spatially due to morphology, and no unique threshold allows one to extract the organs boundaries. The snake or active contour methods have attempted to capture these boundaries as smooth and overall contrasted surfaces. The snake method suffers, however, from severe drawbacks. The contour has to be initialized near the boundary. In addition, many body parts have too complex a topology. In this paper we focus on another idea, which is to preprocess the image before thresholding. The preprocessing aims at the homogeneity of the different parts while preserving small features. Starting from a recent seminal work by Grady and Funka-Lea [in Computer Vision and Mathematical Methods in Medical and Biomedical Image Analysis: ECCV 2004 Workshops CVAMIA and MMBIA, Prague, Czech Republic, May 2004, Revised Selected Papers, Springer, Berlin, 2004, pp. 230–245], several linear heat equations on images will be compared. They stem from nonlinear partial differential equations or from their associated nonlinear filters. By linearizing these processes one obtains more accurate topology preserving methods. These linear filters will be tested comparatively to visualize challenging angiography images of arteries. A salient fact of the method will emerge. By a concentration phenomenon, peaks in the image histogram become much more concentrated under the linear heat equations, thus permitting us to fix the thresholds defining the surfaces without supervision. Automatic extraction can be performed in this way for angiography images taken at a one-year or longer delay.

Many classical image denoising methods are based on a local averaging of the color, which increases the signal/noise ratio. One of the most used algorithms is the neighborhood filter by Yaroslavsky or sigma filter by Lee, also called in a variant "SUSAN" by Smith and Brady or "Bilateral filter" by Tomasi and Manduchi. These filters replace the actual value of the color at a point by an average of all values of points which are simultaneously close in space and in color. Unfortunately, these filters show a "staircase effect," that is, the creation in the image of flat regions separated by artifact boundaries. In this paper, we first explain the staircase effect by finding the subjacent partial differntial equation (PDE) of the filter. We show that this ill-posed PDE is a variant of another famous image processing model, the Perona-Malik equation, which suffers the same artifacts. As we prove, a simple variant of the neighborhood filter solves the problem. We find the subjacent stable PDE of this variant. Finally, we apply the same correction to the recently introduced NL-means algorithm which had the same staircase effect, for the same reason.

State of the art movie restoration methods either estimate motion and filter out the trajectories, or compensate the motion by an optical flow estimate and then filter out the compensated movie. Now, the motion estimation problem is ill posed. This fact is known as the aperture problem: trajectories are ambiguous since they could coincide with any promenade in the space-time isophote surface. In this paper, we try to show that, for denoising, the aperture problem can be taken advantage of. Indeed, by the aperture problem, many pixels in the neighboring frames are similar to the current pixel one wishes to denoise. Thus, denoising by an averaging process can use many more pixels than just the ones on a single trajectory. This observation leads to use for movies a recently introduced image denoising method, the NL-means algorithm. This static 3D algorithm outperforms motion compensated algorithms, as it does not lose movie details. It involves the whole movie isophote and not just a trajectory.

This paper presents a software-based parameter-free method for the demultiplexing of a video stream (L. Rudin et al., 2004) that is missing camera labeling information. The method is based on the observation that frames coming from the same input camera share some common characteristic features. These features are extracted from the input frames and grouped together according to statistical criteria. As a result of this grouping the number of different input sources in the video stream is inferred and it is possible to ascertain the source for each frame.

We propose a new measure, the method noise, to evaluate and compare the performance of digital image denoising methods. We first compute and analyze this method noise for a wide class of denoising algorithms, namely the local smoothing filters. Second, we propose a new algorithm, the nonlocal means (NL-means), based on a nonlocal averaging of all pixels in the image. Finally, we present some experiments comparing the NL-means algorithm and the local smoothing filters.

In this work, we present and analyze an image denoising method, the NL-means algorithm, based on a non local averaging of all pixels in the image. We also introduce the concept of method noise, that is, the difference between the original (always slightly noisy) digital image and its denoised version. Finally, we present some experiences comparing the NL-means results with some classical denoising methods.

In this paper the problem of detecting changes between two photographs of the same scene taken at different dates is addressed. Considering that both images are already accurately registered our problem reduces to a) detect changes between corresponding pixels, and b) determine whether these changes are a clue of a major change in the scene. The solution of a) involves the use of spectral invariant features, since we consider the general situation of comparing photographs coming from different spectral channels. The answer to b) implies the computation of an absolute threshold above which a region of the image will be detected as having "meaningful changes". In order to do that, we apply a recently introduced method yielding accurate false alarm rates for each detection. We show that this method permits to help significantly photo-interprets in their search for major changes in a scene. This will be illustrated by a striking application, the detection of the explosion of the AZF chemical plant in Toulouse (France) in September 2001.

Wertheimer's theory suggests a general perception law according to which objects having a quality in common get perceptually grouped. The Helmholtz principle is a quantitative version of this general grouping law. It states that a grouping is perceptually "meaningful" if its number of occurrences would be very small in a random situation: geometric structures are then characterized as large deviations from randomness. In two previous works, we have applied this principle to the detection of orientation alignments and boundaries in a digital image. In this paper, we show that the method is fully general and can be extended to a grouping by any quality. We treat as an illustration the alignments of objects, their grouping by color and by size, and the vicinity gestalt (clusters). Collaboration of the gestalt grouping laws and their pyramidal structure are illustrated in a case study.

We address the problem of computing a local orientation map in a digital image. We show that standard image gray level quantization causes a strong bias in the repartition of orientations, hindering any accurate geometric analysis of the image. In continuation, a simple dequantization algorithm is proposed, which maintains all of the image information and transforms the quantization noise in a nearby Gaussian white noise (we actually prove that only Gaussian noise can maintain isotropy of orientations). Mathematical arguments are used to show that this results in the restoration of a high quality image isotropy. In contrast with other classical methods, it turns out that this property can be obtained without smoothing the image or increasing the signal-to-noise ratio (SNR). As an application, it is shown in the experimental section that, thanks to this dequantization of orientations, such geometric algorithms as the detection of nonlocal alignments can be performed efficiently. We also point out similar improvements of orientation quality when our dequantization method is applied to aliased images.

A novel approach for shape preserving contrast enhancement is presented in this paper. Contrast enhancement is achieved by means of a local histogram equalization algorithm which preserves the level-sets of the image. This basic property is violated by common local schemes, thereby introducing spurious objects and modifying the image information. The scheme is based on equalizing the histogram in all the connected components of the image, which are defined based both on the grey-values and spatial relations between pixels in the image, and following mathematical morphology, constitute the basic objects in the scene. We give examples for both grey-value and color images.

©1998 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or distribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE. This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.

We discuss possible algorithms for interpolating data given in a set of curves and/or points in the plane. We propose a set of basic assumptions to be satisfied by the interpolation algorithms which lead to a set of models in terms of possibly degenerate elliptic partial differential equations. The absolute minimal Lipschitz extension model (AMLE) is singled out and studied in more detail. We show experiments suggesting a possible application, the restoration of images with poor dynamic range.

A novel approach for shape preserving contrast enhancement is presented. Contrast enhancement is achieved by means of a local histogram equalization algorithm which preserves the level-sets of the image. This basic property is violated by common local schemes, thereby introducing spurious objects and modifying the image information. The scheme is based on equalizing the histogram in all the connected components of the image, which are defined based on the image grey-values and spatial relations between its pixels. Following mathematical morphology, these constitute the basic objects in the scene. We give examples for both grey-valued and color images.

We discuss possible algorithms for interpolating data given in a set of curves and/or points in the plane. We propose a set of basic assumptions to be satisfied by the interpolation algorithms which lead to a set of models in terms of possibly degenerate elliptic partial differential equations. The absolute minimal Lipschitz extension model (AMLE) is singled out and studied in more detail. We show experiments suggesting a possible application, the restoration of images with poor dynamic range.

We discuss the physical generation process of images as a combination of basic operations: occlusions, transparencies and contrast changes. These operations generate the essential singularities which we call junctions. We deduce a mathematical and computational model for image analysis according to which the "atoms" of the image must be "pieces of level lines joining junctions", fitting the phenomenological description of Gaetano Kanizsa (1990). A parameter free junction detection algorithm is proposed for the computation of the previously defined "atoms". Then we propose an adequate modification of the morphological filtering algorithms so that they smooth the "atoms" without altering the junctions. Finally, we give some experiments on real and synthetic images.

The existence of affine invariant scale spaces for shapes opens possibilities for shape recognition. While affine invariant shape recognition is easily performed when shapes are complete, partially occluded or incomplete shapes must be recognized by dividing them into intrinsic parts. The characteristic point method, for instance, focuses on configurations of points with maximal curvature of the shape (in an euclidian invariant framework). Using the affine invariant scale space, we define affine invariant characteristic points and affine invariant parts of a shape. We prove that compatibility scale relations make feasible the matching of scale spaces and show experiments with noisy affine distorted and occluded shapes.

It is proven that, under four simple axioms, the multiscale analysis of shapes is given by a curvature motion equation. The advantages of such an axiomatic analysis are illustrated in order to discuss the psychophysical theory of early vision of B. Julesz, i.e., the texture preattentive discrimination theory. The result is unexpected. It is proved that the Julesz axiomatic is too good for human vision, and that it leads to a hyperdiscrimination algorithm.>