Point cloud visibility estimation is fundamental as it is useful for many computer vision applications including surface reconstruction, 3D segmentation from paired images and point densification. Previous works showed outstanding results on simple object and outdoor datasets. However, unlike the previously studied scenes, the most challenging environments are those providing a high amount of object points in the same direction, typically in complex indoor scenes. In this kind of environments, due to the lack of real data ground truth, quantitative analysis are either missing or based on simulated data. In this work, we present IRIS-VIS (Industrial Room In Saclay - VISibility), a new dataset for point visibility estimation in an indoor environment. It is a high complexity scene due to the large variety in the shape, size and orientation of the objects. To our know-ledge, this is the first dataset on real indoor data providing a dense LiDAR station-based point cloud along with a well-fitted CAD model. The latter is useful to compute automatically, quickly and accurately the visibility from any given viewpoint, enabling evaluations under infinite conditions. We propose new metrics for the visibility estimation task and evaluate state-of-the-art methods in both sparse and dense conditions with the proposed dataset.

Identifying flood affected areas in remote sensing data is a critical problem in earth observation to analyze flood impact and drive responses. While a number of methods have been proposed in the literature, there are two main limitations in available flood detection datasets: (1) a lack of region variability is commonly observed and/or (2) they require to distinguish permanent water bodies from flooded areas from a single image, which becomes an ill-posed setup. Consequently, we extend the globally diverse MMFlood dataset to multi-date by providing one year of Sentinel-1 observations around each flood event. To our surprise, we notice that the definition of flooded pixels in MMFlood is inconsistent when observing the entire image sequence. Hence, we re-frame the flood detection task as a temporal anomaly detection problem, where anomalous water bodies are segmented from a Sentinel-1 temporal sequence. From this definition, we provide a simple method inspired by the popular video change detector ViBe, results of which quantitatively align with the SAR image time series, providing a reasonable baseline for future works.

We propose a statistically based method to detect hotspots in nighttime short-wave infrared Landsat data. The method assumes an independent and identical Gaussian distribution on the background data and looks for parts of the images with abnormally high values. For this, the variance of the background model is estimated using a robust estimator, being able to provide a good estimation even in the presence of outliers (hotspots). Then, a region growing algorithm is used to extract 4-connected regions with high values. Finally, a statistical test is used to decide whether the sum of the values of each region is significantly higher than expected on the background model. Only regions detected in the two shortwave infrared bands are validated. The test level is selected in order to control the number of false detections. Compared to classical pixel-based methods, our approach allows the detection of hotspots with lower radiance while keeping a low commission error rate. Experiments on a time-series of acquisitions over an oil and gas-producing region showed that this greatly increases the number of detections.

This paper presents a methodology for hotspot detection in multispectral images, utilizing the Reed-Xiaoli anomaly detection algorithm. By leveraging short-wave infrared data from Landsat, the Reed-Xiaoli algorithm identifies hotspots with adaptability to sensors similar to OLI in spectral coverage. The proposed approach is formulated as an a-contrario method, eliminating the need for manually set thresholds for hotspot detection and allows for the control of false detections. The application of this method extends to monitoring the activity status of cement plants, demonstrating robust performance across both daytime and nighttime images. The results show the efficacy of the proposed methodology in hotspot detection for monitoring the activity of industrial facilities such as cement plants.

Video manipulation detection plays a vital role in modern multimedia forensics. In particular, double compression detection provides significant clues leading to the video edition history and hinting at potential malevolent manipulation. While such an analysis is well-understood on images, the research on this subject remains lacking in videos and existing methods are not yet able to reliably detect double-compressed videos. This work presents a novel method for identifying double compression in H.264 codec videos. Our technique exploits the periodicity of frame residuals caused by fixed Group of Pictures in the initial compression, and employs an a contrario framework to minimize and control false detections. The proposed method can reliably detect double compression in videos. It does not require threshold tuning, thus enabling automatic detection. The code is available at https://github.com/li-yanhao/gop_detection.

We propose a statistically based method to detect hot sources in night-time data from the Visible Infrared Imaging Radiometer Suite (VIIRS). This instrument is aboard three satellites and collects nearly global night-time imagery every day. Our method looks for bright pixels in at least two spectrally adjacent bands, and for groups of bright pixels close on the ground. Four near- to short-wave infrared bands are used, as well as a middle-wave infrared band after background subtraction. Thresholds are set automatically using the a-contrario framework. The algorithm has thus only one parameter with a clear signification: the expected number of false detections that can be tolerated in one image. A comparison to detections reported in VIIRS Nightfire shows that the proposed technique enables the detection of some supplementary points with weak signals.

Satellite images are becoming an increasingly important part of our world. Such images are used to forecast the weather, track green house gas emissions, monitor agricultural crop health, and many other applications. Such advances are possible thanks to the free availability of a large number of satellite images. Satellite imagery now plays a key role in many areas, including external security. In this context, it is necessary to question the reliability of this data. Can the authenticity of a satellite image be guaranteed? How can one protect oneself against an entity wishing to hide illegal military material or, conversely, to incite action against another entity by falsely suggesting that it possess such material? If the forensic analysis of photographs has attracted a great deal of academic interest in recent years, this is not yet the case for satellite imagery. In this paper, we propose a methodology to create a very simple but interesting dataset to test the performance of state-of-the-art forensic methods on pristine and manipulated satellite images. Despite the strong performance of such algorithms, satellite images require special attention due to the nature of the images themselves.

Multi-view stereo reconstruction of scenes from satellite images is traditionally performed with a pair-wise stereovision approach: (1) multiple views are grouped into pairs, (2) each pair is processed by two-view stereo methods producing an elevation model or point cloud, lastly (3) the pairwise reconstructions are integrated and filtered to obtain a final result. These steps are organized in a pipeline and the end-to-end performance of reconstructions depends on the behavior of these steps. This work introduces two changes that increase the performance of the reconstructions: a new pair selection approach and a new integration method are presented. The new pair selection replaces commonly used heuristics with a principled criterion that predicts the completeness of a pair based on offline simulations. The presented integration method is based on an iterated bilateral filter. Experiments show that these changes yield a systematic improvement on the performance of the pipeline.

A key limitation of supervised learning is the ability to handle data from unknown distributions. Often, such methods fail when presented with samples from a source not represented in the training data. This work proposes an effective way of controlling the behavior of a neural network in the presence of out-of-distribution examples. For this, the training dataset is supplemented with extraneous data assigned to an additional out-of-distribution class. The extraneous data may come from a different dataset or be even noise. By applying a Gaussian mixture model on the latent representation, and by taking advantage of the ability of these models to generalize well, the method described thereafter performs well. Training the model on a segregated dataset helps the model to distinguish out-of-distribution data, including the ones the model were never confronted to during training.

We propose a block-based signal-dependent noise estimation method on videos, that leverages inter-frame redundancy to separate noise from signal. Block matching is applied to find block pairs between two consecutive frames with similar signal. Then Ponomarenko’s method is extended by sorting pairs by their low-frequency energy and estimating noise in the high frequencies. Experiments on three datasets show that this method improves on the state of the art.

We propose 4Point (Forensics with Positional Internal Training), an unsupervised neural network trained to assess the consistency of the image colour mosaic to find forgeries. Positional learning trains the model to learn the modulo-2 position of pixels, leveraging the translation-invariance of CNN to replicate the underlying mosaic and its potential inconsistencies. Internal learning on a single potentially forged image improves adaption and robustness to varied post-processing and counter-forensics measures. This solution beats existing mosaic detection methods, is more robust to various post-processing and counter-forensic artefacts such as JPEG compression, and can exploit traces to which state-of-the-art generic neural networks are blind. Check qbammey.github.io/4point for the code.

We propose a new method to evaluate image forensics tools, that characterizes what image cues are being used by each detector. Our method enables effortless creation of an arbitrarily large dataset of carefully tampered images in which controlled detection cues are present. Starting with raw images, we alter aspects of the image formation pipeline inside a mask, while leaving the rest of the image intact. This does not change the image’s interpretation; we thus call such alterations "non-semantic", as they yield no semantic inconsistencies. This method avoids the painful and often biased creation of convincing semantics. All aspects of image formation (noise, CFA, compression pattern and quality, etc.) can vary independently in both the authentic and tampered parts of the image. Alteration of a specific cue enables precise evaluation of the many forgery detectors that rely on this cue, and of the sensitivity of more generic forensic tools to each specific trace of forgery, and can be used to guide the combination of different methods. Based on this methodology, we create a database and conduct an evaluation of the main state-of-the-art image forensics tools, where we characterize the performance of each method with respect to each detection cue. Check qbammey.github.io/trace for the database and code.

Different methods can be applied to satellite images to derive an altitude map from a set of images. In this article we evaluate a set of representative methods from different approaches. We consider true multi-view stereo methods as well as pair-wise ones, classic methods and deep learning based ones, methods already in use on satellite images and others that were originally devised for close range imaging and are adapted to satellite imagery. While deep learning (DL) methods have taken over multi-view stereo reconstruction in the last years, this tendency has not fully reached satellite stereo pipelines that still largely rely on pair-wise classic algorithms. For the comparison, we set-up a framework that allows to interface a DL-based stereo method taken from the computer vision literature with a satellite stereo pipeline. For multi-view stereo algorithms we build on a recently proposed framework originally devised to apply Colmap method to satellite images. Methods are compared on several datasets that include sets of images taken within a few days and sets of images taken months apart. Results show that DL methods have, in general, a good generalization power. In particular, the use of the GANet DL method as the matching step in a pair-wise stereo pipeline is promising as it already performs better than the classic counterpart, even without a specific training.

This work explores the connections between the minimum description length (MDL) principle as developed by Rissanen, and the a-contrario framework for structure detection proposed by Desolneux, Moisan, and Morel. The MDL principle focuses on the best interpretation for the whole data while the a-contrario approach concentrates on detecting parts of the data with anomalous statistics. Although framed in different theoretical formalisms, we show that both methodologies share many common concepts and tools in their machinery and yield very similar formulations in a number of interesting scenarios ranging from simple toy examples to practical applications such as polygonal approximation of curves and line segment detection in images. We also formulate the conditions under which both approaches are formally equivalent.

The recent proliferation of constellations of recurrent satellites enables the constitution of temporally dense times series of registered images. We therefore propose in this paper a more in depth detection and analysis of observable changes. This approach is intended to be generic and independent of the type of satellite used, whether band limited or multispectral. It is based on a global analysis of the sequence. The detection stage is based on the definition of a residual sequence calculated from the novelty filter. A statistical approach based on the NFA test is then employed to detect significant changes. We then use these detections to classify the changes according to their nature: unique or lasting. To establish the efficiency of the method, we created an open dataset of 28 sequences of 20 images acquired by Sentinel-2 in different regions of the world. We obtain satisfactory results which are consistent with the visual observations of experts.

ESA's Sentinel- 2 satellites have been in orbit for five years, acquiring huge amounts of data all over the world. They are a formidable tool for mass detection as they are freely available. Given their importance in the energetic transition and their spread over countries or continents, wind turbines are natural candidates for such studies. We propose an automatic wind turbine detector for low resolution satellite images based on an a contrario approach, exploiting the geometry of wind turbines' shadows and hubs. Our experiments show promising detection rates, improving the state of the art in the proposed conditions.

Cloud detection is a crucial step for automatic satellite image analysis. Some cloud detection methods exploit specially designed spectral bands, other base the detection on time series, or on the inter-band delay in push-broom satellites. Nevertheless many use cases occur where these methods do not apply. This paper describes a convolutional neural network for cloud detection in panchromatic and single-frame images. Only a per-image annotation is required, indicating which images contain clouds and which are cloud-free. Our experiments show that, in spite of using less information, the proposed method produces competitive results.

The energy sector is a key industry in the global economy and monitoring oil storage provides valuable insights into the economic state of a country. Our aim is to detect oil tank farms as accurately as possible using Sentinel-2 images. An a contrario clustering method is used to group by density the result of a circle detection step. Then, a patch-match procedure is used to complete the tank detection. Although most existing methods are designed to work on high-resolution images, the proposed method is designed for low-resolution images; we also propose an adaptation to high-resolution images.

Neural network methods are nowadays used for a wide variety of tasks, in particular in computer vision. Among those tasks, semantic segmentation aims at labeling every pixel in an image, giving a good understanding of the scene. In this paper, we propose to use two neural network architectures designed for semantic segmentation to detect oil tank depots in Sentinel-2 images. We compare the methods to an unsupervised algorithm designed to solve the same problem.

Automatically finding suspicious regions in a potentially forged image by splicing, inpainting or copy-move remains a widely open problem. Blind detection neural networks trained on benchmark data are flourishing. Yet, these methods do not provide an explanation of their detections. The more traditional methods try to provide such evidence by pointing out local inconsistencies in the image noise, JPEG compression, chromatic aberration, or in the mosaic. In this paper we develop a blind method that can train directly on unlabelled and potentially forged images to point out local mosaic inconsistencies. To this aim we designed a CNN structure inspired from demosaicing algorithms and directed at classifying image blocks by their position in the image modulo (2 × 2). Creating a diversified benchmark database using varied demosaicing methods, we explore the efficiency of the method and its ability to adapt quickly to any new data.

The classic approach to image matching consists in the detection, description and matching of keypoints. In the description, the local information surrounding the keypoint is encoded. This locality enables affine invariant methods. Indeed, smooth deformations caused by viewpoint changes are well approximated by affine maps. Despite numerous efforts, affine invariant descriptors have remained elusive. This has led to the development of IMAS (Image Matching by Affine Simulation) methods that simulate viewpoint changes to attain the desired invariance. Yet, recent CNN-based methods seem to provide a way to learn affine invariant descriptors. Still, as a first contribution, we show that current CNN-based methods are far from the state-of-the-art performance provided by IMAS. This confirms that there is still room for improvement for learned methods. Second, we show that recent advances in affine patch normalization can be used to create adaptive IMAS methods that select their affine simulations depending on query and target images. The proposed methods are shown to attain a good compromise: on the one hand, they reach the performance of state-of-the-art IMAS methods but are faster; on the other hand, they perform significantly better than non-simulating methods, including recent ones. Source codes are available at https://rdguez-mariano.github.io/pages/adimas.

Monitoring oil stocks provides valuable insights on the balance between production and demand of petroleum products. The identification of oil depots is important for estimating storage capacities and measuring oil stocks. To achieve this purpose, we present an oil tank detector. As oil tanks are generally circular, we use as a first step a circle detection algorithm. However, this approach tends to generate a considerable amount of false detections, as circular shapes are not necessarily tanks. To reduce these false detections, we take advantage of the fact that oil tanks are generally densely grouped together, and filter out isolated detections using a clustering algorithm. The clustering method uses the a contrario framework which gives a way to control the number of false detections per image. The method is illustrated on Sentinel-2 images.

The classic approach to image matching consists in the detection, description and matching of keypoints. This defines a zero-order approximation of the mapping between two images, determined by corresponding point coordinates. But the patches around keypoints typically contain more information, which may be exploited to obtain a first-order approximation of the mapping, incorporating local affine maps between corresponding keypoints. In this work, we propose a LOCal Affine Transform Estimator (LOCATE) method based on neural networks. We show that LOCATE drastically improves the accuracy of local geometry estimation by tracking inverse maps. A second contribution on guided matching and refinement is also presented. The novelty here consists in the use of LOCATE to propose new SIFT-keypoint correspondences with precise locations, orientations and scales. Our experiments show that the precision gain provided by LOCATE does play an important role in applications such as guided matching. The third contribution of this paper consists in a modification to the RANSAC algorithm, that uses LOCATE to improve the homography estimation between a pair of images. These approaches outperform RANSAC for different choices of image descriptors and image datasets, and permit to increase the probability of success in identifying image pairs in challenging matching databases. The source codes are available at: https://rdguez-mariano.github.io/ pages/locate .

The classic approach to image matching consists in the detection, description and matching of keypoints. The descriptor encodes the local information around the keypoint. An advantage of local approaches is that viewpoint deformations are well approximated by affine maps. This motivated the quest for affine invariant local descriptors. Despite numerous efforts, such descriptors remained elusive, ultimately resulting in the compromise of using viewpoint simulations to attain affine invariance. In this work we propose a CNN-based patch descriptor which captures affine invariance without the need for viewpoint simulations. This is achieved by training a neural network to associate similar vectorial representations to patches related by affine transformations. During matching, these vectors are compared very efficiently. The invariance to translation, rotation and scale is still obtained by the first stages of SIFT, which produce the keypoints. The proposed descriptor outperforms the state-of-the-art in retaining affine invariant properties.

This article addresses the problem of estimating scene visibility in time series of satellite images. We are especially focused on satellites with few spectral bands and high revisit frequency. Our approach exploits the redundancy of information acquired during these revisits. It is based on an unsupervised algorithm that tracks local ground textures across time and detects ruptures caused by opaque clouds, haze, cirrus and shadows. Experiments have been carried out on 18 PlanetScope time series covering various locations. These time series come with hand-made labeled ground truth. We compare our results with those of the PlanetScope algorithm and demonstrate the effectiveness of the proposed method : success rates of 94% and 84% are reached for the visible and occulted regions classification.

We focus on the problem of affine invariant image comparison in the presence of noise and repetitive structures. The classic scheme of keypoints, descriptors and matcher is used. A local field of image gradient orientation is used as descriptor and two matchers are proposed, based on the a-contrario theory, for handling repetitive structures. The affine invariance is obtained by affine simulations. The proposed methods achieve state-of-the-art performances under repetitive structures.

We propose a line segment and elliptical arc detector that produces a reduced number of false detections on various types of images without any parameter tuning. For a given region of pixels in a grey-scale image, the detector decides whether a line segment or an elliptical arc is present (model validation). If both interpretations are possible for the same region, the detector chooses the one that best explains the data (model selection ). We describe a statistical criterion based on the a contrario theory, which serves for both validation and model selection. The experimental results highlight the performance of the proposed approach compared to state-of-the-art detectors, when applied on synthetic and real images.

This paper addresses the question of identifying the right camera direct or inverse distortion model, permitting a high subpixel precision to fit to real camera distortion. Five classic camera distortion models are reviewed and their precision is compared for direct or inverse distortion. By definition, the three radially symmetric models can only model a distortion radially symmetric around some distortion center. They can be extended to deal with non-radially symmetric distortions by adding tangential distortion components, but still may be too simple for very accurate modeling of real cameras. The polynomial and the rational models instead miss a physical or optical interpretation, but can cope equally with radially and non-radially symmetric distortions. Indeed, they do not require the evaluation of a distortion center. When requiring high precisions, we found that the distortion modeling must also be evaluated primarily as a numerical problem. Indeed, all models except the polynomial involve a non-linear minimization, which increases the numerical risk. The estimation of a polynomial distortion model leads instead to a linear problem, which is secure and much faster. We concluded by extensive numerical experiments that, although high degree polynomials were required to reach a high precision of 1/100 pixels, such polynomials were easily estimated and produced a precise distortion modeling without overfitting. Our conclusion is validated by three independent experimental setups: the models were compared first on the lens distortion database of the Lensfun library by their distortion simulation and inversion power; second by fitting real camera distortions estimated by a non parametric algorithm; and finally by the absolute correction measurement provided by the photographs of tightly stretched strings, warranting a high straightness.

We describe a simple metric for image patches similarity, together with a robust criterion for unsupervised patch matching. The gradient orientations at corresponding positions in the two patches are compared and the normalized errors are accumulated. Based on the a contrario framework, the matching criterion validates a match between two patches when this cumulative error is too small to have occurred as the result of an accidental agreement. The method is illustrated in the validation of keypoint matches.

In spite of many interesting attempts, the problem of automatically finding alignments in a 2D set of points seems to be still open. The difficulty of the problem is illustrated here by very simple examples. We then propose an elaborate solution. We show that a correct alignment detection depends on not less than four interlaced criteria, namely the amount of masking in texture, the relative bilateral local density of the alignment, its internal regularity, and finally a redundancy reduction step. Extending tools of the a contrario detection theory, we show that all of these detection criteria can be naturally embedded in a single probabilistic a contrario model with a single user parameter, the number of false alarms. Our contribution to the a contrario theory is the use of sophisticated conditional events on random point sets, for which expectation we nevertheless find easy bounds. By these bounds the mathematical consistency of our detection model receives a simple proof. Our final algorithm also includes a new formulation of the exclusion principle in Gestalt theory to avoid redundant detections. Aiming at reproducibility, a source code and an online demo open to any data point set are provided. The method is carefully compared to three state-of-the-art algorithms and an application to real data is discussed. Limitations of the final method are also illustrated and explained.

We present a novel method for automatic vanishing point detection based on primal and dual point alignment detection. The very same point alignment detection algorithm is used twice: First in the image domain to group line segment endpoints into more precise lines. Second, it is used in the dual domain where converging lines become aligned points. The use of the recently introduced PClines dual spaces and a robust point alignment detector leads to a very accurate algorithm. Experimental results on two public standard datasets show that our method significantly advances the state-of-the-art in the Manhattan world scenario, while producing state-of-the-art performances in non-Manhattan scenes.

The a contrario theory is a mathematical formalization of the so-called non-accidentalness principle of perception, which states that an observed configuration is relevant only when it is unlikely to appear just by chance. It has been successfully applied to several image processing and computer vision problems. In this paper, human vision is compared to an a contrario based algorithm in a simple perceptual task. For this aim, a psychophysical experiment was set up, in which subjects and the algorithm were asked to detect alignments of Gabor patches. We found that the proposed algorithm predicted accurately the subjects' responses, therefore providing an interpretation to perceptual thresholds.

We will consider the problem of detecting configurations of points regularly spaced and lying on a smooth curve. This corresponds to the notion of good continuation introduced in the Gestalt theory. We present a robust algorithm for clustering points along such curves, whilst at the same time discarding noisy samples. Based on the a contrario methodology, the detector builds upon a simple, symmetric primitive for a triplet of points, and finds statistically meaningful chains of such triplets. An efficient implementation is proposed using the Floyd-Warshall algorithm. Experiments on synthetic and real data show that the method is able to identify the perceptually relevant configuration of points in good continuation.

Plumb line lens distortion correction methods permit to avoid numerical compensation between the camera internal and external parameters in global calibration method. Once the distortion has been corrected by a plumb line method, the cam era is ensured to transform, up to the distortion precision, 3D straight lines into 2D straight lines, and therefore becomes a pinhole camera. This paper introduces a plumb line method for correcting and evaluating camera lens distortion with high precision. The evaluation criterion is defined as the average standard deviation from straightness of a set of approximately equally spaced straight strings photographed uniformly in all directions by the camera, so that their image crosses the whole camera field. The method uses an easily built "calibration harp," namely a frame on which good quality strings have been tightly stretched to ensure a very high physical straight ness. Real experiments confirm that our method produces high precision corrections (less than 0.05 pixel), approximating the distortion with a large number of degrees of freedom given by a polynomial model of order eleven.

We propose a new method for fast detection of planar patches in disparity maps. We first use a region growing algorithm on random seeds. This approach is similar to the one introduced in [1] for fast line segment detection in images. Then, the parameter-free criterion introduced in [2] is used to keep only the patches that are planar. The main advantage of our method is to be able to estimate the disparity map precision which is usually a critical parameter in other methods. This method is specially well suited to 3D reconstruction of urban environments from low-baseline aerial or satellite stereo pairs where a piecewise-planar model can be applied.

This paper points out and attempts to remedy a serious discrepancy in results obtained by global calibration methods: The re-projection error can be rendered very small by these methods, but we show that the optical distortion correction is far less accurate. This discrepancy can only be explained by internal error compensations in the global methods that leave undetected the inadequacy of the distortion model. This fact led us to design a model-free distortion correction method where the distortion can be any image domain diffeomorphism. The obtained precision compares favorably to the distortion given by state of the art global calibration and reaches a RMSE of 0.08 pixels. Nonetheless, we also show that this accuracy can still be improved.

We propose a linear-time line segment detector that gives accurate results, a controlled number of false detections, and requires no parameter tuning. This algorithm is tested and compared to state-of-the-art algorithms on a wide set of natural images.

In this paper we propose a new method for detecting straight line segments in digital images. It improves upon existing methods by giving precise results while controlling the number of false detections and can be applied to any digital image without parameter setting. The method is a nontrivial extension of the approach presented by Desolneux et al. (2000). The core of the method is an algorithm to cut a binary sequences into what we call a multisegment: a set of collinear and disjoint segments. We shall define a functional that measures the so called meaningfulness of a multisegment. This functional allows us to validate detections against an a contrario non-structured model and to select the best ones. The result is a global interpretation, line by line, of the image in terms of straight segments which gives back its geometry with high accuracy. Comparisons with state of the art methods are presented (more examples are available on-line).