Recently, Gaussian splatting has emerged as a strong alternative to NeRF, demonstrating impressive 3D modeling capabilities while requiring only a fraction of the training and rendering time. In this paper, we show how the standard Gaussian splatting framework can be adapted for remote sensing, retaining its high efficiency. This enables us to achieve state-of-the-art performance in just a few minutes, compared to the day-long optimization required by the best-performing NeRF-based Earth observation methods. The proposed framework incorporates remote-sensing improvements from EO-NeRF, such as radiometric correction and shadow modeling, while introducing novel components, including sparsity, view consistency, and opacity regularizations.

Point cloud visibility estimation is fundamental as it is useful for many computer vision applications including surface reconstruction, 3D segmentation from paired images and point densification. Previous works showed outstanding results on simple object and outdoor datasets. However, unlike the previously studied scenes, the most challenging environments are those providing a high amount of object points in the same direction, typically in complex indoor scenes. In this kind of environments, due to the lack of real data ground truth, quantitative analysis are either missing or based on simulated data. In this work, we present IRIS-VIS (Industrial Room In Saclay - VISibility), a new dataset for point visibility estimation in an indoor environment. It is a high complexity scene due to the large variety in the shape, size and orientation of the objects. To our know-ledge, this is the first dataset on real indoor data providing a dense LiDAR station-based point cloud along with a well-fitted CAD model. The latter is useful to compute automatically, quickly and accurately the visibility from any given viewpoint, enabling evaluations under infinite conditions. We propose new metrics for the visibility estimation task and evaluate state-of-the-art methods in both sparse and dense conditions with the proposed dataset.

Abstract. Automatic anomaly detection (AD) in a series of images of industrial parts is a key component of industrial production and an exemplary problem for machine learning. Since it can only realistically function with minimal supervision, unsupervised methods dominate the field. Their principle is that the “normal aspect” of objects is learned from recently observed samples, so that anomalies can be detected as outliers. In this paper, we start by reviewing recent AD methods and their performance-based ranking on recent benchmark datasets. The recent progress of such methods is such that they learn from a few hundred normal samples only. However, we argue that the current method evaluation based on static datasets is limited and biased. Indeed, a main feature of industrial production is that the aspect of objects evolves over time, due to changes in production and acquisition conditions, thus leading to significant probability distribution shifts. By introducing artificial but realistic deviations into the classic MVTec benchmark we show that the smallest deviation is sufficient to make these stationary models collapse. We argue that some of these models, especially the stochastic ones, can be easily adapted to cope with distribution shifts. The Global-to-Local Anomaly Detector (GLAD) is such an example of a method that uses Gaussian Mixture Models to model the distribution of regular objects. Using the stochastic approximation of expectation maximization, we design Online-GLAD, an improved GLAD that can update and adapt online. In the experiments, we show that Online-GLAD is able to maintain good performance even in the presence of multiple progressive deviations, and with constant complexity compatible with real-time implementation.

Identifying flood affected areas in remote sensing data is a critical problem in earth observation to analyze flood impact and drive responses. While a number of methods have been proposed in the literature, there are two main limitations in available flood detection datasets: (1) a lack of region variability is commonly observed and/or (2) they require to distinguish permanent water bodies from flooded areas from a single image, which becomes an ill-posed setup. Consequently, we extend the globally diverse MMFlood dataset to multi-date by providing one year of Sentinel-1 observations around each flood event. To our surprise, we notice that the definition of flooded pixels in MMFlood is inconsistent when observing the entire image sequence. Hence, we re-frame the flood detection task as a temporal anomaly detection problem, where anomalous water bodies are segmented from a Sentinel-1 temporal sequence. From this definition, we provide a simple method inspired by the popular video change detector ViBe, results of which quantitatively align with the SAR image time series, providing a reasonable baseline for future works.

Satellite imaging has proven to be crucial to monitor methane emissions and help reduce them. In this paper, we propose an automatic practical methodology to use time series from geostationary satellites like GOES-16. While these satellites offer a poor spatial and spectral resolution, their revisit time is unmatched: GOES-16 delivers an image every five minutes for the CONUS region. The proposed approach takes advantage of this fast revisit time to monitor the evolution of large methane emissions. We show the performance on an emission in Mexico and another in the US. This is the first step toward a real time monitoring of methane emissions using geostationary satellite imagery.

Reducing methane emissions is essential to tackle climate change. Here, we address the problem of detecting automatically point source methane leaks using high resolution hyperspectral images from the PRISMA satellite. We propose an improvement of the classical matched filter method by using an adjustment coefficient. We introduce this new method under the name: Model Adjusted Matched Filter (MAMF). We show that the MAMF method reduces the fraction of false detections compared to the Matched Filter (MF) and the Adaptive Cosine Estimator (ACE) without preventing the detection of plumes. To validate the method, we use a dataset of manually annotated plumes on PRISMA images. We then show that our method outperforms the matched filter and the adaptive cosine estimator in terms of F1 score.

The use of NeRF to model 3D scenes from satellite images is becoming increasingly common. However, the models proposed to date assume the availability of pre-processed RGB images as input. This contrasts with the multispectral nature of raw satellite products. Optical satellite sensors do not acquire RGB images but a wider variety of spectral bands, which may have different spatial resolution. We propose a NeRF framework to simultaneously handle panchromatic data and lower resolution spectral bands (e.g., color bands), and investigate the contribution of the low-resolution bands to the output model. Our method achieves comparable or better results with respect to previous approaches that rely on a separate pansharpening step. The model can also be used to generate a pansharpened image surrogate for each input view, as it natively performs super-resolution in the color bands.

Neural radiance fields, or NeRF, represent a breakthrough in the field of novel view synthesis and 3D modeling of complex scenes from multi-view image collections. Numerous recent works have shown the importance of making NeRF models more robust, by means of regularization, in order to train with possibly inconsistent and/or very sparse data. In this work, we explore how differential geometry can provide elegant regularization tools for robustly training NeRF-like models, which are modified so as to represent continuous and infinitely differentiable functions. In particular, we present a generic framework for regularizing different types of NeRFs observations to improve the performance in challenging conditions. We also show how the same formalism can also be used to natively encourage the regularity of surfaces by means of Gaussian or mean curvatures.

Sparse keypoints based methods allow to match two images in an efficient manner. However, even though they are sparse, not all generated keypoints are necessary. This uselessly increases the computational cost during the matching step and can even add uncertainty when these keypoints are not discriminatory enough, thus leading to imprecise, or even wrong, alignment. In this paper, we address the important case where the alignment deals with the same scene or the same type of object. This enables a preliminary learning of optimal keypoints, in terms of efficiency and robustness. Our fully unsupervised selection method is based on a statistical a contrario test on a small set of training images to build without any supervision a dictionary of the most relevant points for the alignment. We show the usefulness of the proposed method on two applications, the stabilization of video surveillance sequences and the fast alignment of industrial objects containing repeated patterns. Our experiments demonstrate an acceleration of the method by 20 factor and significant accuracy gain.

Reducing methane emissions is essential to tackle climate change. Here, we address the problem of detecting automatically point source methane leaks using high resolution hyperspectral images from the PRISMA satellite. We use a variation of the Matched Filter (MF) called the Adjusted Spectral Matched Filter (ASMF) to detect methane plumes in satellite images. To remove false positives, the detected plumes are confirmed by comparing their orientation to the wind direction extracted from the standard meteorological reanalysis product ERA5. The ASMF reduces the fraction of false detections compared to the MF and without preventing the detection of plumes. To validate the method, we use a recently proposed dataset of manually annotated plumes on PRISMA images. We also compare our detection rate to the detection rate of methods using deep learning or the standard matched filter. We then show that our method outperforms those methods in terms of F1 score.

Learning to detect automatic anomalies in production plants remains a machine learning challenge. Since anomalies by definition cannot be learned, their detection must rely on a very accurate "normality model". To this aim, we introduce here a global-to-local Gaussian model for neural network features, learned from a set of normal images. This probabilistic model enables unsupervised anomaly detection. A global Gaussian mixture model of the features is first learned using all available features from normal data. This global Gaussian mixture model is then localized by an adaptation of the K-MLE algorithm, which learns a spatial weight map for each Gaussian. These weights are then used instead of the mixture weights to detect anomalies. This method enables precise modeling of complex data, even with limited data. Applied on WideResnet50-2 features, our approach outperforms the previous state of the art on the MVTec dataset, particularly on the object category. It is robust to perturbations that are frequent in production lines, such as imperfect alignment, and is on par in terms of memory and computation time with the previous state of the art.

Denoising and demosaicing are two critical components of the image/video processing pipeline. While historically these two tasks have mainly been considered separately, current neural network approaches allow to obtain state-of-the-art results by treating them jointly. However, most existing research focuses in single image or burst joint denoising and demosaicing (JDD). Although related to burst JDD, video JDD deserves its own treatment. In this work we present an empirical exploration of different design aspects of video joint denoising and demosaicing using neural networks. We compare recurrent and non-recurrent approaches and explore aspects such as type of propagated information in recurrent networks, motion compensation, video stabilization, and network architecture. We found that recurrent networks with motion compensation achieve best results. Our work should serve as a strong baseline for future research in video JDD.

Methane emissions monitoring is essential to control methane pollution. In this paper, we propose an automatic practical methodology using time series to estimate the quantity of methane in a given plume using a multispectral satellite like Sentinel-2. Sentinel-2 proposes a low revisit time, a good spatial resolution and a low acquisition cost. Contrary to previous methods, the proposed approach does not require a manual selection of an optimal reference image. We compared its performance on an oil-and-gas site in Kazakhstan. This is the first step toward an automatic global monitoring system for methane plume detection and quantification with these satellites.

Anomaly detectors are necessary to automatize industrial quality control. However, crafting such detectors is difficult due to the complexity and variability of the object even when working only with rigid objects. We show that adding a deep learning normalization step as a preprocessing step to model based detectors allows for better and more robust detections. This self-supervised normalization neural network is trained on non-anomalous data only. The proposed preprocessing method, followed by an automatic detector, achieves state-of-the-art results on rigid objects from the MvTec dataset.

Recent constellations of satellites, including the Skysat constellation, are able to acquire burst of images. This new acquisition mode allows for modern image restoration techniques, including multi-frame super-resolution. As the satellite moves during the acquisition of the burst, elevation changes in the scene translate into noticeable parallax. This parallax hinders the results of the restoration. To cope with this issue, we propose a novel parallax estimation method. The method is composed of a linear$\text{Plane}+\text{Parallax}$decomposition of the apparent motion and a multi-frame optical flow algorithm that exploits all frames simultaneously. Using SkySat L1A images, we show that the estimated per-pixel displacements are important for applying multi-frame super-resolution on scenes containing elevation changes and that can also be used to estimate a coarse 3D surface model.

Global water storage monitoring is often done using remote sensing to solve the problem of missing gauge station. Moreover, this information is very often not available to the public even when a measure station is available. The methods usually used are however often unpractical for small lakes, often require images that are costly or difficult to acquire (for example SAR images of sufficient resolution) or with a bad revisit time. In this paper we propose an automatic method for visible satellite images based on a precise tracking of shoreline. It uses bathymetry information of the lake when available but we also present different alternatives that still produce precise estimations. The method is applied to accurately track the water volume of lake of La Bultière in 2019 using only PlanetScope images.

Reducing methane emissions is essential to tackle climate change. Here, we address the problem of detecting large methane leaks by using hyperspectral data from the satellite Sentinel-5P. By sampling Sentinel-5P spectral data at fine scale, we detect methane absorption features in the shortwave infrared wavelength range (SWIR). Our method involves two separate steps: i) background subtraction and ii) detection of local maxima in the negative logarithmic spectrum of each pixel. In the first step, we remove the impact of the albedo using albedo maps and the impact of the atmosphere by using a principal component analysis (PCA) over a time series of past observations. In the second step, we count for each pixel the number of local maxima that correspond to a subset of local maxima in the methane absorption spectrum. This counting method allows us to set up a statistical a contrario test that controls the false alarm rate of our detections.

We propose a self-supervised approach for training multi-frame video denoising networks. These networks predict each frame from a stack of frames around it. Our self-supervised approach benefits from the temporal consistency in the video by minimizing a loss that penalizes the difference between the predicted frame and a neighboring one, after aligning them using an optical flow. We use the proposed strategy to denoise a video contaminated with an unknown noise type, by fine-tuning a pre-trained denoising network on the noisy video. The proposed fine-tuning reaches and sometimes surpasses the performance of state-of-the-art networks trained with supervision. We demonstrate this by showing extensive results on video blind denoising of different synthetic and real noises. In addition, the proposed fine-tuning can be applied to any parameter that controls the denoising performance of the network. We show how this can be expoited to perform joint denoising and noise level estimation for heteroscedastic noise.

Modeling the processing chain that has produced a video is a difficult reverse engineering task, even when the camera is available. This makes model based video processing a still more complex task. In this paper we propose a fully blind video denoising method, with two versions off-line and on-line. This is achieved by fine-tuning a pre-trained AWGN denoising network to the video with a novel frame-to-frame training strategy. Our denoiser can be used without knowledge of the origin of the video or burst and the post-processing steps applied from the camera sensor. The on-line process only requires a couple of frames before achieving visually pleasing results for a wide range of perturbations. It nonetheless reaches state-of-the-art performance for standard Gaussian noise, and can be used off-line with still better performance.

Demosaicking and denoising are the first steps of any camera image processing pipeline and are key for obtaining high quality RGB images. A promising current research trend aims at solving these two problems jointly using convolutional neural networks. Due to the unavailability of ground truth data these networks cannot be currently trained using real RAW images. Instead, they resort to simulated data. In this paper we present a method to learn demosaicking directly from mosaicked images, without requiring ground truth RGB data. We apply this to learn joint demosaicking and denoising only from RAW images, thus enabling the use of real data. In addition we show that for this application fine-tuning a network to a specific burst improves the quality of restoration for both demosaicking and denoising.

Non-local patch-based methods were until recently state-of-the-art for image denoising but are now outperformed by convolutional neural networks (CNNs). Yet they are still the best ones for video denoising, as video redundancy is a key factor to attain high denoising performance. In this work we propose a novel video denoising CNN. Non-local self-similarity is incorporated into the network via a first non-trainable layer which finds for each patch in the input image its most similar patches in a 3D spatio-temporal search region centered at the target patch. The central values of these patches are then gathered in a feature vector which is assigned to each image pixel. This information is presented to a CNN which is trained to predict a clean image. The proposed architecture achieves state-of-the-art results. To the best of our knowledge, this is the first successful application of CNNs to video denoising.

Many problems in image/video processing and computer vision require the computation of a dense k-nearest neighbor field (k-NNF) between two images. For each patch in a query image, the k-NNF determines the positions of the k most similar patches in a database image. With the introduction of the PatchMatch algorithm, Barnes et al. demonstrated that this large search problem can be approximated efficiently by collaborative search methods that exploit the local coherency of image patches. After its introduction, several variants of the original PatchMatch algorithm have been proposed, some of them reducing the computational time by two orders of magnitude. In this work we study the convergence of PatchMatch and its variants, and derive bounds on their convergence rate. We consider a generic PatchMatch algorithm from which most specific instances found in the literature can be derived as particular cases. We also derive more specific bounds for two of these particular cases: the original PatchMatch and Coherency Sensitive Hashing. The proposed bounds are validated by contrasting them to the convergence observed in practice.

Anomaly detectors address the difficult problem of detecting automatically exceptions in an arbitrary background image. Detection methods have been proposed by the thousands because each problem requires a different background model. By analyzing the existing approaches, we show that the problem can be reduced to detecting anomalies in residual images (extracted from the target image) in which noise and anomalies prevail. Hence, the general and impossible background modeling problem is replaced by simpler noise modeling, and allows the calculation of rigorous thresholds based on the a contrario detection theory. Our approach is therefore unsupervised and works on arbitrary images.

In this article we propose a new recursive video denoising method with high performance. The method is recursive and uses only the current frame and the previous denoised one. It considers the video as a set of overlapping temporal patch trajectories. Following a Bayesian approach each trajectory is modeled as linear dynamic Gaussian model and denoised by a Kalman filter. To estimate its parameters, similar patches are grouped and their trajectories are considered as sharing the same model parameters. The filtering is mainly temporal; non-local spatial similarity is only used to estimate the parameters. This temporally causal method obtains results comparable (in terms of PSNR and SSIM) to state-of-the-art methods using several frames per frame denoised, but with a higher temporal consistency.