This work introduces the MUlti-modal MUlti-class Change Detection (MUMUCD) dataset, which comprises 70 globally distributed georeferenced bitemporal pairs obtained by multiple space-borne sensors. Acquisitions over heterogenous terrains (e.g., urban, rural, forests, deserts) for the period 2019-2024 are processed to create the first large scale curated dataset combining Synthetic Aperture Radar (Sentinel-1), multispectral (Sentinel-2) and hyperspectral (PRISMA) data with ancillary information. Provided with a resampled image resolution of 10 m and a size of 1536×1536 square pixels, MUMUCD allows to extract several thousands non-overlapping patches with size 128×128 square pixels, enabling data-intensive machine learning (ML) applications. Seasonality plays a critical role in the analysis of environmental data, so we carefully selected scenes representing all times of the year and a variety of geographical contexts relevant to key impact sectors, taking also into account the availability of PRISMA acquisitions. While scenes with low cloudiness are prioritized, cloudy pixels are not excluded as different data combinations (e.g., SAR/optical) can be exploited to mitigate the atmospheric effects. Beyond coregistered data, we include surface elevation, land cover and binary change maps obtained by processing the Dynamic World dataset, based on the provided multispectral data. Benchmarking of machine and deep learning algorithms indicates that the provided labels should be augmented to get the most out of the multiple modalities. MUMUCD is meant to fill a research gap by offering multi-sensor and task-oriented data, which make it ideal to fine-tune standard and foundational ML models for a variety of tasks, and even unique when these tasks involve change detection or hyperspectral measurements.

A significant amount of remotely sensed data is generated daily by many Earth observation (EO) spaceborne and airborne sensors over different countries of our planet. Different applications use those data, such as natural hazard monitoring, global climate change, urban planning, and more. Many challenges are brought by the use of these big data in the context of remote sensing (RS) applications. In recent years, the employment of machine learning (ML) and deep learning (DL)-based algorithms has allowed a more efficient use of these data, but the issues in managing, processing, and efficiently exploiting them have even increased as classical computers have reached their limits. This article highlights a significant shift toward leveraging quantum computing (QC) techniques in processing large volumes of RS data. The proposed Quanv4EO framework introduces a quanvolution method for (pre)processing multidimensional EO data. Its effectiveness was first demonstrated on standard image classification datasets (MNIST and FashionMNIST), achieving accuracies of 99.84% and 96.81%, respectively, with a significantly reduced model size of 42 k parameters and 16 frozen qubits. Its capabilities were then checked on EO datasets, such as EuroSAT, with a mean accuracy of 96% using balanced iterative reducing and clustering using hierarchies (BIRCHs) clustering and 93% using automated DL (AutoDL), surpassing or matching state-of-the-art (SOTA) classical nonquantum models. Applying the framework to synthetic aperture radar (SAR) data, the QSPeckleFilter demonstrates notable improvements in speckle noise reduction, achieving a peak signal-to-noise ratio (PSNR) of 21.72 and a structural similarity index measure (SSIM) of 0.81, surpassing all tested classical counterparts. The proposed results underscore the potential of quantum-enhanced approaches in RS data analysis, paving the way for more efficient and effective solutions for wide geographical area EO data exploitation.

The trend of building larger and more complex imaging satellite constellations leads to the challenge in managing multiple acquisition requests of the Earth surface. Optimally planning these acquisitions is an intractable optimization problem, and heuristic algorithms are used today for finding sub-optimal solutions. Recently, quantum algorithms have been considered for this purpose, due to the potential breakthroughs that they can bring in optimization, expecting either a speedup or an increase in the solution quality. Hybrid quantum-classical methods have been considered as a short-term solution for taking advantage of small quantum machines. In this paper, we propose reverse quantum annealing as a method for improving the acquisition plan obtained by a classical optimizer. We investigate the benefits of the method with different annealing schedules and different problem sizes. The obtained results provide guidelines on designing a larger hybrid quantum-classical framework based on reverse quantum annealing for this application.

Data from satellites or aerial vehicles are most of the times unlabelled. Annotating such data accurately is difficult, requires expertise, and is costly in terms of time. Even if Earth Observation (EO) data were correctly labelled, labels might change over time. Learning from unlabelled data within a semi-supervised learning framework for segmentation of aerial images is challenging. In this paper, we develop a new model for semantic segmentation of unlabelled images, the Non-annotated Earth Observation Semantic Segmentation (NEOS) model. NEOS performs domain adaptation as the target domain does not have ground truth masks. The distribution inconsistencies between the target and source domains are due to differences in acquisition scenes, environment conditions, sensors, and times. Our model aligns the learned representations of the different domains to make them coincide. The evaluation results show that it is successful and outperforms other models for semantic segmentation of unlabelled data.

Massive amounts of unlabelled data are captured by Earth Observation (EO) satellites, with the Sentinel-2 constellation generating 1.6TB of data daily. This makes Remote Sensing a data-rich domain well suited to Machine Learning (ML) solutions. However, a bottleneck in applying ML models to EO is the lack of annotated data as annotation is a labour-intensive and costly process. As a result, research in this domain has focused on Self-Supervised Learning and Foundation Model approaches. This paper addresses the need to evaluate different Foundation Models on a fair and uniform benchmark by introducing the PhilEO Bench, a novel evaluation framework for EO Foundation Models. The framework comprises of a testbed and a novel 400GB Sentinel-2 dataset containing labels for three downstream tasks, building density estimation, road segmentation, and land cover classification. We present experiments using our framework evaluating different Foundation Models, including Prithvi and SatMAE, at multiple n-shots and convergence rates.

Recent advances in remote sensing and artificial intelligence offer exciting opportunities in an array of fields, with precision agriculture being a notable use case. Here, estimating soil parameters from remotely-sensed hyperspectral imagery at a global scale can play a pivotal role in day-to-day operations, as it may help optimize agricultural management processes, hence positively affecting our planet. In this paper, we tackled the problem of estimating soil parameters from hyperspectral images and introduced heterogeneous regression ensembles for this task. They not only benefit from both classic and deep machine learning models but were also thoroughly investigated and fine-tuned in our rigorous experimental study, performed over a well-established HYPERVIEW benchmark dataset. The experiments showed that such heterogeneous ensembles outperform other techniques and offer a high level of model flexibility.

Current technological advancements bring exciting opportunities in hyperspectral image analysis across various Earth observation applications, with precision agriculture being one of their notable examples. However, gathering high-quality and representative ground-truth data for training supervised machine learners is extremely challenging in emerging use cases, as it is cost-inefficient and user-dependent. Thus, building (deep) machine learning models from very few training samples is of paramount practical importance. To tackle this issue, we propose a semi-supervised learning pipeline for elaborating deep learning models for multi-class hyperspectral image classification, while benefiting from the available unlabeled samples, leveraging pseudo-labeling and consistency regularization. Although our technique is model-agnostic, we exploit a deep residual network for classifying hyperspectral images according to the magnesium content in the imaged soil. The experimental study performed over a real-world dataset (HYPERVIEW) encompassing environmental tasks crucial to food sustainability showed that the proposed technique significantly outperforms the pre-trained models fine-tuned in a fully supervised way.

This paper explores an innovative fusion of Quantum Computing (QC) and Artificial Intelligence (AI) through the development of a Hybrid Quantum Graph Convolutional Neural Network (HQGCNN), combining a Graph Convolutional Neural Network (GCNN) with a Quantum Multilayer Perceptron (MLP). The study highlights the potentialities of GCNNs in handling global-scale dependencies and proposes the HQGCNN for predicting complex phenomena such as the Oceanic Niño Index (ONI). Preliminary results suggest the model potential to surpass state-of-the-art (SOTA). The code will be made available with the paper publication.

We investigate the runtime of quantum kernel estimation in the view of quantum kernel concentration effect. The study is performed for projected quantum kernel family evaluated on hyperspectral remote sensing data. The effect of exponential value concentration leads to the indistinguishability of the kernel matrix entries as the size of the quantum device grows. In order to prevent that, kernel values have to be estimated with a better precision. Increasing precision inevitably connects to an increasing number of circuit runs, which influences the runtime of quantum algorithm. This, in turn, frequently obstructs a possible advantage for quantum machine learning methods. We find that, against popular opinions, the effect of exponential value concentration does not rule out the utility of quantum kernel methods and the severity of the issue depends highly on the data used.

Nowadays the accurate geo-localization of ground-view images has an important role across domains as diverse as journalism, forensics analysis, transports, and Earth Observation. This work addresses the problem of matching a query ground-view image with the corresponding satellite image without GPS data. This is done by comparing the features from a ground-view image and a satellite one, innovatively leveraging the corresponding latter’s segmentation mask through a three-stream Siamese-like network. The proposed method, Semantic Align Net (SAN), focuses on limited Field-of-View (FoV) and ground panorama images (images with a FoV of 360°). The novelty lies in the fusion of satellite images in combination with their segmentation masks, aimed at ensuring that the model can extract useful features and focus on the significant parts of the images. This work shows how SAN through semantic analysis of images improves the performance on the unlabelled CVUSA dataset for all the tested FoVs.

Satellite imaging brings exciting opportunities in an array of fields, with precision agriculture being a notable example. Soil analysis at scale with the use of Earth observation satellites coupled with on-board and on-the-ground artificial intelligence algorithms offers actionable items that may be exploited by practitioners to optimize their operations, including the fertilization process. Here, bare soil detection is a pivotal step in the processing chain to limit the detailed analysis to the areas of interest. In this paper, we tackle this task with quantum-kernel support vector machines and verify the utility of quantum machine learning in practical Earth observation. Our experimental study, performed over a real-world hyperspectral scene, indicates that the proposed quantum-kernel models are competitive with well-established classical support vector machines, as well as with approaches based on thresholding spectral indices that are widely exploited in the field.

The amount of data captured in the emerging satellite missions has been continuously growing. Thus, developing resource-efficient predictive models is of paramount importance in an array of onboard space applications, where downlinking the data for further analysis is extremely costly or impossible. In such scenarios, we should extract actionable items on board an edge device, using e.g., a machine learning model. Reducing the model’s complexity which may be significant in deep learning algorithms is not only about fitting a full-size neural net into resource-restrictive hardware, but it may result in decreasing the latency and energy consumption. We tackle this issue and exploit knowledge distillation to elaborate a simpler and memory-efficient version of the large-capacity learner, aiming to preserve the large model quality. The experimental study shows that knowledge distillation may not only improve the classification capability of the original model, but can also dramatically (up to 425×) reduce its size for the on-board classification of Mars imagery.

The optimization of Kernel-Target Alignment (TA) has been recently proposed as a way to reduce the number of hardware resources in quantum classifiers. It allows to exchange highly expressive and costly circuits to moderate size, task oriented ones. In this work we propose a simple toy model to study the optimization landscape of the Kernel-Target Alignment. We find that for underparameterized circuits the optimization landscape possess either many local extrema or becomes flat with narrow global extremum. We find the dependence of the width of the global extremum peak on the amount of data introduced to the model. The experimental study was performed using multispectral satellite data, and we targeted the cloud detection task, being one of the most fundamental and important image analysis tasks in remote sensing.

sifier effectively deployed in an array of pattern recognition and classification tasks. In this work, we consider extending classic SVMs with quantum kernels and applying them to satellite data analysis. The design and implementation of SVMs with quantum kernels (hybrid SVMs) is presented. It consists of the Quantum Kernel Estimation (QKE) procedure combined with a classic SVM training routine. The pixel data are mapped to the Hilbert space using ZZ-feature maps acting on the parameterized ansatz state. The parameters are optimized to maximize the kernel target alignment. We approach the problem of cloud detection in satellite image data, which is one of the pivotal steps in both on-the-ground and on-board satellite image analysis processing chains. The experiments performed over the benchmark Landsat-8 multi-spectral dataset revealed that the simulated hybrid SVM successfully classifies satellite images with accuracy on par with classic SVMs.

Hyperspectral unmixing is one of the most challenging tasks in the analysis of such data. There have been an array of algorithms proposed for this problem so far, but they are virtually always verified using random sampling, where training and test examples are drawn from the same image. Since such samples are spatially correlated and may be positioned close to each other, random sampling can induce the training-test information leak in the techniques that exploit spatial information during the unmixing process. We want to raise the attention of the community about this validation flaw in the context hyperspectral unmixing. We introduce the algorithm for unbiased validation of the unmixing techniques through splitting hyperspectral images into training and test samples that do not suffer from the training-test information leak. The experiments showed that the widely-used random sampling verification strategy leads to overly optimistic conclusions concerning the algorithm’s performance. This problem was mitigated with the proposed approach which allows us to rigorously validate unmixing techniques.

Improving agricultural practices through exploiting the recent imaging and machine learning advancements plays a key role nowadays to ensure sustainable food security, and to help us deal with the climate change. Quantifying soil parameters can lead to optimizing the fertilization process but it is cumbersome, time-consuming and difficult to scale, as it requires performing in-situ soil measurements that are later analyzed in the laboratory settings. In the HYPER-VIEW challenge, we aim at automating the soil analysis thanks to the utilization of hyperspectral images that capture very detailed information about the scanned objects in hundreds of contiguous hyperspectral bands. Such imagery can be effectively analyzed using an array of classical and deep machine learning approaches. Also, the AI techniques can be deployed on-board the imaging satellites— it opens new doors related to the scalability of the solution. The winners of the challenge will be offered a unique opportunity to run their proposed solution in orbit, on-board the Intuition-1 satellite, equipped with a hyperspectral imager and on-board AI capabilities.

The amount of study on Quantum Machine Learning (QML) is increasing extensively due to its potential advantages in terms of representational power and computational resources. These advances suggest a possibility to extend its usage into the context of Earth Observations, where Machine Learning (ML) plays an important role due to its extensive amount of data to be manipulated. This paper presents our preliminary results of binary quantum classifiers, which consist of Quantum Convolutional Neural Networks (QCNNs), applied on Earth Observation datasets, EuroSAT and SAT4, with classically-reduced features. Especially, we compare the performance of different data embedding techniques and quantum circuits for binary classification tasks.

Remote sensing visual question answering (RSVQA) opens new avenues to promote the use of satellites data, by interfacing satellite image analysis with natural language processing. Capitalizing on the remarkable advances in natural language processing and computer vision, RSVQA aims at finding an answer to a question formulated by a human user about a remote sensing image. This is achieved by extracting representations from images and questions, and then fusing them in a joint representation. Focusing on the language part of the architecture, this study compares and evaluates the adequacy to the RSVQA task of two language models, a traditional recurrent neural network (Skip-thoughts) and a recent attentionbased Transformer (BERT). We study whether large transformer models are beneficial to the task and whether fine-tuning is needed for these models to perform at their best. Our findings show that the models benefit from fine-tuning language models and that RSVQA with BERT is slightly but consistently better when properly fine-tuned.

Transfer learning is a powerful way to adapt existing deep learning models to new emerging use-cases in remote sensing. Starting from a neural network already trained for semantic segmentation, we propose to modify its label space to swiftly adapt it to new classes under weak supervision. To alleviate the background shift and the catastrophic forgetting problems inherent to this form of continual learning, we compare different regularization terms and leverage a pseudo-label strategy. We experimentally show the relevance of our approach on three public remote sensing datasets. Code is open-source and released in this repository: https://github.com/alteia-ai/ICSS.

Maintaining farm sustainability through optimizing the agricultural management practices helps build more planet-friendly environment. The emerging satellite missions can acquire multi- and hyperspectral imagery which captures more detailed spectral information concerning the scanned area, hence allows us to benefit from subtle spectral features during the analysis process in agricultural applications. We introduce an approach for extracting 2.5m cultivated land maps from 10m Sentinel-2 multispectral image series which benefits from a compact graph convolutional neural network. The experiments indicate that our models not only outperform classical and deep machine learning techniques through delivering higher-quality segmentation maps, but also dramatically reduce the memory footprint when compared to U-Nets (almost 8k trainable parameters of our models, with up to 31M parameters of U-Nets). Such memory frugality is pivotal in the missions which allow us to uplink a model to the AI-powered satellite once it is in orbit, as sending large nets is impossible due to the time constraints.

Hyperspectral unmixing remains one of the most challenging tasks in the analysis of such data. Deep learning has been blooming in the field and proved to outperform other classic unmixing techniques, and can be effectively deployed onboard Earth observation satellites equipped with hyperspectral imagers. In this letter, we follow this research pathway and propose a multi-branch convolutional neural network that benefits from fusing spectral, spatial, and spectral-spatial features in the unmixing process. The results of our experiments, backed up with the ablation study, revealed that our techniques outperform others from the literature and lead to higher-quality fractional abundance estimation. Also, we investigated the influence of reducing the training sets on the capabilities of all algorithms and their robustness against noise, as capturing large and representative ground-truth sets is time-consuming and costly in practice, especially in emerging Earth observation scenarios.

Deep learning, together with the availability of large amounts of data, has transformed the way we process Earth observation (EO) tasks, such as land cover mapping or image registration. Yet, today, new models are needed to push further the revolution and enable new possibilities. This work focuses on a recent framework for generative modeling and explores its applicability to the EO images. The framework learns an energy-based model (EBM) to estimate the underlying joint distribution of the data and the categories, obtaining a neural network that is able to classify and synthesize images. On these two tasks, we show that EBMs reach comparable or better performances than convolutional networks on various public EO datasets and that they are naturally adapted to semisupervised settings, with very few labeled data. Moreover, models of this kind allow us to address high-potential applications, such as out-of-distribution analysis and land cover mapping with confidence estimation.

Deep learning has changed unbelievably the processing of Earth Observation tasks such as land cover mapping or image registration. Yet, today new models are needed to push further the revolution and enable new possibilities. We propose a new framework for generative modelling of Earth Observation images. It learns an energy-based model to estimate the underlying distribution of the data while jointly training a deep neural network for classification. On the varied image types of the EuroSAT benchmark, we show this model obtains classification results on par with state-of-the-art and moreover allows us to tackle a wide range of high-potential applications: image synthesis, out-of-distribution testing for domain adaptation, and image completion or denoising.

This article aims to explore the potential of current approaches for quantum image classification in the context of remote sensing. After a brief outline of quantum computers and an analysis of the current bottlenecks, it shows for the first time experiments with quantum neural networks on a reference Earth observation (EO) dataset: EuroSAT. Moreover, it establishes the proof of concept of quantum computing for EO: the models trained and run on a quantum simulator are on par with classical ones. We make the open-source code available for further developments11QNN4EO repository: https://github.com/ESA-PhiLab/QNN4EO..

Aerial or satellite imagery is a great source for land surface analysis, which might yield land-use maps or elevation models. In this letter, we present a neural network framework for learning semantics and local height together. We show how this joint multitask learning benefits to each task on the large data set of the 2018 Data Fusion Contest. Moreover, our framework also yields an uncertainty map that allows assessing the prediction of the model. Code is available at https://github.com/marcelampc/mtl_aerial_images.

In this paper we discuss the issues of using inexact and inaccurate ground truth in the context of supervised learning. To leverage large amounts of Earth observation data for training algorithms, one often has to use ground truth which was not been carefully assessed. We address both the problems of training and evaluation. We first propose a weakly supervised approach for training change classifiers which is able to detect pixel-level changes in aerial images. We then propose a data poisoning approach to get a reliable estimate of the accuracy that can be expected from a classifier, even when the only ground-truth available does not match the reality. Both are assessed on practical land use and land cover applications.

Depth estimation from a single monocular image has reached great performances thanks to recent works based on deep networks. However, as various choices of losses, architectures and experimental conditions are proposed in the literature, it is difficult to establish their respective influence on the performances. In this paper we propose an in-depth study of various losses and experimental conditions for depth regression, on NYUv2 dataset. From this study we propose a new network for depth estimation combining an encoder-decoder architecture with an adversarial loss. This network reaches top scores in the competitive evaluation of NUYv2 dataset while being simpler to train in a single phase.

This paper presents three fully convolutional neural network architectures which perform change detection using a pair of coregistered images. Most notably, we propose two Siamese extensions of fully convolutional networks which use heuristics about the current problem to achieve the best results in our tests on two open change detection datasets, using both RGB and multispectral images. We show that our system is able to learn from scratch using annotated change detection images. Our architectures achieve better performance than previously proposed methods, while being at least 500 times faster than related systems. This work is a step towards efficient processing of data from large scale Earth observation systems such as Copernicus or Landsat.

This work addresses the scarcity of annotated hyperspectral data required to train deep neural networks. Especially, we investigate generative adversarial networks and their application to the synthesis of consistent labeled spectra. By training such networks on public datasets, we show that these models are not only able to capture the underlying distribution, but also to generate genuine-looking and physically plausible spectra. Moreover, we experimentally validate that the synthetic samples can be used as an effective data augmentation strategy. We validate our approach on several public hyperspectral datasets using a variety of deep classifiers.

This paper proposes a method of denoising SAR images, using a deep learning method, which takes advantage of the abundance of data to learn on large stacks of images of the same scene. The approach is based on the use of convolutional networks, used as auto-encoders. Learning is led on a large pile of images acquired on the same area, and assumes that the images of this stack differ only by the speckle noise. Several pairs of images are chosen randomly in the stack, and the network tries to predict the slave image from the master image. In this prediction, the network can not predict the noise because of its random nature. Also the application of this network to a new image fulfills the speckle filtering function. Results are given on Sentinel 1 images. They show that this approach is qualitatively competitive with literature.

The Copernicus Sentinel-2 program now provides multispectral images at a global scale with a high revisit rate. In this paper we explore the usage of convolutional neural networks for urban change detection using such multispectral images. We first present the new change detection dataset that was used for training the proposed networks, which will be openly available to serve as a benchmark. The Onera Satellite Change Detection (OSCD) dataset is composed of pairs of multispectral aerial images, and the changes were manually annotated at pixel level. We then propose two architectures to detect changes, Siamese and Early Fusion, and compare the impact of using different numbers of spectral channels as inputs. These architectures are trained from scratch using the provided dataset.

Over the recent years, there has been an increasing interest in large-scale classification of remote sensing images. In this context, the Inria Aerial Image Labeling Benchmark has been released online in December 2016. In this paper, we discuss the outcomes of the first year of the benchmark contest, which consisted in dense labeling of aerial images into building / not building classes, covering areas of five cities not present in the training set. We present four methods with the highest numerical accuracies, all four being convolutional neural network approaches. It is remarkable that three of these methods use the U-net architecture, which has thus proven to become a new standard in image dense labeling.

This paper deals with classification of remote sensing data to extract objects for industrial mapping. While land-cover or urban mapping have been extensively studied, industrial cartography remains a field yet to explore, in spite of tremendous needs. We present and compare here four approaches for railway detection in very high resolution images. They use various kind of filtering approaches, including the trained filters of fully convolutional networks. Moreover, they benefit from different a-priori and post-processing techniques to make them more robust. We evaluate all approaches on a challenging dataset captured on an operating station site with complex objects.

This is a placeholder for the invited session presenting the results of the Data Fusion Contest 2017. Papers in the session do not enter the usual review process. Instead, winners of the 2017 Data Fusion Contest will write 4-page papers evaluated by the Contest Committee, that will be submitted as camera-ready papers in due date. So, DO NOT REVIEW THIS PAPER. If by mistake it enters the usual review process, please contact Devis Tuia ([email protected]) and Bertrand Le Saux ([email protected]).

In this paper, we investigate the impact of segmentation algorithms as a preprocessing step for classification of remote sensing images in a deep learning framework. Especially, we address the issue of segmenting the image into regions to be classified using pre-trained deep neural networks as feature extractors for an SVM-based classifier. An efficient segmentation as a preprocessing step helps learning by adding a spatially-coherent structure to the data. Therefore, we compare algorithms producing superpixels with more traditional remote sensing segmentation algorithms and measure the variation in terms of classification accuracy. We establish that superpixel algorithms allow for a better classification accuracy as a homogenous and compact segmentation favors better generalization of the training samples.

In this paper, we address the task of semantic labeling of multisource earth-observation (EO) data. Precisely, we benchmark several concurrent methods of the last 15 years, from expert classifiers, spectral support-vector classification and high-level features to deep neural networks. We establish that (1) combining multisensor features is essential for retrieving some specific classes, (2) in the image domain, deep convolutional networks obtain significantly better overall performances and (3) transfer of learning from large generic-purpose image sets is highly effective to build EO data classifiers.

This paper deals with the interactive design of generic classifiers for aerial images. In many real-life cases, object detectors that work are not available, due to a new geographical context or a need for a type of object unseen before. We propose an approach for on-line learning of such detectors using user interactions. Variants of gradient boosting and support-vector machine classification are proposed to cope with the problems raised by interactivity: unbalanced and partially mislabeled training data. We assess our framework for various visual classes (buildings, vegetation, cars, visual changes) on challenging data corresponding to several applications (SAR or optical sensors at various resolutions). We show that our model and algorithms outperform several state-of-the-art baselines for feature extraction and learning in remote sensing.

Data from satellite and aerial images are now widely used by everyone. These images contain information from different frequency bands that help to characterize areas of interest. In this paper we study a framework for object detection in aerial image based on discriminatively-trained models trained on multimodal data. Specifically, we investigate a method to merge outputs of large margin classifiers trained on images from different sensors: we use the ranking ability of these classifiers to learn a probabilistic model.

In this paper we apply the deformable part model by Felzenszwalb et al., which is at this moment the state of the art in many computer vision related tasks, to detect different types of man made structures in very high resolution aerial images — a reputedly difficult problem in our field. We test the framework on a database of crops of aerial images at a definition of 10 cm/pixel and investigate how the model performs on several classes of objects. The results show that the model can achieve reasonable performance in this context. However, depending on the type of object, there are specific issues which will have to be taken into account to build an effective semi-supervised annotation tool based on this model.

This paper focuses on finding changes in an urban environment (new or demolished buildings, activity monitoring) using Synthetic Aperture Radar (SAR) imagery. We propose a novel approach to characterize changes between two registered images. First, “what is a change” is learned interactively using user-provided examples in order to adapt the detection to the query context. Second, we propose the Change-Index Histogram of Oriented Gradients (CI-HOG), a new change descriptor that captures local statistics of change indices. We assess our system on TerraSAR-X data captured over challenging locations.

We propose solutions to provide unmanned aerial vehicles (UAV) with features to understand the scene below and help the operational planning. First, using a visual mapping of the environnement, interactive learning of specific targets of interest is performed on the ground control station to build semantic maps useful for planning. Then, the learned target detectors are transformed to be applied to new images captured by the UAV. On the technical side, we present: (i) an online gradient boost algorithm to interactively design context-dependent detectors; (ii) a video-domain adaptation method to use object detectors on on-board-camera images. We verify our approach on challenging data captured in real-world conditions.

We describe an interactive framework for man-made structure classification. Our system is able to help an image analyst to define a query that is adapted to various image and geographic contexts. It offers a GIS-like interface for visually selecting the training region samples and a fast and efficient sample description by histogram of oriented gradients and local binary patterns. To learn a discrimination rule in this feature space, our system relies on the online gradient-boost learning algorithm for which we defined a new family of loss functions. We chose non-convex loss-functions in order to be robust to mislabelling and proposed a generic way to incorporate prior information about the training data. We show it achieves better performances than other state-of-the-art machine-learning methods on various man-structure detection problems.

We present a machine-learning based method for the exploration of remote sensing data. Our framework mixes an intuitive interface and a one-class support-vector machine to look for rare patterns in satellite images. It benefits from a fast implementation on the Graphics Process Unit that allows reasonable times for system-user interactions. We validate our approach with ground-truth experiments and demonstrate the method on real-world datasets. We achieve faster computations when compared with sequential implementations of the same methods (up to 80 times faster for feature extraction) and with other classification methods (such as local distribution comparison).

To allow efficient browsing of large image collection, we have to provide a summary of its visual content. We present in this paper a new robust approach to categorize image databases: Adaptive Robust Competition with Proximity-Based Merging (ARC-M). This algorithm relies on a non-supervised database categorization, coupled with a selection of prototypes in each resulting category. Each image is represented by a high-dimensional vector in the feature space. A principal component analysis is performed for every feature to reduce dimensionality. Then, clustering is performed in challenging conditions by minimizing a Competitive Agglomeration objective function with an extra noise cluster to collect outliers. Agglomeration is improved by a merging process based on cluster proximity verification.