Geospatial models must adapt to the diversity of Earth observation data in terms of resolutions, scales, and modalities. However, existing approaches expect fixed input configurations, which limits their practical applicability. We propose AnySat, a multimodal model based on joint embedding predictive architecture (JEPA) and scale-adaptive spatial encoders, allowing us to train a single model on highly heterogeneous data in a self-supervised manner. To demonstrate the advantages of this unified approach, we compile GeoPlex, a collection of 5 multimodal datasets with varying characteristics and 11 distinct sensors. We then train a single powerful model on these diverse datasets simultaneously. Once fine-tuned or probed, we reach state-of-the-art results on the test sets of GeoPlex and for 6 external datasets across various environment monitoring tasks: land cover mapping, tree species identification, crop type classification, change detection, climate type classification, and segmentation of flood, burn scar, and deforestation. Our code and models are available at https://github.com/gastruc/AnySat.

Bi-temporal change detection at scale based on Very High Resolution (VHR) images is crucial for Earth monitoring. This remains poorly addressed so far: methods either require large volumes of annotated data (semantic case), or are limited to restricted datasets (binary set-ups). Most approaches do not exhibit the versatility required for temporal and spatial adaptation: simplicity in architecture design and pretraining on realistic and comprehensive datasets. Synthetic datasets are the key solution but still fail to handle complex and diverse scenes. In this paper, we present HySCDG a generative pipeline for creating a large hybrid semantic change detection dataset that contains both real VHR images and inpainted ones, along with land cover semantic map at both dates and the change map. Being semantically and spatially guided, HySCDG generates realistic images, leading to a comprehensive and hybrid transfer-proof dataset FSC-180k. We evaluate FSC-180k on five change detection cases (binary and semantic), from zero-shot to mixed and sequential training, and also under low data regime training. Experiments demonstrate that pretraining on our hybrid dataset leads to a significant performance boost, outperforming SyntheWorld, a fully synthetic dataset, in every configuration. All codes, models, and data are available here: https://yb23.github.io/projects/cywd/.

Land-use/land-cover (LULC) maps describe the Earth’s surface with discrete classes at a specific spatial resolution. The chosen classes and resolution highly depend on peculiar uses, making it mandatory to develop methods to adapt these characteristics for a large range of applications. Recently, a convolutional neural network (CNN)-based method was introduced to take into account both spatial and geographical context to translate a LULC map into another one. However, this model only works for two maps: one source and one target. Inspired by natural language translation using multiple-language models, this article explores how to translate one LULC map into several targets with distinct nomenclatures and spatial resolutions. We first propose a new data set based on six open access LULC maps to train our CNN-based encoder-decoder framework. We then apply such a framework to convert each of these six maps into each of the others using our Multi-Landcover Translation network (MLCT-Net). Extensive experiments are conducted at a country scale (namely France). The results reveal that our MLCT-Net outperforms its semantic counterparts and gives on par results with mono-LULC models when evaluated on areas similar to those used for training. Furthermore, it outperforms the mono-LULC models when applied to totally new landscapes.

This paper presents a framework for simultaneously translating multiple land-cover maps into a given one in a supervised way. Conversely to existing approaches working on 1–1 translation, we propose a multi-translation setup that increases the generalizability and translation performance, especially on land-cover maps covering restricted spatial extents. The proposed method mainly assumes that the map of interest spatially overlaps at least with one of the other maps. High performance translation is achieved with a Convolutional Neural Network (CNN) based encoder-decoder frame-work trained with three goals: (i) high-quality translation; (ii) self-reconstruction ability; (iii) mapping of all datasets into a common representation space. Country-scale experimental results show the method effectiveness in translating six highly heterogeneous land-cover maps, achieving significantly better results than the traditional semantic-based method and better results than CNN trained for a 1–1 translation task (+ 9.7% in Overall Accuracy (OA) and +12% in macro F1-score (mF1)).

This paper presents a framework for translating a land-cover map into another one in a supervised way. This links to numerous applications (updating, completion, etc.). Conversely to existing approaches, we jointly perform spatial and semantic transformation without any prior knowledge. The proposed method assumes that: i) examples of the source and target maps already exist, ii) the spatial resolution of the source map is equal or higher than the target one. The translation is performed using an asymmetric Convolutional Neural Network with positional encoding. Experimental results show the effectiveness of the method in retrieving a yearly version of Corine Land Cover (CLC) at country-scale (France) using an existing high-resolution map and with similar accuracy than existing CLC maps (~80%).

One key factor to exhaustive vegetation monitoring lies in the dense temporal sampling of the measurements. Areas subject to multiple human interventions, such as grasslands, are particularly concerned. A Recurrent Neural Network multi-sensor regression approach (SenRVM), relying on the systematic acquisitions of Sentinel-1 SAR satellite, has been thereby proposed. It permits to retrieve vegetation indexes, derived from Sentinel- 2 optical imagery, despite significant cloud cover and with high sampling (6 days). The benefit of SenRVM for filling gaps in vegetation time-series describing agricultural practices is assessed. The proposed approach is compared with classical mono-sensor optical strategies. We adopt a synthetic dataset with large gaps. This realistically mimicks challenging conditions in grassland exploitation detection. Results obtained both for exploited and stable parcels satisfactorily demonstrate the relevance of our approach.

Label noise is known to negatively impact the performance of classification algorithms. In this paper, we develop a model robust to label noise that uses both labelled and unlabelled samples. In particular, we propose a novel algorithm to optimize the model parameters that scales efficiently w.r.t. the number of training samples. Our contribution relies on a consensus formulation of the original objective function that is highly parallelizable. The optimization is performed with the Alternating Direction Method of Multipliers framework. Experimental results on synthetic datasets show an improvement of several orders of magnitude in terms of processing time, with no loss in terms of accuracy. Our method appears also tailored to handle real data with significant label noise.

The generation of 3D building models from Very High Resolution geospatial data is now an automatized procedure. However, urban areas are very complex and practitioners still have to visually assess the correctness of these models and detect reconstruction errors. We proposed an approach for automatically evaluating the quality of 3D building models. It is cast as a supervised classification task based on a hierarchical taxonomy and multimodal handcrafted features (building geometry, optical images, height data). In this paper, we evaluate how the urban area composition impacts prediction transferability and scalability of our framework to unseen scenes. This allows to define minimal feature and training sets for a problem where no benchmark data has been released so far.

It has been now widely assessed in the literature that both multi/hyperspectral optical images and 3D lidar point clouds are necessary inputs for tree species based forest stand detection. Nevertheless, no comprehensive analysis of the genuine relevance of each data source has been performed so far: existing strategies are limited to a single spatial and spectral resolution. This paper investigates which is the optimal combination of geospatial optical images and lidar point clouds. A supervised semantic segmentation framework is fed with various sources (multispectral satellite and airborne images, hyperspectral airborne images, low, medium and high density lidar point clouds), ablation cases are defined, and the discrimination performance of several fusion schemes is assessed under a challenging mountainous area in France.

Supervised classification is the fundamental task for land-cover map generation. Deep neural networks recently outperformed other state-of-the-art classifiers in many machine learning challenges, from semantic segmentation to speech recognition. Such strategies are now commonly employed in the literature for the purpose of land-cover mapping. This paper develops the strategy for the use of deep networks to label very high resolution satellite images, with the perspective of mapping regions at country scale. Therefore, a superpixel based method is introduced in order to (i) ensure correct delineation of objects and (ii) perform the classification in a dense way but with decent computing times.

Semantic segmentation of remote sensing images enables in particular land-cover map generation for a given set of classes. Very recent literature has shown the superior performance of deep convolutional neural networks (DCNN) for many tasks, from object recognition to semantic labelling, including the classification of Very High Resolution (VHR) satellite images. However, while plethora of works aim at improving object delineation on geographically restricted areas, few tend to solve this classification task at very large scales. New issues occur such as intra-class class variability, diachrony between surveys, and the appearance of new classes in a specific area, that do not exist in the predefined set of labels. Therefore, this work intends to (i) perform large scale classification and to (ii) expand a set of land-cover classes, using the off-the-shelf model learnt in a specific area of interest and adapting it to unseen areas.

A strong intern dynamic characterizes towns, a very high spatial heterogeneity of their elements, their 3D geometric shapes (horizontal and vertical) inducing shadows, and their large variety of materials. These characteristics make the collection of information of land surface properties and urban descriptors more delicate. Due to the enhancement of spatial to deepen the observation of urban areas. Nevertheless, such a type of sensors would not contribute to the characterization of the urban land surface properties (chemical composition of materials, species of vegetation, quality of soils, etc.). They and show great potentials might consider Hyperspectral imagery capacities as providing useful products but it becomes mandatory to define which type of information these different sensors can deliver. The ANR HYEP project has the purpose to demonstrate the benefit of a second generation of hyperspectral space borne mission characterized by a high spatial resolution (8m GSD) and a high temporal revisit. After a detailed description of the motivation of such a proposal, applications are given focused on urban vegetation, sealed and impervious areas, solar panel area estimation.

Forest stands are a basic unit of analysis for forest inventory and mapping. Stands are defined as large forested areas of homogeneous tree species composition and age. Their accurate delineation is usually performed by human operators through visual analysis of very high resolution (VHR) infra-red and visible images. This task is tedious, highly time consuming, and needs to be automated for scalability and efficient updating purposes. The most appropriate fusion of two remote sensing modalities (lidar and multispectral images) is investigated here. The multispectral images give information about the tree species while 3D lidar point clouds provide geometric information. The fusion is operated at three different levels within a semantic segmentation workflow: over-segmentation, classification, and regularization. Results show that over-segmentation can be performed either on lidar or optical images without performance loss or gain, whereas fusion is mandatory for efficient semantic segmentation. Eventually, the fusion strategy dictates the composition and nature of the forest stands, assessing the high versatility of our approach.

In this paper, we focus on the classification of lidar point cloud data acquired via mobile laser scanning, whereby the classification relies on a context model based on a Conditional Random Field (CRF). We present two approximate inference algorithms based on belief propagation, as well as a graph-cut-based approach not yet applied in this context. To demonstrate the performance of our approach, we present the classification results derived for a standard benchmark dataset. These results clearly indicate that the graph-cut-based method is able to retrieve a labeling of higher likelihood in only a fraction of the time needed for the other approaches. The higher likelihood, in turn, translates into a significant gain in the accuracy of the obtained classification.

Canopy structure plays an essential role in biophysical activities in forest environments. However, quantitative descriptions of a 3-D canopy structure are extremely difficult because of the complexity and heterogeneity of forest systems. Airborne laser scanning (ALS) provides an opportunity to automatically measure a 3-D canopy structure in large areas. Compared with other point cloud technologies such as the image-based Structure from Motion, the power of ALS lies in its ability to penetrate canopies and depict subordinate trees. However, such capabilities have been poorly explored so far. In this paper, the potential of ALS-based approaches in depicting a 3-D canopy structure is explored in detail through an international benchmarking of five recently developed ALS-based individual tree detection (ITD) methods. For the first time, the results of the ITD methods are evaluated for each of four crown classes, i.e., dominant, codominant, intermediate, and suppressed trees, which provides insight toward understanding the current status of depicting a 3-D canopy structure using ITD methods, particularly with respect to their performances, potential, and challenges. This benchmarking study revealed that the canopy structure plays a considerable role in the detection accuracy of ITD methods, and its influence is even greater than that of the tree species as well as the species composition in a stand. The study also reveals the importance of utilizing the point cloud data for the detection of intermediate and suppressed trees. Different from what has been reported in previous studies, point density was found to be a highly influential factor in the performance of the methods that use point cloud data. Greater efforts should be invested in the point-based or hybrid ITD approaches to model the 3-D canopy structure and to further explore the potential of high-density and multiwavelengths ALS data.

Land-cover geodatabases are key products for the understanding of environmental systems and for setting up national and international prevention and protection policies. However, their automatic generation and update remain complicated with high accuracy over large scales. In natural environments, most of the existing solutions are semi-automatic in order to achieve a suitable discrimation of the large number of forest and crop classes. A large amount of remote sensing possibilities is at the moment available and data fusion appears to be the most suitable solution for that purpose. The paper tackles the issue of land-cover mapping in such areas assuming the existence of a partly non-updated 5-class geodatabase: buildings, roads, water, crops, forests. Lidar point clouds and Radar images at two spatial resolutions and bands are merged at the feature level and fed into an efficient supervised classification framework. Results show that some classes benefit from the joint exploitation of multiple observations in terms of accuracy or recall.

Land-Cover databases (LC-DB) are mandatory for environmental purposes, but need to be regularly updated to provide robust and instructive spatial indicators. Moreover, an increasing number of sensors, such as optical and SAR satellite images or Lidar point cloud, allow to cover large areas regularly, and with a very high precision. Thus, automatic methods have to be developed to take into account the complementarity of available observations. In this paper, several fusion methods are proposed and introduced in an existing Land-Cover mapping framework. Those methods are compared on several scenarii (based on optical, SAR and Lidar datasets), and evaluated thanks to a very high resolution LC-DB.

Land-Cover databases (LC-DB) are very useful for environmental purposes, but need to be semantically detailed to provide robust and instructive spatial indicators. Moreover, remote sensed data allow to cover large areas with high temporal resolution. Such multi-temporal data are very useful input to discriminate LC classes. Nevertheless, automatic fusion method need to be developed to provide high quality LC-DB. In this paper, several fusion methods are proposed and introduced in an existing Land-Cover mapping framework. Those fusion methods allow to take advantage of multi-temporal data. Those methods are compared, and assessed thanks to a very high resolution LC-DB.

The canopy density model (CDM), a new product interpolated from airborne laser scanner (ALS) data and dedicated to forest structure characterization is presented. It exploits both the multiecho capability of the ALS and a nonparametric density estimation technique called kernel density estimators (KDEs). The CDM is used to delineate the outmost perimeter of vegetation features and to compute forest crown cover (CrCO). Contrary to other works that focus on single-layer forest canopies, CrCo is derived here for each layer, namely, the overstory, the understory, and ground vegetation. The root-mean-square error of prediction determined by using field data acquired over 44 forest stands in a forest in Portugal allows the testing of the reliability of the method: It ranges from 6.21% (overstory) to 13.76% (ground vegetation). In addition, we investigate the ability of the CDM to map the CrCo for individual trees. Finally, two existing methods have been applied to our study site in order to assess improvements, advantages, and drawbacks of our approach.

2D land-cover databases (LC-DB) have been established at various levels (global, national or regional scales), various spatial samplings and for various themes of interest (forest, agriculture, urban areas, etc.). However, they exhibit many flaws (limited geometric accuracy, low coverage) and require to be updated with automatic algorithms. Very High Resolution satellite imagery offers a suitable solution for setting up such on-purpose algorithms, and a large body of literature has tackled this topic. This paper proposes a framework that is able to deal with both LC-DB update of any kind and their enrichment in case of incomplete DB. The supervised classification-based solution integrates an efficient learning strategy that allows to capture the heterogeneity of the appearances of the various themes of interest. The proposed framework is favorably compared with two state-of-the-art methods, on a reconstructed dataset, composed of sub-metric satellite image patches.

In this work, we present an approach that is able to deal with large-scale road network mapping. While former methods focus on delineating patches of roads without computing a coherent road network, we formulate a very large number of road hypothesis that are pruned using a graph reasoning and weak a priori knowledge on road behavior. The initial solution is computed by means of two machine learning and pattern recognition state-of-the-art methods (namely, Random Forest classification and Marked Point Process) that allow to process very large areas in little time with very satisfactory results.

Land-Cover databases (LC-DB) are very useful for environmental purposes, but need to be regularly updated to provide robust and instructive spatial indicators. Moreover, very high resolution satellite images allow to cover large areas regularly. Thus, automatic methods have to been developed to tackle this issue. In this paper, a hierarchical inspection method is proposed to both update and extend LC-DB using satellite image. This framework is successfully applied on the French National LC-DB using a single VHR satellite image.

Timely and accurate measurements of forest parameters are critical for ecosystem studies, sustainable forest resources management, monitoring and planning. This paper presents a processing chain for individual tree segmentation over large areas with airborne LiDAR 3D point cloud and very high resolution (VHR) optical imagery. The proposed processing chain consists of forest stand level delineation with optical imagery, individual tree segmentation with Canopy Height Model (CHM) derived from LiDAR point cloud, rough characterization of trees at forest stand level, and point clustering of individual tree with an Adaptive Mean Shift 3D (AMS3D) algorithm. The processing chain is developed with the expectation of supporting operational forest inventory at individual tree level. Experiment is conducted using LiDAR data acquired in Ventoux region, France. Results suggest that the proposed processing chain can be successfully adopted for individual tree characterization over large areas with different forest stands.

Building detection from geospatial optical images has been a popular topic of research for the last twenty years and in particular with the emergence of very high resolution satellites. Existing methods exhibit various flaws and prevent them from being efficient at large scales of space and time: they are context-dependent, require a tedious parameter tuning or several data sources. In this paper, we propose a fully automatic method that alleviates some of these issues by combining the strengths of bottom-up and top-down approaches, i.e., of both classification and pattern recognition algorithms. This allows to correctly detect the objects by geometric prior knowledge while finely delineating their borders and preserving their shapes. The method is evaluated over a complex area of more than 230 buildings using a 0.5 m multispectral pansharpened Pleiades image.

Canopy height and canopy cover are two important biophysical variables in forest characterization. As far as airborne laser scanning (ALS) studies is concerned, canopy height has been mapped at a high spatial resolution through canopy height models (CHM). Conversely, canopy cover had been computed at a coarser resolution (plot-level) by means of penetration rate metrics (PRM). In this work we present a method dedicated to compute single strata canopy cover at a spatial resolution similar to the CHM. Vegetation density maps are interpolated from the ALS point cloud pattern using a novel method based on the kernel density estimators (KDE). We developed an automatic variable-bandwidth approach that accommodates for both the point cloud density variability inherent to any ALS survey and the shading effect induced by taller canopy on laser beams. Results are compared with the PRM approach.

Accurate Digital Terrain Models (DTM) are inevitable inputs for mapping areas subject to natural hazards. Topographic lidar scanning has become an established technique to characterize the Earth surface: and reconstruct the topography. For flood hazard modeling in coastal areas, the key step before terrain modeling is the discrimination of land and water surfaces within the delivered point clouds. Therefore, instantaneous shoreline, river borders, inland waters can be extracted as a basis for more reliable DTM generation. This paper presents an automatic, efficient, and versatile workflow for land/water classification of airborne topographic lidar data. For that purpose, a classification framework based on Support Vector Machines (SVM) is designed. First, a set of features, based only 3D lidar point coordinates and flightline information, is defined. Then, the SVM learning step is performed on small but well-targeted areas thanks to an automatic region growing strategy. Finally, label probabilities given by the SVM are merged during a probabilistic relaxation step in order to remove pixel-wise misclassification. Results over two large areas show that survey of millions of points are labelled with high accuracy (>95%) and that small features of interest are still well classified though we work at low point densities (2-3pts/m2). Finally, our approach provides a strong basis for further discrimination of coastal land-cover classes and habitats.

Current methods for accurately estimating vegetation biomass with remote sensing data require extensive, representative and time consuming field measurements to calibrate the sensor signal. In addition, such techniques focus on the topmost vegetation canopy and thus they are of little use over multi-layered forest ecosystems where the underneath strata hold considerable amounts of biomass. This work is the first attempt to estimate biomass by remote sensing without the need for massive in situ measurements. Indeed, we use small-footprint airborne laser scanning (ALS) data to derive key forest metrics, which are used in allometric equations that were originally established to assess biomass using field measurements. Field- and ALS-derived biomass estimates are compared over 40 plots of a multi-layered Mediterranean forest. Linear regression models explain up to 99% of the variability associated with surface vegetation, understory, and overstory biomass.

We present a robust method for modeling cities from unstructured point data. Our algorithm provides a more complete description than existing approaches by reconstructing simultaneously buildings, trees and topologically complex grounds. Buildings are modeled by an original approach which guarantees a high generalization level while having semantized and compact representations. Geometric 3D-primitives such as planes, cylinders, spheres or cones describe regular roof sections, and are combined with mesh-patches that represent irregular roof components. The various urban components interact through a non-convex energy minimization problem in which they are propagated under arrangement constraints over a planimetric map. We experimentally validate the approach on complex urban structures and large urban scenes of millions of points.

Lidar waveforms are 1-D signals representing a train of echoes caused by reflections at different targets. Modeling these echoes with the appropriate parametric function is useful to retrieve information about the physical characteristics of the targets. This paper presents a new probabilistic model based upon a marked point process which reconstructs the echoes from recorded discrete waveforms as a sequence of parametric curves. Such an approach allows to fit each mode of a waveform with the most suitable function and to deal with both, symmetric and asymmetric, echoes. The model takes into account a data term, which measures the coherence between the models and the waveforms, and a regularization term, which introduces prior knowledge on the reconstructed signal. The exploration of the associated configuration space is performed by a reversible jump Markov chain Monte Carlo (RJMCMC) sampler coupled with simulated annealing. Experiments with different kinds of lidar signals, especially from urban scenes, show the high potential of the proposed approach. To further demonstrate the advantages of the suggested method, actual laser scans are classified and the results are reported.

Airborne lidar systems have become an alternative source for the acquisition of altimeter data. In addition to multi-echo laser scanner systems, full-waveform systems are able to record the whole backscattered signal for each emitted laser pulse. These data provide more information about the structure and the physical properties of the surface. This paper is focused on the classification of full-waveform lidar and airborne image data on urban scenes. Random forests are used since they provide an accurate classification and run efficiently on large datasets. Moreover, they provide measures of variable importance for each class. This is crucial to analyze the relevance of each feature for the classification of urban scenes. Random Forests provide more accurate results than Support Vector Machines with an overall accuracy of 95.75%. The most relevant features show the contribution of lidar waveforms for classifying dense urban scenes and improve the classification accuracy for all classes.

Lidar waveforms are 1D signal consisting of a train of echoes where each of them correspond to a scattering target of the Earth surface. Modeling these echoes with the appropriate parametric function is necessary to retrieve physical information about these objects and characterize their properties. This paper presents a marked point process based model to reconstruct a lidar signal in terms of a set of parametric functions. The model takes into account both a data term which measures the coherence between the models and the waveforms, and a regularizing term which introduces physical knowledge on the reconstructed signal. We search for the best configuration of functions by performing a Reversible Jump Markov Chain Monte Carlo sampler coupled with a simulated annealing. Results are finally presented on different kinds of signals in urban areas.

Frequency scaling concerns the variation of propagation effects with respect to frequency. The objective is to find the relationship between attenuation at a given frequency from the attenuation measured at another frequency, generally lower. Two different kinds of frequency scaling model, corresponding to different interests, can be considered: Long term frequency scaling, describes the relationship between attenuation for the same probability level. It allows studying the design of system operating at high frequency bands (Ka or V band) from the performances of existing systems operating at lower frequency band (Ku-band). Short term frequency scaling or instantaneous frequency scaling (IFS), describes the relationship between simultaneous attenuation at different frequencies. It allows performing uplink power control, where the attenuation on the uplink is estimated from the attenuation measured on the downlink. The different contributions: rains, gas, clouds, which contribute to the total attenuation, depend on frequency in different ways, that's why this technique is most satisfactory when one cause predominates. The present study focus on IFS of rain, the aim is to deduce the attenuation due to rain for one frequency (higher than 40 GHz) from the measurements at another lowers frequencies (Ka Band).