In recent decades, rapid urbanization in India, fueled by population growth, has spurred the construction of new cities and the expansion of existing urban centers, extending into larger peripheral areas—a trend common in developing nations globally. Despite its widespread occurrence, accurately mapping human settlements and building distributions remains a challenge. State authorities and private enterprises, including Microsoft and Google, have sought to comprehensively capture this data. While existing initiatives offer a global perspective, challenges persist, especially in precision, for countries like India where cities boast highly dense and mixed urban development. This study explores the potential of Sentinel-2 images for building footprint mapping and change detection at 2.5 m spatial resolution, focusing on the dynamic Delhi National Capital Region (NCR). Recent works demonstrate sub-pixel accuracy in deriving building footprint maps through deep learning on Sentinel-2 imagery. The research aims to develop on existing findings taking into account seasonal variations and using improved training labels to further extend these findings to India.

Unsupervised transfer learning-based change detection (CD) methods exploit the feature extraction capability of pretrained networks to distinguish changed pixels from unchanged ones. However, their performance may vary significantly depending on several geographical and model-related aspects. In many applications, it is of utmost importance to provide trustworthy or confident results, even if over a subset of pixels. The core challenge in this problem is to identify changed pixels and confident pixels in an unsupervised manner. To address this, we propose a two-network model—one tasked with mere CD and the other with confidence estimation. While the CD network can be used in conjunction with popular transfer learning-based CD methods such as deep change vector analysis, the confidence estimation network operates similarly to a randomized smoothing model. By ingesting ensembles of inputs perturbed by noise, it creates a distribution over the output and assigns confidence to each pixel’s outcome. The novelty of this work lies in methodologically identifying confident pixels during unsupervised deep transfer learning-based CD, a feature typically absent in these methods, which generally do not offer an indicator of confidence or uncertainty. We tested the proposed method on two different Earth observation sensors: optical and synthetic aperture radar (SAR). The proposed method achieved an increase in the$F1$score by approximately eight points for the optical dataset and five points for the SAR dataset compared to no confidence estimation.

In this study, we examine the potential of several self-supervised deep learning models in predicting forest attributes and detecting forest changes using ESA Sentinel-1 and Sentinel-2 images. The performance of the proposed deep learning models is compared to established conventional machine learning approaches. Studied use-cases include mapping of forest disturbance (windthrown forests, snowload damages) using deep change vector analysis, forest height mapping using UNet+ based models, Momentum contrast and regression modeling. Study areas were represented by several boreal forest sites in Finland. Our results indicate that developed methods allow to achieve superior classification and prediction accuracies compared to traditional methodologies and mimimize the amount of necessary in-situ forestry data.

In the field of urban monitoring, accurate mapping of building structures and the corresponding changes is one of the most essential pieces of information. In many cases, this problem is approached with very high-resolution images, needed due to the spatial complexity in urban environments. And still, many binary change detection (CD) methods cannot segregate the changes introduced by the generation of new building structures from seasonal changes or other semantic changes. In this paper, we investigate a simple approach for building CD from freely available Sentinel-2 images, that neither depends on high-resolution imagery nor is prone to be negatively affected by seasonal changes. The proposed approach is simple and mainly based on the utilization of Sentinel-2 and existing corresponding building footprint information. We discuss in detail all the necessary steps to produce sub-resolution CD maps and shine a light on the critical influence of georeferencing correction.

Precipitation nowcasting (up to a few hours) remains a challenge due to the highly complex local interactions that need to be captured accurately. Convolutional Neural Networks rely on convolutional kernels convolving with grid data and the extracted features are trapped by limited receptive field, typically expressed in excessively smooth output compared to ground truth. Thus they lack the capacity to model complex spatial relationships among the grids. Geometric deep learning aims to generalize neural network models to non-Euclidean domains. Such models are more flexible in defining nodes and edges and can effectively capture dynamic spatial relationship among geographical grids. Motivated by this, we explore a geometric deep learning-based temporal Graph Convolutional Network (GCN) for precipitation nowcasting. The adjacency matrix that simulates the interactions among grid cells is learned automatically by minimizing the L1 loss between prediction and ground truth pixel value during the training procedure. Then, the spatial relationship is refined by GCN layers while the temporal information is extracted by 1D convolution with various kernel lengths. The neighboring information is fed as auxiliary input layers to improve the final result. We test the model on sequences of radar reflectivity maps over the Trento/Italy area. The results show that GCNs improves the effectiveness of modeling the local details of the cloud profile as well as the prediction accuracy by achieving decreased error measures.

Out-of-distribution (OOD) detection is an emerging research topic in remote sensing where existing works focus on single sensor analysis. However, many remote sensing works use multi-modal data to benefit from different characteristics of the sensors. Data that is in-domain for one sensor may be OOD for another sensor. In this work, we address such a scenario focusing on Synthetic Aperture Radar (SAR) and optical data fusion for multi-label scene classification. Besides data distribution shifts caused by unknown classes and snow, we also consider cases where only one modality is affected. Optical images acquired with significant cloud coverage are considered as OOD, while their corresponding SAR images can be in-distribution. We propose a weighted feature propagation strategy based on the in-distribution probabilities of the single modalities. We show, that we not only improve the prediction performance on the cloudy samples but also receive a higher predictive uncertainty when both modalities are OOD.

Distributional mismatch between training and test data may cause the remote sensing models to behave in unpredictable manner, thus reducing the trustworthiness of such models. Most existing methods for out-of-distribution (OOD) detection rely on availability of OOD samples during training. However, access to OOD data during training is counter intuitive and may be impractical sometimes. Considering this, we propose an unsupervised OOD detection model that does not require training OOD data. The proposed method works by projecting the in-domain samples as a union of 1-dimensional subspaces. Due to the compact feature representation of in-domain samples, OOD samples are less likely to occupy the same feature space, thus they are easily identified. Experimental results demonstrate the capability of the proposed method to detect OOD samples.

Distribution shift may pose significant challenges in Earth observation, especially when dealing with significantly differ-ent sensors like multispectral optical and Synthetic Aperture Radar (SAR). Deep learning models trained for optical image classification generally do not generalize well for SAR images. This is due to very marked differences between them. Though there is a considerable amount of works on domain adaptation, only few deal with such strong differences. Towards this, we propose a co-teaching based domain adaptation method using dual classifier head, a Multi-layer Perceptron (MLP) classi-fier and a Graph Neural Network (GNN) classifier. The two classifier heads teach each other in an iterative manner, thus gradually adapting both of them for target classification. We experimentally demonstrate the efficacy of the proposed approach on Sentinel 2 (optical) as source and Sentinel 1 (SAR) images as target - both product of Copernicus program of European Space Agency.

High-resolution remote sensing images are now available with the progress of remote sensing technology. With respect to popular remote sensing tasks like scene classification, image captioning provides comprehensible information about such images by summarizing the image content in human-readable text. Most existing remote sensing image captioning methods are based on deep learning-based encoder-decoder frameworks, using Convolutional Neural Network or Recurrent Neural Network as the backbone of such frameworks. Such frameworks show a limited capability to analyze sequential data and cope with the lack of captioned remote sensing training images. Recently introduced Transformer architecture exploits self-attention to obtain superior performance for sequence-analysis tasks. Inspired by this, in this work, we employ a Transformer as an encoder-decoder for remote sensing image captioning. Moreover, to deal with the limited training data, an auxiliary decoder is used that further helps the encoder in the training process. The auxiliary decoder is trained for multi-label scene classification due to its conceptual similarity to image captioning and capability of highlighting semantic classes. To the best of our knowledge, this is the first work exploiting multi-label classification to improve remote sensing image captioning. Experimental results on the UC Merced caption data set show the efficacy of the proposed method. The implementation details can be found in https://gitlab.lrz.de/ai4eo/captioningMultilabel.

Landslide inventory mapping (LIM) plays an important role in landslide susceptibility analysis. Many LIM approaches based on change detection techniques have been proposed, but with various drawbacks. For example, existing approaches have limited capability to capture the objects of varying shapes/sizes present in an area impacted by landslide. Many existing approaches are supervised and require parameter tuning. Moreover, some methods are prone to salt-and-pepper noise. To overcome these limitations, in this letter, an algorithm based on automatic adaptive region extension using very-high-resolution remote sensing images is developed. First, a simple yet effective k-means clustering method is used to generate training samples for landslide and nonlandslide classes, which refer to changed and unchanged areas, respectively. Second, an automatic adaptive region extension algorithm is developed and applied to each pixel of the postevent image, and the label of an extended region around a pixel is determined by the nearest distance between the central pixel and the changed or unchanged samples. Finally, the labels of a pixel are recorded because a pixel in different adaptive regions may be reassigned dissimilar labels, and the final label of the pixel is consistent with its maximum assigned label. To verify the performance of the proposed approach, we conducted experiments on two different landslide sites with VHR remote sensing images in Lantau Island, Hong Kong, China. Experimental results clearly demonstrate that the proposed approach has several advantages in improving the performance of LIM with VHR remote sensing images.

Deep learning-based unsupervised change detection (CD) methods compare a prechange and a postchange image in deep feature space and require precise knowledge of the event date for selecting proper pre-/post-change images. However, in many applications changes may occur gradually over a span of time making pre-/post-dates difficult to establish or prior knowledge of event date is unknown. On the other hand, deep learning-based time-series analysis methods are generally supervised. Considering such scenarios, we propose a novel unsupervised deep learning-based method to detect changes in an image time-series. The method does not make any assumption on the date of the occurrence of the change event. It treats CD as an anomaly detection problem by exploiting multilayer long short term memory (LSTM) network to learn a representation of the time series. The proposed method ingests a shuffled time series and uses an encoder–decoder LSTM model to rearrange the input sequence in correct order. While the model fails to rearrange the changed pixels, unchanged data can be rearranged in the correct order. This enables the identification of the changed pixels. To show the effectiveness of the proposed method, we tested it on two multitemporal Sentinel-1 data sets over Brumadinho, Brazil, and Bhavanisagar, India.

Change detection (CD) is critical for analyzing data collected by planetary exploration missions, e.g., for identification of new impact craters. However, CD is still a relatively new topic in the context of planetary exploration. Sheer variation of planetary data makes CD much more challenging than in the case of Earth observation (EO). Unlike CD for EO, patch-level decision is preferred in planetary exploration as it is difficult to obtain perfect pixelwise alignment/coregistration between the bi-temporal planetary images. Lack of labeled bi-temporal data impedes supervised CD. To overcome these challenges, we propose an unsupervised CD method that exploits a pretrained feature extractor to obtain bi-temporal deep features that are further processed using global max-pooling to obtain patch-level feature description. Bi-temporal patch-level features are further analyzed based on difference to determine whether a patch is changed. Additionally, a self-supervised method is proposed to estimate the decision boundary between the changed and unchanged patches. Experimental results on three planetary CD datasets from two different planetary bodies (Mars and Moon) demonstrate that the proposed method often outperforms supervised planetary CD methods. Code is available athttps://gitlab.lrz.de/ai4eo/cd/-/tree/main/planetaryCDUnsup.

Remote sensing deals with huge variations in geography, acquisition season, and a plethora of sensors. Considering the difficulty of collecting labeled data uniformly representing all scenarios, data-hungry deep learning models are often trained with labeled data in a source domain that is limited in the above-mentioned aspects. Domain adaptation (DA) methods can adapt such model for applying on target domains with different distributions from the source domain. However, most remote sensing DA methods are designed for single-target, thus requiring a separate target classifier to be trained for each target domain. To mitigate this, we propose multitarget DA in which a single classifier is learned for multiple unlabeled target domains. To build a multitarget classifier, it may be beneficial to effectively aggregate features from the labeled source and different unlabeled target domains. Toward this, we exploit coteaching based on the graph neural network that is capable of leveraging unlabeled data. We use a sequential adaptation strategy that first adapts on the easier target domains assuming that the network finds it easier to adapt to the closest target domain. We validate the proposed method on two different datasets, representing geographical and seasonal variation. Code is available athttps://gitlab.lrz.de/ai4eo/da-multitarget-gnn/.

The use of deep learning (DL) methods for change detection (CD) is currently dominated by supervised models that require a large number of labeled samples. However, these samples are difficult to acquire in the multitemporal case. A possible alternative is leveraging methods that exploit transfer learning for CD by reusing DL models pretrained for other tasks. However, the performance of the transfer-learning-based models decreases as much as the target images differ from the ones used for training the model. To overcome this limit, we propose an unsupervised CD method that exploits multiresolution deep feature maps derived by a convolutional autoencoder (CAE). It automatically learns spatial features from the input during the training phase without requiring any labeled data. The proposed method processes the bitemporal images to obtain and compare multiresolution bitemporal feature maps. These feature maps are then analyzed by a feature-selection technique to select the most discriminant ones. Furthermore, an aggregated multiresolution difference image is computed and used for a detail-preserving multiscale CD. In the context of this CD approach, we propose two alternative strategies to retrieve multiscale reliability maps. We tested the proposed method on bitemporal multispectral images acquired by Landsat-5 and Landsat-8 representing burned areas and Sentinel-2 images representing deforested areas. Results confirm the effectiveness of the proposed CD technique.

Remote sensing deals with a plethora of sensors, a large number of classes/categories, and a huge variation in geography. Due to the difficulty of collecting labeled data uniformly representing all scenarios, data-hungry deep learning models are often trained with labeled data in a source domain that is limited in the above-mentioned aspects. However, during the test/inference phase, such deep learning models are often subjected to a distributional shift, also called out-of-distribution (OOD) samples, in the form of unseen classes, geographic differences, and multisensor differences. Deep learning models can behave in an unexpected manner when subjected to such distributional uncertainties. Vulnerability to OOD data severely reduces the reliability of deep learning models and trusting on such predictions in the absence of any reliability indicator may lead to wrong policy decisions or mishaps in time-bound remote sensing applications. Motivated by this, in this work, we propose a Dirichlet prior network-based model to quantify the distributional uncertainty of deep learning-based remote sensing models. The approach seeks to maximize the representation gap between the in-domain and OOD examples for better segregation of OOD samples at test time. Extensive experiments on several remote sensing image classification datasets demonstrate that the proposed model can quantify distributional uncertainty. To the best of our knowledge, this is the first work to elaborately study distributional uncertainty in context of remote sensing. The codes are publicly available athttps://gitlab.lrz.de/ai4eo/Uncertainty/-/tree/main/DPN-RS.

Change detection is a long-standing and challenging problem in remote sensing. Very often, features about changes are difficult to model beforehand, thus making the collection of changed samples a challenging task. In comparison, it is much easier to collect numerous no-change samples. It is possible to define a change detection approach by using only easily available annotated no-change samples, which we henceforth call one-class change detection. Autoencoder networks being trained on no-change data are natural candidates for addressing this task due to their superior performance as compared to other one-class classification models. However, the autoencoders usually suffer from the problem of overgeneralization, i.e., they tend to generalize too well, thus risking properly reconstructing changed samples. In this paper, we propose a novel data-enclosing-ball minimizing autoencoder (DebM-AE) that is trained with dual objectives—a reconstruction error criterion and a minimum volume criterion. The network learns a compact latent space, where encodings of no-change samples have low intra-class variance, which as counter part has the identification of changed instances. We conducted extensive experiments on three real-world data sets. Results demonstrate advantages of the proposed method over other competitors. We make our data and code publicly available1.

Band selection refers to the process of choosing the most relevant bands in a hyperspectral image. By selecting a limited number of optimal bands, we aim at speeding up model training, improving accuracy, or both. It reduces redundancy among spectral bands while trying to preserve the original information of the image. By now, many efforts have been made to develop unsupervised band selection approaches, of which the majorities are heuristic algorithms devised by trial and error. In this article, we are interested in training an intelligent agent that, given a hyperspectral image, is capable of automatically learning policy to select an optimal band subset without any hand-engineered reasoning. To this end, we frame the problem of unsupervised band selection as a Markov decision process, propose an effective method to parameterize it, and finally solve the problem by deep reinforcement learning. Once the agent is trained, it learns a band-selection policy that guides the agent to sequentially select bands by fully exploiting the hyperspectral image and previously picked bands. Furthermore, we propose two different reward schemes for the environment simulation of deep reinforcement learning and compare them in experiments. This, to the best of our knowledge, is the first study that explores a deep reinforcement learning model for hyperspectral image analysis, thus opening a new door for future research and showcasing the great potential of deep reinforcement learning in remote sensing applications. Extensive experiments are carried out on four hyperspectral data sets, and experimental results demonstrate the effectiveness of the proposed method. The code is publicly available.

Most change detection (CD) methods assume that prechange and postchange images are acquired by the same sensor. However, in many real-life scenarios, e.g., natural disasters, it is more practical to use the latest available images before and after the occurrence of incidence, which may be acquired using different sensors. In particular, we are interested in the combination of the images acquired by optical and synthetic aperture radar (SAR) sensors. SAR images appear vastly different from the optical images even when capturing the same scene. Adding to this, CD methods are often constrained to use only target image-pair, no labeled data, and no additional unlabeled data. Such constraints limit the scope of traditional supervised machine learning and unsupervised generative approaches for multisensor CD. The recent rapid development of self-supervised learning methods has shown that some of them can even work with only few images. Motivated by this, in this work, we propose a method for multisensor CD using only the unlabeled target bitemporal images that are used for training a network in a self-supervised fashion by using deep clustering and contrastive learning. The proposed method is evaluated on four multimodal bitemporal scenes showing change, and the benefits of our self-supervised approach are demonstrated. Code is available athttps://gitlab.lrz.de/ai4eo/cd/-/tree/main/sarOpticalMultisensorTgrs2021.

Earth observation data has huge potential to enrich our knowledge about our planet. An important step in many Earth observation tasks is semantic segmentation. Generally, a large number of pixelwise labeled images are required to train deep models for supervised semantic segmentation. On the contrary, strong inter-sensor and geographic variations impede the availability of annotated training data in Earth observation. In practice, most Earth observation tasks use only the target scene without assuming availability of any additional scene, labeled or unlabeled. Keeping in mind such constraints, we propose a semantic segmentation method that learns to segment from a single scene, without using any annotation. Earth observation scenes are generally larger than those encountered in typical computer vision datasets. Exploiting this, the proposed method samples smaller unlabeled patches from the scene. For each patch an alternate view is generated by simple transformations, e.g., addition of noise. Both views are then processed through a two-stream network and weights are iteratively refined using deep clustering, spatial consistency, and contrastive learning in the pixel space. The proposed model automatically segregates the major classes present in the scene and produces the segmentation map. Extensive experiments on four Earth observation datasets collected by different sensors show the effectiveness of the proposed method. Implementation is available at https://gitlab.lrz.de/ai4eo/cd/-/tree/main/unsupContrastiveSemanticSeg.

The current remote sensing (RS) open data policy for multispectral (MS) missions such as Sentinel-2 and Landsat-8, together with the availability of free cloud distributed processing platforms such as Google Earth Engine, makes it possible the quick generation of burned area (BA) products even for nonexperts in the field. Indeed, fires and BAs can be detected using burn severity indices, which are usually obtained by simple band algebra operations. However, simple approaches can aid BA estimation only if typical error patterns are known and accounted for, especially when working at large (e.g., continental) scales. This article proposes an automatic BA detection system based on burn severity index thresholding, which integrates dedicated false and missed alarm mitigation strategies to improve the detection accuracy. The system is tested on Sentinel-2 and Landsat-8 data over ten different locations in Europe and spanning year 2018. Three known burn severity indices plus a custom one defined to improve the performance in the considered study area are under study. Results show that burned index thresholding is possible within accuracy bounds slightly larger than the state of the art, which is acceptable by considering the proposed simplified processing framework.

In remote sensing, distributional mismatch between the training and test data may arise due to several reasons, including unseen classes in the test data, differences in the geographic area, and multi-sensor differences. Deep learning based models may behave in unexpected manners when subjected to test data that has such distributional shifts from the training data, also called out-of-distribution (OOD) examples. Vulnerability to OOD data severely reduces the reliability of deep learning based models. In this work, we address this issue by proposing a model to quantify distributional uncertainty of deep learning based remote sensing models. In particular, we adopt a Dirichlet Prior Network for remote sensing data. The approach seeks to maximize the representation gap between the in-domain and OOD examples for a better identification of unknown examples at test time. Experimental results on three exemplary test scenarios show that the proposed model can detect OOD images in remote sensing.

Change detection (CD) is one of the most researched areas in remote sensing. However, most CD methods assume that the pre-change and post-change images are acquired by the same sensor, having the same set of spectral bands and same spatial resolution. This severely limits the applicability of CD methods. It is not trivial to apply the existing CD methods in multisensor scenario. Towards this direction, we propose an unsupervised CD method that can handle large differences in spatial resolution and can work with completely different set of spectral bands. The proposed method uses a self-supervised super-resolution strategy to upsample the lower resolution image, thus mitigating differences in spatial resolution. To mitigate spectral differences, a self-supervised learning strategy is used that ingests both images as input and trains a network using self-supervised loss accounting for the spectral differences in both images. Once trained this network is used in deep change vector analysis framework for change detection. We validated the proposed method in an experimental setup where the pre-change and post-change images have different spatial resolution (10m and 20 m/pixel) and completely disjoint set of spectral bands.

Deep learning (DL) based supervised change detection (CD) models require large labeled training data. Due to the difficulty of collecting labeled multi-temporal data, unsupervised methods are preferred in the CD literature. However, unsupervised methods cannot fully exploit the potentials of data-driven deep learning and thus they are not absolute alternative to the supervised methods. This motivates us to look deeper into the supervised DL methods and investigate how they can be adopted intelligently for CD by minimizing the requirement of labeled training data. Towards this, in this work we show that geographically diverse training dataset can yield significant improvement over less diverse training datasets of the same size. We propose a simple confidence indicator for verifying the trustworthiness/confidence of supervised models trained with small labeled dataset. Moreover, we show that for the test cases where supervised CD model is found to be less confident/trustworthy, unsupervised methods often produce better result than the supervised ones.

To overcome the limited capability of most state-of-the-art change detection (CD) methods in modeling spatial context of multispectral high spatial resolution (HR) images and exploiting all spectral bands jointly, this letter presents a novel unsupervised deep-learning-based CD method that can effectively model contextual information and handle the large number of bands in multispectral HR images. This is achieved by exploiting all spectral bands after grouping them into spectral-dedicated band groups. To eliminate the necessity of multitemporal training data, the proposed method exploits a data set targeted for image classification to train spectral-dedicated Auxiliary Classifier Generative Adversarial Networks (ACGANs). They are used to obtain pixelwise deep change hypervector from multitemporal images. Each feature in deep change hypervector is analyzed based on the magnitude to identify changed pixels. An ensemble decision fusion strategy is used to combine change information from different features. Experimental results on the urban, Alpine, and agricultural Sentinel-2 data sets confirm the effectiveness of the proposed method.

Most change detection (CD) methods are unsupervised as collecting substantial multitemporal training data is challenging. Unsupervised CD methods are driven by heuristics and lack the capability to learn from data. However, in many real-world applications, it is possible to collect a small amount of labeled data scattered across the analyzed scene. Such a few scattered labeled samples in the pool of unlabeled samples can be effectively handled by graph convolutional network (GCN) that has recently shown good performance in semisupervised single-date analysis, to improve change detection performance. Based on this, we propose a semisupervised CD method that encodes multitemporal images as a graph via multiscale parcel segmentation that effectively captures the spatial and spectral aspects of the multitemporal images. The graph is further processed through GCN to learn a multitemporal model. Information from the labeled parcels is propagated to the unlabeled ones over training iterations. By exploiting the homogeneity of the parcels, the model is used to infer the label at a pixel level. To show the effectiveness of the proposed method, we tested it on a multitemporal Very High spatial Resolution (VHR) data set acquired by Pleiades sensor over Trento, Italy.

Building change detection (CD), important for its application in urban monitoring, can be performed in near real time by comparing prechange and postchange very-high-spatial-resolution (VHR) synthetic-aperture-radar (SAR) images. However, multitemporal VHR SAR images are complex as they show high spatial correlation, prone to shadows, and show an inhomogeneous signature. Spatial context needs to be taken into account to effectively detect a change in such images. Recently, convolutional-neural-network (CNN)-based transfer learning techniques have shown strong performance for CD in VHR multispectral images. However, its direct use for SAR CD is impeded by the absence of labeled SAR data and, thus, pretrained networks. To overcome this, we exploit the availability of paired unlabeled SAR and optical images to train for the suboptimal task of transcoding SAR images into optical images using a cycle-consistent generative adversarial network (CycleGAN). The CycleGAN consists of two generator networks: one for transcoding SAR images into the optical image domain and the other for projecting optical images into the SAR image domain. After unsupervised training, the generator transcoding SAR images into optical ones is used as a bitemporal deep feature extractor to extract optical-like features from bitemporal SAR images. Thus, deep change vector analysis (DCVA) and fuzzy rules can be applied to identify changed buildings (new/destroyed). We validate our method on two data sets made up of pairs of bitemporal VHR SAR images on the city of L'Aquila (Italy) and Trento (Italy).

Very-high-resolution (VHR) multi-temporal images are important in remote sensing to monitor the dynamics of the Earth surface. Image semantic segmentation classifies pixels and assigns them label from meaningful object groups. It has been extensively studied in context of single image analysis, however not explored for multi-temporal one. In this paper we propose to extend supervised semantic segmentation to the unsupervised joint segmentation of multi-temporal images. The proposed method processes multi-temporal images by separately feeding them to a deep network comprising of trainable convolutional layers. The training process does not involve any external label. Segmentation labels are obtained from argmax classification of the final layer. Multi-temporal segmentation labels and weights of the trainable layers are jointly optimized in iterations. We tested the method on a VHR dataset from Trento, Italy. Both quantitative and qualitative results demonstrated the effectiveness of the proposed approach.

High/very-high-resolution (HR/VHR) multitemporal images are important in remote sensing to monitor the dynamics of the Earth's surface. Unsupervised object-based image analysis provides an effective solution to analyze such images. Image semantic segmentation assigns pixel labels from meaningful object groups and has been extensively studied in the context of single-image analysis, however not explored for multitemporal one. In this article, we propose to extend supervised semantic segmentation to the unsupervised joint semantic segmentation of multitemporal images. We propose a novel method that processes multitemporal images by separately feeding to a deep network comprising of trainable convolutional layers. The training process does not involve any external label, and segmentation labels are obtained from the argmax classification of the final layer. A novel loss function is used to detect object segments from individual images as well as establish a correspondence between distinct multitemporal segments. Multitemporal semantic labels and weights of the trainable layers are jointly optimized in iterations. We tested the method on three different HR/VHR data sets from Munich, Paris, and Trento, which shows the method to be effective. We further extended the proposed joint segmentation method for change detection (CD) and tested on a VHR multisensor data set from Trento.

Change Detection (CD) using multitemporal satellite images is an important application of remote sensing. In this work, we propose a Convolutional-Neural-Network (CNN) based unsupervised multiple-change detection approach that simultaneously accounts for the high spatial correlation among pixels in Very High spatial Resolution (VHR) images and the differences in multisensor images. We accomplish this by learning in an unsupervised way a transcoding between multisensor multitemporal data by exploiting a cycle-consistent Generative Adversarial Network (CycleGAN) that consists of two generator CNN networks. After unsupervised training, one generator of the CycleGAN is used to mitigate multisensor differences, while the other is used as a feature extractor that enables the computation of multitemporal deep features. These features are then compared pixelwise to generate a change detection map. Changed pixels are then further analyzed based on multitemporal deep features for identifying different kind of changes (multiple-change detection). Results obtained on multisensor multitemporal dataset consisting of Quickbird and Pleiades images confirm the effectiveness of the proposed approach.

Change detection (CD) in multitemporal images is an important application of remote sensing. Recent technological evolution provided very high spatial resolution (VHR) multitemporal optical satellite images showing high spatial correlation among pixels and requiring an effective modeling of spatial context to accurately capture change information. Here, we propose a novel unsupervised context-sensitive framework-deep change vector analysis (DCVA)-for CD in multitemporal VHR images that exploit convolutional neural network (CNN) features. To have an unsupervised system, DCVA starts from a suboptimal pretrained multilayered CNN for obtaining deep features that can model spatial relationship among neighboring pixels and thus complex objects. An automatic feature selection strategy is employed layerwise to select features emphasizing both high and low prior probability change information. Selected features from multiple layers are combined into a deep feature hypervector providing a multiscale scene representation. The use of the same pretrained CNN for semantic segmentation of single images enables us to obtain coherent multitemporal deep feature hypervectors that can be compared pixelwise to obtain deep change vectors that also model spatial context information. Deep change vectors are analyzed based on their magnitude to identify changed pixels. Then, deep change vectors corresponding to identified changed pixels are binarized to obtain a compressed binary deep change vectors that preserve information about the direction (kind) of change. Changed pixels are analyzed for multiple CD based on the binary features, thus implicitly using the spatial information. Experimental results on multitemporal data sets of Worldview-2, Pleiades, and Quickbird images confirm the effectiveness of the proposed method.

Change Detection (CD) using multi-temporal satellite images is a fundamental application of remote sensing. To effectively capture change information from Very High spatial Resolution (VHR) optical images, spatial context needs to be modelled as VHR images are characterized by high spatial correlation among pixels. We propose a context-sensitive framework for CD in multitemporal VHR images using pre-trained Convolutional-Neural-Network (CNN)-based feature extraction. Such a framework, while unsupervised, can effectively model the spatial relationship among neighbouring pixels in VHR images. A CNN, pre-trained for semantic segmentation, enables us to obtain multi-temporal deep features that are compared pixelwise to identify changed pixels. Changed pixels are further clustered for multiple change detection. Results obtained on multi-temporal datasets of Worldview-2 and Pleiades images demonstrate effectiveness of our approach.