Local spectral features and global spatial context are essential for hyperspectral image (HSI) classification. However, existing methods based on convolutional neural networks (CNNs), graph convolutional networks (GCNs), and Transformers often rely on multibranch structures to separately extract and fuse local and global features, resulting in high computational complexity and redundant information that can negatively affect classification performance. To address these issues, we propose a two-stage graph convolutional mamba network (TGMN) that enables efficient modeling of local and global features through sequential intrasubgraph local feature extraction and intersubgraph global information learning. Specifically, in the first stage, we partition the HSI into superpixel regions and treat each superpixel as a subgraph, where a GCN is applied to aggregate spectral-spatial features within each subgraph. We further design a downsampled subgraph feature reconstruction (DSFR) module that dynamically selects key nodes to reduce redundancy, highlight critical features, and enhance model representation capability. In the second stage, the Mamba network models the global dependencies between subgraphs and introduces a region-relation aware absolute positional encoding (RAPE) module. This module encodes spatial positional information into embedded vectors by integrating the relative distance and direction between the geometric center of each superpixel and the image center, which are then deeply fused with the feature matrix to improve spatial relationship comprehension. The two-stage sequential structure ensures effective local and global feature extraction, avoiding the high computational complexity and redundancy issues commonly associated with multibranch models. Experiments on three benchmark datasets demonstrate its superiority, achieving classification accuracies of 98.54%, 98.30%, and 96.94% on the Indian Pines, Dioni, and Honghu datasets, respectively. Compared to state-of-the-art methods, TGMN achieves higher classification accuracy with significantly lower computational cost, demonstrating its efficiency and effectiveness for HSI classification.

Large vision-language models (VLMs) have made great achievements in Earth vision. However, complex disaster scenes with diverse disaster types, geographic regions, and satellite sensors have posed new challenges for VLM applications. To fill this gap, we curate the first remote sensing vision-language dataset (DisasterM3) for global-scale disaster assessment and response. DisasterM3 includes 26,988 bi-temporal satellite images and 123k instruction pairs across 5 continents, with three characteristics: **1) Multi-hazard**: DisasterM3 involves 36 historical disaster events with significant impacts, which are categorized into 10 common natural and man-made disasters. **2) Multi-sensor**: Extreme weather during disasters often hinders optical sensor imaging, making it necessary to combine Synthetic Aperture Radar (SAR) imagery for post-disaster scenes. **3) Multi-task**: Based on real-world scenarios, DisasterM3 includes 9 disaster-related visual perception and reasoning tasks, harnessing the full potential of VLM's reasoning ability with progressing from disaster-bearing body recognition to structural damage assessment and object relational reasoning, culminating in the generation of long-form disaster reports. We extensively evaluated 14 generic and remote sensing VLMs on our benchmark, revealing that state-of-the-art models struggle with the disaster tasks, largely due to the lack of a disaster-specific corpus, cross-sensor gap, and damage object counting insensitivity. Focusing on these issues, we fine-tune four VLMs using our dataset and achieve stable improvements (up to 10.4\%$\uparrow$QA, 2.1$\uparrow$Report, 40.8\%$\uparrow$Referring Seg.) with robust cross-sensor and cross-disaster generalization capabilities. Project: https://github.com/Junjue-Wang/DisasterM3.

Multimodal large language models (MLLMs) have demonstrated remarkable capabilities in visual understanding, but their application to long-term Earth observation analysis remains limited, primarily focusing on single-temporal or bi-temporal imagery. To address this gap, we introduce DVL-Suite, a comprehensive framework for analyzing long-term urban dynamics through remote sensing imagery. Our suite comprises 14,871 high-resolution (1.0m) multi-temporal images spanning 42 major cities in the U.S. from 2005 to 2023, organized into two components: DVL-Bench and DVL-Instruct. The DVL-Bench includes six urban understanding tasks, from fundamental change detection (pixel-level) to quantitative analyses (regional-level) and comprehensive urban narratives (scene-level), capturing diverse urban dynamics including expansion/transformation patterns, disaster assessment, and environmental challenges. We evaluate 18 state-of-the-art MLLMs and reveal their limitations in long-term temporal understanding and quantitative analysis. These challenges motivate the creation of DVL-Instruct, a specialized instruction-tuning dataset designed to enhance models' capabilities in multi-temporal Earth observation. Building upon this dataset, we develop DVLChat, a baseline model capable of both image-level question-answering and pixel-level segmentation, facilitating a comprehensive understanding of city dynamics through language interactions. Project: https://github.com/weihao1115/dynamicvl.

Abstract In the context of Earth observation, the trade-off between spatial, spectral, and temporal resolution often limits the versatility of remote sensing images in many important applications. In response, this paper introduces a novel deep learning diffusion model, specifically tailored to improve the spatial resolution of the optical products acquired by the Sentinel-3 (S3) satellite. Our framework employs a diffusion probabilistic model, benefiting from the higher spatial resolution of the Sentinel-2 satellite during training via a new multi-modal loss formulation. This ensures consistency with the original S3 images while enhancing the spatial details. Two distinct conditional low-resolution encoders were experimented with, providing insights into their respective contributions to the diffusion process. The efficacy of the proposed model is demonstrated through extensive ablation studies and comparisons with state-of-the-art methods, using both synthetic and real S3 products. The findings indicate that our model successfully improves spatial resolution while maintaining the integrity of the spectral information, contributing to the field of remote sensing single-image super-resolution.

Motivated by the increasing demand for robust segmentation in unlabeled remote sensing data, we propose DAM-Former, a novel UDA model that fuses high-resolution multimodal imagery with multi-temporal multispectral data. Current UDA approaches in remote sensing rarely exploit the complementary strengths of spatial and temporal features. To address this gap, our framework integrates two interconnected branches: a transformer-based network for high-resolution multimodal data and a lightweight convolutional network with temporal attention for multi-temporal imagery. To improve segmentation accuracy and lower noise, the extracted features are robustly combined through a deep temporal fusion module and a new mixed loss with an ensemble pseudo-label strategy. Extensive experiments and an ablation study on the FLAIR-2 dataset demonstrate that DAM-Former outperforms state-of-the-art methods, marking the first in-depth study of temporal information fusion in UDA segmentation for remote sensing data.

Transformer-based deep learning (DL) methods have gradually been advocated for remote sensing (RS) image semantic segmentation due to the great global modeling capability. Nevertheless, Transformer-based DL methods have not yet been sufficiently explored on the large-scale hyperspectral image (HSI) semantic segmentation. Current algorithms lack a comprehensive consideration of the impact of positional encoding (PE) interpolation when constructing Transformer-based decoders. Moreover, existing segmentation heads usually directly concatenate multiscale features to achieve segmentation, which ignores the inherent semantic differences between different features. To address the above issues, a U-shaped multimixed Transformer network (UM2Former) is proposed for large-scale HSI semantic segmentation. First, a weight encoder consisting of two modules, the overlap-down and the channel-weight, is built to extract hierarchical discriminative spectral-spatial features and decrease spectral redundancy. Second, the proposed multimixed Transformer block (MMTB) develops a PE-free module, spatial-feature-retention attention (SFRA) mechanism, in which “multimixed” represents the global dependency modeling of each pixel with the retented average spatial characteristics of different locations in the input feature maps. Finally, a linear fuse segmentation head (LFSH) is designed to align semantic information among multiscale feature maps and achieve accurate segmentation. Experiments were conducted in single cities and the entire large-scale WHU-OHS HSI dataset. The segmentation results indicated that the proposed method achieved higher accuracy compared to the existing semantic segmentation methods, with performance improvements of 17.80% and 4.16% in terms of intersection over union (mIoU) and overall accuracy (OA), respectively. The source code will be available athttps://github.com/ZhaohuiXue/UM2Former.

We introduce innovative unsupervised domain adaptation (UDA) techniques that leverage the integration of DAFomer and cross-attention mechanisms, tailored to effectively handle multi-modal datasets. We investigate the methods on the FLAIR-1 dataset with different domains, including RGB, NIR bands, and height information. Experimental findings strongly support the idea that integrating inter-modal information significantly enhances segmentation accuracy, making the model more versatile and effective in handling multi-modal datasets across diverse conditions and domains.

We present the OpenEarthMap benchmark suite, designed for global high-resolution land cover mapping and change analysis, and showcase its applications. This comprehensive expansion aims to strengthen OpenEarthMap’s versatility, covering various aspects such as increasing dataset diversity through synthetic data, developing lightweight models, improving land cover change detection with OpenStreetMap, and facilitating high-resolution mapping on a national scale.

Global semantic 3D understanding from single-view high-resolution remote sensing (RS) imagery is crucial for Earth observation (EO). However, this task faces significant challenges due to the high costs of annotations and data collection, as well as geographically restricted data availability. To address these challenges, synthetic data offer a promising solution by being unrestricted and automatically annotatable, thus enabling the provision of large and diverse datasets. We develop a specialized synthetic data generation pipeline for EO and introduce SynRS3D, the largest synthetic RS dataset. SynRS3D comprises 69,667 high-resolution optical images that cover six different city styles worldwide and feature eight land cover types, precise height information, and building change masks. To further enhance its utility, we develop a novel multi-task unsupervised domain adaptation (UDA) method, RS3DAda, coupled with our synthetic dataset, which facilitates the RS-specific transition from synthetic to real scenarios for land cover mapping and height estimation tasks, ultimately enabling global monocular 3D semantic understanding based on synthetic data. Extensive experiments on various real-world datasets demonstrate the adaptability and effectiveness of our synthetic dataset and the proposed RS3DAda method. SynRS3D and related codes are available at https://github.com/JTRNEO/SynRS3D.

Learning with limited labeled data is a challenging problem in various applications, including remote sensing. Few-shot semantic segmentation is one approach that can encourage deep learning models to learn from few labeled examples for novel classes not seen during the training. The generalized few-shot segmentation setting has an additional challenge which encourages models not only to adapt to the novel classes but also to maintain strong performance on the training base classes. While previous datasets and benchmarks discussed the few-shot segmentation setting in remote sensing, we are the first to propose a generalized few-shot segmentation benchmark for remote sensing. The generalized setting is more realistic and challenging, which necessitates exploring it within the remote sensing context. We release the dataset augmenting OpenEarthMap (OEM) with additional classes labeled for the generalized few-shot evaluation setting. The dataset is released during the OEM land cover mapping generalized few-shot challenge in the learning with limited labeled data for image and video understanding (L3D-IVU) workshop in conjunction with computer vision and pattern recognition (CVPR) 2024. In this work, we summarize the dataset and challenge details in addition to providing the benchmark results on the two phases of the challenge for the validation and test sets.

Optical high-resolution imagery and OpenStreetMap (OSM) data are two important data sources of land-cover change detection (CD). Previous related studies focus on utilizing the information in OSM data to aid the CD on optical high-resolution images. This article pioneers the direct detection of land-cover changes utilizing paired OSM data and optical imagery, thereby expanding the scope of CD tasks. To this end, we propose an object-guided Transformer (ObjFormer) by naturally combining the object-based image analysis (OBIA) technique with the advanced vision Transformer architecture. This combination can significantly reduce the computational overhead in the self-attention module without adding extra parameters or layers. Specifically, ObjFormer has a hierarchical pseudo-Siamese encoder consisting of object-guided self-attention modules that extract multilevel heterogeneous features from OSM data and optical images; a decoder consisting of object-guided cross-attention modules can recover land-cover changes from the extracted heterogeneous features. Beyond basic binary CD (BCD), this article raises a new semi-supervised semantic CD (SCD) task that does not require any manually annotated land-cover labels to train semantic change detectors. Two lightweight semantic decoders are added to ObjFormer to accomplish this task efficiently. A converse cross-entropy (CCE) loss is designed to fully utilize negative samples, contributing to the great performance improvement in this task. A large-scale benchmark dataset called OpenMapCD containing 1287 map–image pairs covering 40 regions on six continents is constructed to conduct the detailed experiments. The results show the effectiveness of our methods in this new kind of CD task. In addition, case studies in two Japanese cities demonstrate the framework’s generalizability and practical potential. The code and dataset will be open-sourced inhttps://github.com/ChenHongruixuan/ObjFormer.

Convolutional neural networks (CNNs) and Transformers have made impressive progress in the field of remote sensing change detection (CD). However, both architectures have inherent shortcomings: CNN is constrained by a limited receptive field that may hinder their ability to capture broader spatial contexts, while Transformers are computationally intensive, making them costly to train and deploy on large datasets. Recently, the Mamba architecture, based on state space models (SSMs), has shown remarkable performance in a series of natural language processing tasks, which can effectively compensate for the shortcomings of the above two architectures. In this article, we explore for the first time the potential of the Mamba architecture for remote sensing CD tasks. We tailor the corresponding frameworks, called MambaBCD, MambaSCD, and MambaBDA, for binary CD (BCD), semantic CD (SCD), and building damage assessment (BDA), respectively. All three frameworks adopt the cutting-edge Visual Mamba architecture as the encoder, which allows full learning of global spatial contextual information from the input images. For the change decoder, which is available in all three architectures, we propose three spatiotemporal relationship modeling mechanisms, which can be naturally combined with the Mamba architecture and fully utilize its attribute to achieve spatiotemporal interaction of multitemporal features, thereby obtaining accurate change information. On five benchmark datasets, our proposed frameworks outperform current CNN- and Transformer-based approaches without using any complex training strategies or tricks, fully demonstrating the potential of the Mamba architecture in CD tasks. Further experiments show that our architecture is quite robust to degraded data. The source code is available at:https://github.com/ChenHongruixuan/MambaCD.

Supervised learning methods assume that training and test data are sampled from the same distribution. However, this assumption is not always satisfied in practical situations of land cover semantic segmentation when models trained in a particular source domain are applied to other regions. This is because domain shifts caused by variations in location, time, and sensor alter the distribution of images in the target domain from that of the source domain, resulting in significant degradation of model performance. To mitigate this limitation, domain generalization (DG) has gained attention as a way of generalizing from source domain features to unseen target domains. One approach is style randomization (SR), which enables models to learn domain-invariant features through randomizing styles of images in the source domain. Despite its potential, existing methods face several challenges, such as inflexible frequency decomposition, high computational and data preparation demands, slow speed of randomization, and lack of consistency in learning. To address these limitations, we propose a frequency-based optimal style mix (FOSMix), which consists of three components: 1) full mix (FM) enhances the data space by maximally mixing the style of reference images into the source domain; 2) optimal mix (OM) keeps the essential frequencies for segmentation and randomizes others to promote generalization; and 3) regularization of consistency ensures that the model can stably learn different images with the same semantics. Extensive experiments that require the model’s generalization ability, with domain shift caused by variations in regions and resolutions, demonstrate that the proposed method achieves superior segmentation in remote sensing. The source code is available athttps://github.com/Reo-I/FOSMix.

Hyperspectral and multispectral image fusion, denoted as HSI-MSI fusion, involves merging a pair of hyperspectral (HSI) and multispectral (MSI) images to generate a high spatial resolution hyperspectral image (HR-HSI). The primary challenge in HSI-MSI fusion is to find the best way to extract one-dimensional spectral features and two-dimensional (2-D) spatial features from HSI and MSI and harmoniously combine them. In recent times, coupled tensor decomposition (CTD)-based methods have shown promising performance in the fusion task. However, the tensor decompositions (TDs) used by these CTD-based methods face difficulties in extracting complex features and capturing 2-D spatial features, resulting in suboptimal fusion results. To address these issues, we introduce a novel method called Coupled Tensor Double-Factor Decomposition (CTDF). Specifically, we propose a Tensor Double-Factor (TDF) decomposition, representing a 3rd-order HR-HSI as a 4th-order spatial factor and a 3rd-order spectral factor, connected through tensor contraction. Compared to other TDs, the TDF has better feature extraction capability since it has a higher order factor than that of HR-HSI, whereas the other TDs only have the same order factor as the HR-HSI. Moreover, the TDF can extract 2-D spatial features using the 4th-order spatial factor. We apply the TDF to the HSI-MSI fusion problem and formulate the CTDF model. Furthermore, we design an algorithm based on proximal alternating minimization to solve this model and provide insights into its computational complexity and convergence analysis. The simulated and real experiments validate the effectiveness and efficiency of the proposed CTDF method. The code is available at https://github.com/tingxu113/CTDF.

Remote sensing image change detection (CD) is crucial for disaster assessment, land use change, and urban management. Most CD methods are realized by CNN and Transformer. However, these methods are not satisfied with modeling global dependencies while keeping a low computational complexity. Recently, the emergence of Mamba architectures based on state space models (SSMs) can remedy the above problems. In this article, we propose a visual Mamba-based multiscale feature extraction network to efficiently interactively fuse global and local information, which is named as MF-VMamba (MF: multiscale feature). First, a VMamba-based encoder is used to extract multiscale semantic features from bitemporal images. Then, a feature enhancement module (FEM) is proposed to capture the difference information between images. In addition, we employ a multilevel attention decoder (MAD) based on large kernel convolution (LKC) to obtain the information in spatial and spectral dimensions to realize the information interaction between global and local features. After the sequential processing of these three modules, the discriminative ability of changing objects is significantly improved. Notably, the computational complexity of our VMamba-based model grows linearly, which can significantly reduce the computational cost. In the experiments, our method performs well on CDD, DSIFN-CD, LEVIR-CD, and SYSU-CD datasets, with$F1$scores and OA reaching$95.69\%/88.05\%/90.64\%{/86.95\%}$and$98.97\%/96.01\%/99.07\%{/90.75\%}$, respectively. The code can be accessed athttps://github.com/121zzy/MF-Mamba.git.

We introduce OpenEarthMap, a benchmark dataset, for global high-resolution land cover mapping. OpenEarth-Map consists of 2.2 million segments of 5000 aerial and satellite images covering 97 regions from 44 countries across 6 continents, with manually annotated 8-class land cover labels at a 0.25–0.5m ground sampling distance. Se-mantic segmentation models trained on the OpenEarth-Map generalize worldwide and can be used as off-the-shelf models in a variety of applications. We evaluate the performance of state-of-the-art methods for unsupervised domain adaptation and present challenging problem settings suitable for further technical development. We also investigate lightweight models using automated neural architecture search for limited computational resources and fast mapping. The dataset is available at https: //open-earth-map.org.

Compared to standard deep convolutional neural networks (CNN) which include a global average pooling operator, second-order neural networks have a global covariance pooling operator which allows to capture richer statistics of CNN features. They have been shown to improve representation and generalization abilities. However, this covariance pooling is performed only on the deepest CNN feature maps. To benefit from different levels of abstraction, we propose to extend these models by using a multi-layer approach. In addition, to obtain better predictive performance, an end-to-end ensemble learning architecture is proposed. Experiments are conducted on four datasets and have confirmed the potential of the proposed model for various image processing applications such as remote sensing scene classification, indoor scene recognition and texture classification.

Satellite image time series classification has become a research focus with the launch of new remote sensing sensors capable of capturing images with high spatial, spectral, and temporal resolutions. In particular, in the field of crop classification, time dimension information is particularly important. Although some advanced machine learning algorithms, such as random forests (RFs), can achieve good results, they often ignore the time series information. To make full use of temporal and spectral information in multitemporal remote sensing images, a channel attention-based temporal convolutional network (CA-TCN) is proposed in this letter. Specifically, the proposed method is composed of two main modules: temporal convolutional network and attention block. The temporal convolutional network can capture long-range dependence by using a hierarchy of temporal convolutional filters. To capture relevant information inside the sequence and enhance the important information, the attention block is used to enhance the important features in the channel dimension since not all bands contain equal information in crop type classification. The proposed CA-TCN can excavate deeper phenological characteristics. Compared to the temporal attention-based temporal convolutional network and other deep learning-based models, the proposed CA-TCN has achieved state-of-the-art performance in the Breizhcrops dataset with fewer parameters.

Synthetic aperture radar (SAR) images provide all-weather and all-time capabilities for Earth observation, which becomes highly beneficial in the field of intelligent remote sensing (RS) image interpretation. Due to these advantages, SAR images have been widely exploited in automatic building segmentation tasks under poor weather conditions, especially when disasters happen. However, compared to optical images, the semantics inherent to SAR images are less rich and interpretable due to factors such as speckle noise and imaging geometry. In this scenario, most state-of-the-art methods are focused on designing advanced network architectures or loss functions for building footprint extraction. However, few works have been oriented toward improving segmentation performance through knowledge transfer from optical images. In this article, we propose a novel method based on theDisOptNetnetwork, which can distill the useful semantic knowledge from optical images into a network only trained with SAR data. Specifically, we first analyze the multilevel feature discrepancies between multiple stages of the networks pretrained on the two image modalities. We observe that feature discrepancies start to increase as the encoding stage gradually changes from low level to high level. Based on such observation, we reuse the early stage features and construct parallel convolutional neural network (CNN) branches that are responsible for capturing high-level domain-specific knowledge for each image modality. The optical branch is aimed at mimicking feature generation at the optical pretrained network given the input SAR images. Then, an aggregation module is introduced to calibrate and fuse the features from different modalities while generating the building segments. Extensive experiments were conducted on a large-scale multisensor all-weather building segmentation dataset with state-of-the-art methods used for comparison. Our experimental results validate the effectiveness ofDisOptNet, which demonstrates great potential in the task of weather-independent building footprint generation under real scenarios. The codes of this article will be made publicly available athttps://github.com/jiankang1991/TGRS_DisOptNet.

This work critically analyzes the problems arising from differ-modality building semantic segmentation in the remote sensing domain. With the growth of multimodality datasets, such as optical, synthetic aperture radar (SAR), light detection and ranging (LiDAR), and the scarcity of semantic knowledge, the task of learning multimodality information has increasingly become relevant over the last few years. However, multimodality datasets cannot be obtained simultaneously due to many factors. Assume that we have SAR images with reference information in one place and optical images without reference in another; how to learn relevant features of optical images from SAR images? We refer to it as differ-modality learning (DML). To solve the DML problem, we propose novel deep neural network architectures, which include image adaptation, feature adaptation, knowledge distillation, and self-training (SL) modules for different scenarios. We test the proposed methods on the differ-modality remote sensing datasets (very high-resolution SAR and RGB from SpaceNet 6) to build semantic segmentation and to achieve a superior efficiency. The presented approach achieves the best performance when compared with the state-of-the-art methods.

Disasters such as earthquakes, hurricanes, and flooding are responsible for large-scale infrastructure damages and loss of human lives. Immediately after disaster strikes, one of the most critical and difficult tasks is accurately assessing the extent and severity of the disaster. This task is especially challenging in areas isolated by the disaster; in such cases, remote sensing information provides the best alternative to tackle this problem. This paper presents a damage mapping framework using remote sensing imagery acquired from previous disasters. The proposed deep learning-based framework is trained to learn features related to building damage using imagery from previous disasters that were collected from different regions around the world. Then, it is tested to recognize damage from a different urban environment.

To fuse hyperspectral and Light Detection And Ranging (LiDAR), we propose a semisupervised graph fusion (SSGF) approach. We apply morphological filters to LiDAR and the first few components of hyperspectral data to model the height and spatial information, respectively. Then, the proposed SSGF is used to project the spectral, elevation, and spatial features onto a lower subspace to obtain the new features. In particular, the objective of SSGF is to maximize the class separation ability and preserve the local neighborhood structure by using both labeled and unlabeled samples. Experimental results on the hyperspectral and LiDAR data from the 2013 IEEE Geoscience and Remote Sensing Society (GRSS) Data Fusion Contest demonstrated the superiority of the SSGF.

In this work, a novel deep rotation forest is proposed to fuse hyperspectral (HS) and LiDAR. First, we extract the spatial and elevation information of two datasets by using morphological filters. Then, each feature source is applied to superpixel segmentation and then are treated as the input of deep rotation forest. In the deep rotation forest, the spatial relationships are fully considered, and the output probability of each layer is used as the input of the next layer. Experimental results demonstrate that the excellent performance of the proposed method.

In this paper, we propose a novel ensemble margin based semi-supervised random forest (EMRF) algorithm for the classification of the hyperspectral image with limited training data. The proposed method tries to improve the effectiveness of the ensemble model via adaptively labeling the unlabeled instances with high classification probability then adding them into the training set. The classification probability of a training instance is reflected by the unsupervised margin value of this instance. The higher ensemble margin of an instance, the higher probability the instance being classified correctly and added into to the training set in the next iteration.

This paper presents a novel application of remote sensing data and machine learning technologies for damage classification in a real-world cross-domain application. The proposed methodology trains models to learn the building damage characteristics recorded in the 2011 Tohoku Tsunami from multi-sensor and multi-temporal remote sensing images. Then, the trained models are tested in the recent 2018 Sulawesi Tsunami. Additionally, a simulation of high-resolution SAR image was carried to deal with missing data modality. Our initial results show that the ResNet-derived features from optical images acquired after the disaster together with moderate- and high-resolution synthetic aperture radar (SAR) post-event intensity data showed significant accuracy in classifying two levels of tsunami-induced damage, with an average f-score of approximately 0.72. Taking into account that no training data from the 2018 Sulawesi Tsunami was used, our methodology shows excellent potential for future implementation of a rapid response system based on a database of building damage constructed from previous majors disasters.

We present a nonlocal low-rank denoising algorithm for synthetic aperture radar (SAR) image stacks. The method extends the widely known DespecKS algorithm by integrating low-rank approximation, outlier removal, and total variation (TV) regularization into the estimation process. Preliminary experiments shows increased robustness against outliers and comparable performance to state-of-the-art stack despeckling algorithms.

This study tested and compared the suitability of SPOT-5 image and LiDAR data both separately and combined for the classification of the urban area using Rotation Forest (ROF) classifier. Experimental results revealed that the integration of the SPOT-5 image and LiDAR data classification scheme gave better classification accuracies, when compared to the classification schedules using one of the two solely. Furthermore, RoF classifier produced better classification results than that of other two classifiers (i.e., SVMs (Support Vector Machines) and RF (Random Forests)). Finally, it should be noted that RoF classifier provided an effective way of combining SPOT-5 image and LiDAR data for classification, which is robust to the important parameter M (i.e. the number of features in each subset).

This study employed Sentinel-1A C-band and Sentinel-2A multispectral data combined with the decision tree ensemble algorithms to map the spatial distribution of five mangrove communities in a coastal area in North Vietnam. The results show that the rotation forests (RoFs) model achieved better overall accuracy and kappa coefficient in mapping mangrove species than those of the canonical correlation forests (CCFs) and the random forests (RFs) models. This research demonstrates the potential of using optical and SAR data together with machine learning techniques to map mangrove species in tropical areas.

This paper presents building damage mapping based on transfer learning techniques. Due to the different spatial resolutions of optical (WorldView, 0.5m) and SAR (Sentinel-1, 10m), we adopt different methods: pixel-level for moderate-resolution SAR images, and patch-level for very high-resolution optical images. For SAR images, the performance of fast unsupervised transfer learning methods, such as overall centroid alignment (OCA) and CORrelation ALignment (CORAL), are investigated. For the optical images, two public databases are used to predict the building damage mapping of Palu with WorldView-3 images via ResNet50. Experimental results indicate the effectiveness of transfer learning on the building damage mapping using different data sources.

In this paper, we propose to fuse multiple sources remotely sensed datasets, such as hyperspectral (HS) and Light Detection and Ranging (LiDAR)-derived digital surface model (DSM) using a novel deep learning method. Morphological openings and closings with partial reconstruction are taken into account to model spatial and elevation information for both sources. Then, the stacked features directly input to a deep learning classifier, namely Deep Forest (DF). In particular, Deep Forest can be viewed as the cascade or the ensembles of Rotation Forests (RoF) and Random Forests (RF). We applied the proposed method to the datasets obtained from Tama forest, Japan. Experimental results demonstrate that Deep Forest can achieve better classification results than other approaches. Compared to deep neural networks, deep forest pays little effort in parameter tuning and has a significant reduction in computational complexity.

Domain adaptation and transfer learning adapt the priori information of source domain to train a classier used to predict the label in the target domain. The parameter and instance transfer methods have shown excellent performance. The former adjusts the parameters of transitional classifiers and the latter re-weights the training sample to the different training set, which is similar to the AdaBoost. To further improve the performance, we proposed to combine the two techniques mentioned above. More specifically, we select the Transfer Boosting and domain adaptation extreme learning machine (DAELM) as the instance and parameter transfer methods, respectively. We refer the proposed method to the boosting for DAELM (BDAELM). We compare the proposed method with DAELM and other methods on the real cross-domain hyperspectral remote sensing images acquired over a Japanese mixed forest, showing improved classification accuracies.

Noises in hyper spectral data make it difficult to accurately classify the domains. In order to solve this problem, some filtering methods are proposed; however, filtering processes disturb small size classes and sharp boundaries. To protect the domain boundaries in hyperspectral data classification, we applied normalized cuts segmentation in the preprocessing before classification. The proposed methodologies show higher OA and AA, which means that the domain boundaries are maintained, and classification accuracies of small classes are improved. We also found that the classification accuracies are not sensitive to the number of clusters. The proposed methodology is useful for the combination of smoothing filters that show high classification performances.

Due to the development of sensors and data acquisition technology, the fusion of features from multiple sensors is a very hot topic. In this letter, the use of morphological features to fuse a hyperspectral (HS) image and a light detection and ranging (LiDAR)-derived digital surface model (DSM) is exploited via an ensemble classifier. In each iteration, we first apply morphological openings and closings with a partial reconstruction on the first few principal components (PCs) of the HS and LiDAR data sets to produce morphological features to model spatial and elevation information for HS and LiDAR data sets. Second, three groups of features (i.e., spectral and morphological features of HS and LiDAR data) are split into several disjoint subsets. Third, data transformation is applied to each subset and the features extracted in each subset are stacked as the input of a random forest classifier. Three data transformation methods, including PC analysis, linearity preserving projection, and unsupervised graph fusion, are introduced into the ensemble classification process. Finally, we integrate the classification results achieved at each step by a majority vote. Experimental results on coregistered HS and LiDAR-derived DSM demonstrate the effectiveness and potentialities of the proposed ensemble classifier.

In this letter, multisource earth observation (EO) data sets, including multitemporal Landsat-8, digital surface model, and spatial information, were integrated for land-cover classification by random forest (RF) and support vector machines (SVMs). We demonstrated in this letter that both RF and SVM are useful tools for classification of land cover in the local climate zones featured with highly heterogeneous landscape. Classification of land cover by RF was with an overall accuracy (OA) of 86.2%, while the OA was 85.5% for SVM. However, we found that RF was more stable than SVM for multisource EO data in classifying land cover without normalizing different feature data sets. Experiments showed that the thermal features were more important than temporal and spatial ones in discriminating impervious objects, while the temporal and spatial features were generally better than thermal ones in separating the distinct vegetation categories. Another finding was that our experiments indicated that spectral features were the most important in classification of land cover, followed by temporal, thermal, and spatial features, respectively. As to the spectral features, red channels were the most important, followed by short-wave infrared, near-infrared, and green channels. Thus, it could be concluded that the combination of spectral, thermal, spatial, and temporal information would be an optimal approach to increase the OA of land-cover classification in the zones featured with highly heterogeneous landscape.

To adequately represent the nonlinearities in the high-dimensional feature space for hyperspectral images (HSIs), we propose a multiple kernel collaborative representation-based classifier (CRC) in this paper. Extended morphological profiles are first extracted from the original HSIs, because they can efficiently capture the spatial and spectral information. In the proposed method, a novel multiple kernel learning (MKL) model is embedded into CRC. Multiple kernel patterns, e.g., Naive, Multimetric, and Multiscale are adopted for the optimal set of basic kernels, which are helpful to capture the useful information from different pixel distributions, kernel metric spaces, and kernel scales. To learn an optimal linear combination of the predefined basic kernels, we add an extra training stage to the typical CRC where kernel weights are jointly learned with the representation coefficients from the training samples by minimizing the representation error. Moreover, by considering different contributions of dictionary atoms, the adaptive representation strategy is applied to the MKL framework via a dissimilarity-weighted regularizer to obtain a more robust representation of test pixels in the fused kernel space. Experimental results on three real HSIs confirm that the proposed classifiers outperform the other state-of-the-art representation-based classifiers.

Multiple types of features, e.g., spectral, filtering, texture, and shape features, are helpful for hyperspectral image (HSI) classification tasks. Combining multiple features can describe the characteristics of pixels from different perspectives, and always results in better classification performance. Recently, multifeature combination learning has been widely employed to the multitask-learning-based representation-based model to obtain a multifeature representation vector. However, the linear sparse representation-based classifier (SRC) cannot handle the HSI with highly nonlinear distribution, and kernel sparse representation-based classifier (KSRC) can remedy the drawback of linear SRC. By adopting nonlinear mapping, the samples in kernel space are often of high or even infinite dimensionality. In this paper, we integrate kernel principal component analysis into multifeature-based KSRC and propose a novel multiple feature kernel sparse representation-based classifier (namely, MFKSRC) for hyperspectral imagery. More specifically, spatial features, Gabor textures, local binary patterns, and difference morphological profiles are adopted and then each kind of feature is transformed nonlinearly into a new low-dimensional kernel space. The proposed framework can handle data with nonlinear distribution and add a dimensionality reduction stage in kernel space before optimizing the corresponding cost function. Experimental results on different HSIs demonstrate that the proposed MFKSRC algorithm outperforms the state-of-the-art classifiers.

Classification techniques for hyperspectral images based on random forest (RF) ensembles and extended multiextinction profiles (EMEPs) are proposed as a means of improving performance. To this end, five strategies - bagging, boosting, random subspace, rotation-based, and boosted rotation-based - are used to construct the RF ensembles. EPs, which are based on an extrema-oriented connected filtering technique, are applied to the images associated with the first informative components extracted by independent component analysis, leading to a set of EMEPs. The effectiveness of the proposed method is investigated on two benchmark hyperspectral images: the University of Pavia and Indian Pines. Comparative experimental evaluations reveal the superior performance of the proposed methods, especially those employing rotation-based and boosted rotation-based approaches. An additional advantage is that the CPU processing time is acceptable.

Due to the benefits and limitation of different remote sensing sensors, fusion of the features from multiple sensors, such as hyperspectral and light detection and ranging (LiDAR) is an effective method for land cover mapping. In this paper, we propose a novel ensemble classifier to fuse hyperspectral and LiDAR datasets for classification. First, morphological features are used to model spatial and elevation information from the first few principal components (PCs) of the original hyperspetcral (HS) image and LiDAR data. Second, we split different kinds of features (i.e., spectral bands, morphological features of hyperspectral and LiDAR), into several disjoint subsets and apply the data transformation method to each subset. In particular, three data transformation methods, including principal component analysis (PCA), linearity preserving projection (LPP) and unsupervised graph fusion (UGF) are considered. Third, the features extracted in each subset are concatenated to classify by a random forest (RF) classifier. Experimental results on a co-registered HS and LiDAR data provide the effectiveness and potentialities of the proposed ensemble classifier.

In this work, we develop a new framework to combine ensemble learning and composite kernel learning for hyperspectral image classification. We refer it as the multiple composite kernel learning, which is based on an iterative architecture. More specifically, in each iteration, we use the rotation-based ensemble to create rotation matrix, which is used to generate rotated features for both spectral and spatial information (e.g., extinction profiles). Then, the new spectral and spatial features are integrated into the composite kernels based on support vector machines classifier. Different rotation matrices will lead to obtaining various newly spectral and spatial characteristics, thereby they further increase the diversity and the classification performance. Experimental results on Indian Pines benchmark hyperspectral dataset demonstrate the excellent performance of the proposed method.

In the hyperspectral remote sensing community, decision forests combine the predictions of multiple decision trees (DTs) to achieve better prediction performance. Two well-known and powerful decision forests are Random Forest (RF) and Rotation Forest (RoF). In this work, a novel decision forest, called Partial Least Square Forest (PLSF), is proposed. In the PLSF, we adapt PLS to obtain the components for the hyperplane splitting. Moreover, the projection bootstrap technique is used to retain the full spectral bands for the selection of split in the projected space. Experimental results on three hyperspectral datasets indicated the effectiveness of the proposed PLSF because it enhances the diversity and accuracy within the ensemble when compared to RF and RoF.

In this work, we address the problem of unsupervised domain transfer learning via an ensemble strategy in the context of classification between multiple hyperspectral images. The objective of domain adaption is to assign the label to an image of interest (the target image) using the labeled samples in the source image. The proposed method is based on the rotation-based ensemble and transfer component analysis (TCA). In this method, the feature space in both source and target image is divided into several disjoint feature subsets. Then, the features induced by the TCA technique in the source domain are used as the input space to a random forest (RF) classifier. Finally, the results achieved by each step are fused by a majority vote. We compare the proposed method, ensemble of TCA (E-TCA), with a regular RF and an RF with the reduced features by the TCA. Experiments on the real hyperspectral image acquired over a Japanese mixed forest show remarkable cross-image classification performances.

This paper presents a new methodology for classification of local climate zones based on ensemble learning techniques. Landsat-8 data and open street map data are used to extract spectral-spatial features, including spectral reflectance, spectral indexes, and morphological profiles fed to subsequent classification methods as inputs. Canonical correlation forests and rotation forests are used for the classification step. The final classification map is generated by majority voting on different classification maps obtained by the two classifiers using multiple training subsets. The proposed method achieved an overall accuracy of 74.94% and a kappa coefficient of 0.71 in the 2017 IEEE GRSS Data Fusion Contest.

In this letter, we propose a kernel fused representation-based classifier (KFRC) for hyperspectral images (HSIs), which combines sparse representation (SR) and collaborative representation (CR) into a unified kernel representation-based classification framework. First, we present two individual kernel methods, i.e., kernel SR (KSR) and kernel CR (KCR), which kernelize the representation methods by projecting the samples into a high-dimensional kernel space to improve the samples separability between different classes. Once obtaining the two kernel representation coefficients, KFRC attempts to achieve a balance between KSR and KCR via an adjusting parameter $\theta $ in the kernel residual domain. Subsequently, the class label of each test sample is determined by the minimum residual for each class. Experimental results on two HSIs demonstrate the proposed kernel fused method performs better than the other state-of-the-art representation-based classifiers.

To improve the capability of a traditional sparse representation-based classifier (SRC), we propose a novel dissimilarity-weighted SRC (DWSRC) for hyperspectral image (HSI) classification. In particular, DWSRC computes the weights for each atom according to the distance or dissimilarity information between the test pixel and the atoms. First, a locality constraint dictionary set is constructed by the Gaussian kernel distance with a suitable distance metric (e.g., Euclidean distance). Second, the test pixel is sparsely coded over the new weighted dictionary set based on the 11-norm minimization problem. Finally, the test pixel is classified by using the obtained sparse coefficients with the minimal residual rule. Experimental results on two widely used public HSIs demonstrate that the proposed DWSRC is more efficient and accurate than other state-of-the-art SRCs.

Scene classification from remote sensing images provides new possibilities for potential application of high spatial resolution imagery. How to efficiently implement scene recognition from high spatial resolution imagery remains a significant challenge in the remote sensing domain. Recently, convolutional neural networks (CNN) have attracted tremendous attention because of their excellent performance in different fields. However, most works focus on fully training a new deep CNN model for the target problems without considering the limited data and time-consuming issues. To alleviate the aforementioned drawbacks, some works have attempted to use the pretrained CNN models as feature extractors to build a feature representation of scene images for classification and achieved successful applications including remote sensing scene classification. However, existing works pay little attention to exploring the benefits of multilayer features for improving the scene classification in different aspects. As a matter of fact, the information hidden in different layers has great potential for improving feature discrimination capacity. Therefore, this paper presents a fusion strategy for integrating multilayer features of a pretrained CNN model for scene classification. Specifically, the pretrained CNN model is used as a feature extractor to extract deep features of different convolutional and fully connected layers; then, a multiscale improved Fisher kernel coding method is proposed to build a mid-level feature representation of convolutional deep features. Finally, the mid-level features extracted from convolutional layers and the features of fully connected layers are fused by a principal component analysis/spectral regression kernel discriminant analysis method for classification. For validation and comparison purposes, the proposed approach is evaluated via experiments with two challenging high-resolution remote sensing data sets, and shows the competitive performance compared with fully trained CNN models, fine-tuning CNN models, and other related works.

Multiple classifier systems or ensemble learning is an effective tool for providing accurate classification results of hyperspectral remote sensing images. Two well-known ensemble learning classifiers for hyperspectral data are random forest (RF) and rotation forest (RoF). In this paper, we proposed to use a novel decision tree (DT) ensemble method, namely, canonical correlation forest (CCF). More specifically, several individual canonical correlation trees (CCTs) that are binary DTs, which use canonical correlation components for the hyperplane splitting, are used to construct the CCF. Additionally, we adopt the projection bootstrap technique in CCF, in which the full spectral bands are retained for split selection in the projected space. The techniques aforementioned allow the CCF to improve the accuracy of member classifiers and diversity within the ensemble. Furthermore, the CCF is extended to the spectral-spatial frameworks that incorporate Markov random fields, extended multiattribute profiles (EMAPs), and the ensemble of independent component analysis and rolling guidance filter (E-ICA-RGF). Experimental results on six hyperspectral data sets are used to indicate the comparative effectiveness of the proposed method, in terms of accuracy and computational complexity, compared with RF and RoF, and it turns out that CCF is a promising approach for hyperspectral image classification not only with spectral information but also in the spectral-spatial frameworks.

Conventional feature extraction methods cannot fully exploit both the spectral and spatial information of hyperspectral imagery. In this paper, we propose an ensemble method of subspace independent component analysis (ICA) and edge-preserving filtering (EPF) for the classification of hyper-spectral data to achieve this task. First, several subsets are randomly selected from the original feature space. Second, ICA is used to extract spectral independent components followed by a recent and effective EPF method, rolling guidance filter (RGF), to produce spatial features. The spatial features are treated as the input of a random forest (RF) classifier. Finally, the classification results from each subset are integrated together to produce the final map. Experimental results on real hyperspectral data demonstrate the effectiveness of the proposed method. A sensitivity analysis of this new classifier is also performed.

Images are usually represented by different groups of features, such as color, shape and texture attributes. In this paper, we propose a classification approach that integrates multiple features, such as spectral and spatial information. We refer this approach to multiple feature learning via rotation (MFL-R) strategy, which adopt a rotation-based ensemble method by using a data transformation approach. Five data transformation methods, including principal component analysis (PCA), neighborhood preserving embedding (NPE), linear local tangent space alignment (LLTSA), linearity preserving projection (LPP) and multiple feature combination via manifold learning and patch alignment (MLPA) are used in the MFL-R framework. Experimental results over two hyperspectral remote sensing images demonstrate that MFL-R with MLPA gains better performances and is not sensitive to the tuning parameters.

Rotation Forest (RoF) is a decision tree ensemble classifier, which uses random feature selection and data transformation techniques to improve both the diversity and accuracy of base classifiers. Traditional RoF only considers data transformation on spectral information. In order to further improve the performance of RoF, we introduce spectral-spatial data transformation into RoF and thus propose a spectral-spatial Rotation Forest (SSRoF). The proposed method is experimentally investigated on a hyperspectral remote sensing image collected by the Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) sensor. Experimental results indicate that the proposed methodology achieves excellent performance.

Morphological attribute profiles (APs) are discriminant features in the spectral–spatial classification of hyperspectral data. However, the optimal range of parameters in each filter is always a challenging yet important task, since an unsuitable range of parameters likely leads to inferior results. In order to alleviate this problem, we propose an ensemble method, which integrates multiple classification results based on a series of APs. The APs are obtained by using different filters with thresholds that are randomly selected from an arbitrarily defined range of parameters. Experimental results conducted on two hyperspectral images demonstrate the robustness and effectiveness of the proposed method.

In this letter, we propose a new version of the rotation forest (RoF) method for the pixelwise classification of hyperspectral images. RoF, which is an ensemble of decision tree classifiers, uses random feature selection and data transformation techniques (i.e., principal component analysis) to improve both the accuracy of base classifiers and the diversity within the ensemble. Traditional RoF performs data transformation on the training samples of each subset. In order to further improve the performance of RoF, the data transformation is separately performed on each class, extracting sets of transformation matrices that are strictly dependent on the training samples of each single class. The approach, namely, class-separation-based RoF (RoFCS), is experimentally investigated on a hyperspectral image collected by Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) sensor. Experimental results demonstrate that the proposed methodology achieves excellent performances, in comparison with random forest and RoF classifiers.

To obtain accurate classification results of hyperspectral images, both spectral and spatial information should be fully exploited in the classification process. In this paper, we propose a novel method using independent component analysis (ICA) and edge-preserving filtering (EPF) via an ensemble strategy for the classification of hyperspectral data. First, several subsets are randomly selected from the original feature space. Second, ICA is used to extract spectrally independent components followed by an effective EPF method, to produce spatial features. Two strategies (i.e., parallel and concatenated) are presented to include the spatial features in the analysis. The spectral-spatial features are then classified with a random forest or a rotation forest classifier. Experimental results on two real hyperspectral data sets demonstrate the effectiveness of the proposed methods. A sensitivity analysis of the new classifiers is also performed.

With different principles, support vector machines (SVMs) and multiple classifier systems (MCSs) have shown excellent performances for classifying hyperspectral remote sensing images. In order to further improve the performance, we propose a novel ensemble approach, namely, rotation-based SVM (RoSVM), which combines SVMs and MCSs together. The basic idea of RoSVM is to generate diverse SVM classification results using random feature selection and data transformation, which can enhance both individual accuracy and diversity within the ensemble simultaneously. Two simple data transformation methods, i.e., principal component analysis and random projection, are introduced into RoSVM. An empirical study on three hyperspectral data sets demonstrates that the proposed RoSVM ensemble method outperforms the single SVM and random subspace SVM. The impacts of the parameters on the overall accuracy of RoSVM (different training sets, ensemble sizes, and numbers of features in the subset) are also investigated in this paper.

This paper proposes a semi-supervised graph-based fusion framework to couple dimensionality reduction and the fusion of multi-sensor data for classification. First, morphological features are used to model the elevation and spatial information contained in both LiDAR data and on the first few principal components (PCs) of the original hyperspectral (HS) image. Then, we fuse the features by projecting the spectral, spatial and elevation features onto a lower subspace through our proposed semi-supervised fusion graph. Experimental results on fusion of HS and LiDAR data from the 2013 IEEE GRSS Data Fusion Contest demonstrate effectiveness of the proposed method. Compared to the methods using single data source or unsupervised graph fusion, with the proposed method, overall classification accuracies were improved by 9% and 4%, respectively.

In this letter, we propose a novel approach for improving Random Forest (RF) in hyperspectral image classification. The proposed approach combines the ensemble of features and the semisupervised feature extraction (SSFE) technique. The main contribution of our approach is to construct an ensemble of RF classifiers. In this way, the feature space is divided into several disjoint feature subspaces. Then, the feature subspaces induced by the SSFE technique are used as the input space to an RF classifier. This method is compared with a regular RF and an RF with the reduced features by the SSFE on two real hyperspectral data sets, showing an improved performance in ill-posed, poor-posed, and well-posed conditions. An additional study shows that the proposed method is less sensitive to the parameters.

The quantitative estimation of the fractional cover of carbonate rock (CR) is critical for natural resource management and ecological conservation in karst areas. Based on the analysis of spectral properties of CR together with other land cover types, we proposed two CR indices (CRIs) and established the model that represents the relationships between the CRIs and the fractional cover of CR. Then, the fractional cover of CR was estimated by using the developed model. Experimental results on Landsat-8 Operational Land Imager images acquired at Southwestern China demonstrated the effectiveness of the developed model. Compared with other indices, the proposed CRIs show the highest correlations with the fractional cover of CR.

In this paper, we propose a new spectral-spatial classification strategy to enhance the classification performances obtained on hyperspectral images by integrating rotation forests and Markov random fields (MRFs). First, rotation forests are performed to obtain the class probabilities based on spectral information. Rotation forests create diverse base learners using feature extraction and subset features. The feature set is randomly divided into several disjoint subsets; then, feature extraction is performed separately on each subset, and a new set of linear extracted features is obtained. The base learner is trained with this set. An ensemble of classifiers is constructed by repeating these steps several times. The weak classifier of hyperspectral data, classification and regression tree (CART), is selected as the base classifier because it is unstable, fast, and sensitive to rotations of the axes. In this case, small changes in the training data of CART lead to a large change in the results, generating high diversity within the ensemble. Four feature extraction methods, including principal component analysis (PCA), neighborhood preserving embedding (NPE), linear local tangent space alignment (LLTSA), and linearity preserving projection (LPP), are used in rotation forests. Second, spatial contextual information, which is modeled by MRF prior, is used to refine the classification results obtained from the rotation forests by solving a maximum a posteriori problem using the α-expansion graph cuts optimization method. Experimental results, conducted on three hyperspectral data with different resolutions and different contexts, reveal that rotation forest ensembles are competitive with other strong supervised classification methods, such as support vector machines. Rotation forests with local feature extraction methods, including NPE, LLTSA, and LPP, can lead to higher classification accuracies than that achieved by PCA. With the help of MRF, the proposed algorithms can improve the classification accuracies significantly, confirming the importance of spatial contextual information in hyperspectral spectral-spatial classification.

Classification is one of the most important techniques to the analysis of hyperspectral remote sensing images. Nonetheless, there are many challenging problems arising in this task. Two common issues are the curse of dimensionality and the spatial information modeling. In this paper, we present a new general framework to train series of effective classifiers with spatial information for classifying hyperspectral data. The proposed framework is based on the two key observations: 1) the curse of dimensionality and the high feature-to-instance ratio can be alleviated by using random subspace (RS) ensembles; and 2) the spatial-contextual information is modeled by the extended multiattribute profiles (EMAPs). Two fast learning algorithms, i.e., decision tree (DT) and extreme learning machine (ELM), are selected as the base classifiers. Six RS ensemble methods, namely, RS with DT, random forest (RF), rotation forest, rotation RF (RoRF), RS with ELM (RSELM), and rotation subspace with ELM (RoELM), are constructed by the multiple base learners. Experimental results on both simulated and real hyperspectral data verify the effectiveness of the RS ensemble methods for the classification of both spectral and spatial information (EMAPs). On the University of Pavia Reflective Optics Spectrographic Imaging System image, our proposed approaches, i.e., both RSELM and RoELM with EMAPs, achieve the state-of-the-art performances, which demonstrates the advantage of the proposed methods. The key parameters in RS ensembles and the computational complexity are also investigated in this paper.

In this letter, an ensemble learning approach, Rotation Forest, has been applied to hyperspectral remote sensing image classification for the first time. The framework of Rotation Forest is to project the original data into a new feature space using transformation methods for each base classifier (decision tree), then the base classifier can train in different new spaces for the purpose of encouraging both individual accuracy and diversity within the ensemble simultaneously. Principal component analysis (PCA), maximum noise fraction, independent component analysis, and local Fisher discriminant analysis are introduced as feature transformation algorithms in the original Rotation Forest. The performance of Rotation Forest was evaluated based on several criteria: different data sets, sensitivity to the number of training samples, ensemble size and the number of features in a subset. Experimental results revealed that Rotation Forest, especially with PCA transformation, could produce more accurate results than bagging, AdaBoost, and Random Forest. They indicate that Rotation Forests are promising approaches for generating classifier ensemble of hyperspectral remote sensing.

The development of multisensor systems in recent years has led to great increase in the amount of available remote sensing data. Image fusion techniques aim at inferring high quality images of a given area from degraded versions of the same area obtained by multiple sensors. This paper focuses on pansharpening, which is the inference of a high spatial resolution multispectral image from two degraded versions with complementary spectral and spatial resolution characteristics: a) a low spatial resolution multispectral image; and b) a high spatial resolution panchromatic image. We introduce a new variational model based on spatial and spectral sparsity priors for the fusion. In the spectral domain we encourage low-rank structure, whereas in the spatial domain we promote sparsity on the local differences. Given the fact that both panchromatic and multispectral images are integrations of the underlying continuous spectra using different channel responses, we propose to exploit appropriate regularizations based on both spatial and spectral links between panchromatic and the fused multispectral images. A weighted version of the vector Total Variation (TV) norm of the data matrix is employed to align the spatial information of the fused image with that of the panchromatic image. With regard to spectral information, two different types of regularization are proposed to promote a soft constraint on the linear dependence between the panchromatic and the fused multispectral images. The first one estimates directly the linear coefficients from the observed panchromatic and low resolution multispectral images by Linear Regression (LR) while the second one employs the Principal Component Pursuit (PCP) to obtain a robust recovery of the underlying low-rank structure. We also show that the two regularizers are strongly related. The basic idea of both regularizers is that the fused image should have low-rank and preserve edge locations. We use a variation of the recently proposed Split Augmented Lagrangian Shrinkage (SALSA) algorithm to effectively solve the proposed variational formulations. Experimental results on simulated and real remote sensing images show the effectiveness of the proposed pansharpening method compared to the state-of-the-art.

Support vector machine (SVM) and Random Forest (RF) have been developed to improve the accuracy of hyperspectral remote sensing (HRS) image classification significantly in recent years. Due to the different characteristics and obvious diversity between SVM and RF, we propose two integration approaches which combine SVM and Random Forest to classify the HRS image. The proposed method called DWDCS is examined by two hyperspectral images and it can acquire the higher overall accuracy and also improve the accuracy of each classes. Experimental results indicate that the proposed approaches have a great deal of advantages in classifying HRS image.

In remote sensing, pansharpening refers to the technique that combines the complementary spectral and spatial resolution characteristics of a multispectral image and a panchromatic image, with the objective to generate a high-resolution color image. This paper presents a new pansharpening method based on the minimization of a variant of total variation. We consider the fusion problem as the colorization of each pixel in the panchromatic image. A new term concerning the gradient of the panchromatic image is introduced in the functional of total variation so as to preserve edges. Experimental results on IKONOS satellite images demonstrate the effectiveness of the proposed method.

Decision level fusion, using a specific criterion or algorithm to integrate the classified results from different classifiers, has shown great benefits to improve classification accuracy of multi-source remote sensing images. In this paper, three decision level fusion methods and four schemes for input data are used to hyperspectral remote sensing image classification. Different feature combination and decision level fusion approaches are experimented and analyzed, and the results show that decision level fusion is effective to improve the performance of hyperspectral remote sensing image classification.