Forest tree species classification has great significance for sustainable development of forest resource. Multi-source remote sensing data provides abundant temporal, spatial, and spectral information for tree species classification. However, there lacks tree species classification methods which comprehensively capture and fuse spatio-temporal-spectral information. Therefore, a tree species classification method based on deep ensemble learning of multi-source spatio-temporal-spectral remote sensing data is proposed. Firstly, multi-temporal, high-resolution and hyperspectral data are utilized for training temporal, spatial, and spectral deep networks. Furtherly, deep ensemble learning is developed for fusion of spatio-temporal-spectral network outputs, where weighted fusion is implemented via dynamic weight optimization based on the spatio-temporal-spatial features. Experimental results indicate that the importance of temporal features is higher than that of spatial information, and spectral networks perform best among all network structures. After the spatio-temporal-spectral ensemble learning, the performance of tree species classification is further improved, and the overall accuracy of the proposed method reaches above 90%. The proposed algorithm realizes precise and fine-scale tree species classification, and provides technique support for the monitoring and conservation of forest resource.

Semantic change detection (SCD) endeavors to identify land-cover changes from multitemporal remote sensing images, providing essential information for various applications. Nevertheless, conventional supervised SCD methods necessitate extensive pixel-level annotations, limiting their applicability. The capability of self-supervised methods to learn feature representations with large amounts of unlabeled data and minimal annotation, and to achieve superior performance, has made them one of the hot topics in remote sensing. However, most self-supervised methods in remote sensing are primarily designed to learn general semantic representations of images, which limits their effectiveness for tasks like SCD that require the analysis of complex semantic transformations. To address this, we propose a multistage, multitask, and multilevel self-supervised network, named S3CD, that learns semantic changes from bi-temporal remote sensing images across scene, pixel, and prototype levels in two stages. In particular, in Stage 2, the network enhances the robustness of SCD by learning semantic consistency within the semantic stable categories across different temporal and capturing the temporal patterns of semantic change categories. We evaluate S3CD on two widely used remote sensing change detection (CD) datasets, where it outperformed state-of-the-art self-supervised and supervised SCD methods. Notably, in the binary CD (BCD) task (i.e., detecting the locations of changes), S3CD also outperforms most supervised learning methods. Therefore, this approach facilitates the application of self-supervised learning in the field of remote sensing CD.

Semantic segmentation techniques for extracting building footprints from high-resolution remote sensing images have been widely used in many fields such as urban planning. However, large-scale building extraction demands a higher diversity in training samples. In this paper, we construct a Global Building Semantic Segmentation (GBSS) dataset (The dataset will be released), which comprises 116.9k pairs of samples (about 742k buildings) from six continents. There are significant variations in building samples in terms of size and style, making the dataset a more challenging benchmark for evaluating the generalization and robustness of building semantic segmentation models. We validated through quantitative and qualitative comparisons between different datasets, and further confirmed the potential application in the field of transfer learning by conducting experiments on subsets.

Recent advancements in fully-supervised change detection (CD) have been notable, yet applying CD effectively in unlabeled scenarios remains a challenge. Our study focuses on domain adaptation of change detection (DACD), transferring knowledge from a labeled dataset (source domain) to an unlabeled dataset (target domain). However, we’ve found that this transfer often leads to reduced discrimination between change and non-change classes because of cross-domain discrepancy, adversely affecting CD performance in the target domain. Aiming at it, we propose a source discriminative learning (SDL) method to learn more discriminative interclass feature representations in the source domain, which can improve the transferability of change knowledge across different CD domains. To evaluate its effectiveness, we established two cross-domain CD scenarios using well-known CD datasets and applied our SDL method to them. The experimental results affirm that our method effectively boosts DACD performance.

Exposed surface for buildings (ESB), which refers to exposed surfaces with traces of building construction, often leads to urban dust. Accurate ESB detection is important for planning urban development and improving urban environment. Fine-grained monitoring of ESB typically needs massive high-quality pixel-level labels, which are demanding and expensive. In contrast, obtaining cost-efficient image-level labels is more promising. Most image-level weakly supervised methods can extract pixel-level pseudo labels using the class activation map (CAM) generated by the classification network. Subsequently, these labels are applied to train the semantic segmentation network. However, the CAM is easy to miss fine-grained information, which leads to label noise. Moreover, the downsampling in the segmentation networks will further loss the spatial information. Furthermore, the sparse distribution and irregular shape of ESB pose additional challenges. Given these problems, we propose a stepwise refining image-level weakly supervised semantic segmentation method (SRIWS): 1) we introduce a new data augmentation method called SRMix to oversample the classification dataset; 2) we propose a two-branch network with a superpixel pooling layer (SPNet) as the semantic segmentation network to capture both global semantic information and spatial details; and 3) to alleviate the impact of potential noise in the initial labels, we design the high-confidence sample filtering operation (HSF) during the SPNet training. The evaluation experiments for the SRIWS were performed on three datasets. The results confirm that our proposed SRIWS presents a superior performance in recognizing ESB compared with existing state-of-the-art methods. In addition, numerous ablation experimental results indicate the effectiveness and robustness of our SRIWS.

We propose a novel network for Multi-View Stereo reconstruction in the field of remote sensing, which considers Clustering-based Semantic Consistency into depth estimation optimization, referred to as CSC-MVS. In this approach, high-level semantic information acquired from multiple views is utilized to construct semantic consistency and assist in guiding the optimization of the MVS network. Specifically, the Non-negative Matrix Factorization (NMF) branch and the Deep Spectral Decomposition (DSD) branch, are designed to generate local and global semantic guidance, respectively. We then propose an uncertainty multi-task optimization method to adaptively combine matching and semantic metrics. The performance of CSC-MVS is evaluated on representative benchmarks, including the WHU TLC dataset and LuoJia-MVS dataset, demonstrating its effectiveness and generality across diverse remote sensing scenarios. Comprehensive experimental results show that our CSC-MVS significantly improves the performance of various MVS baseline networks and achieves notable accuracy in depth reconstruction. We also conduct ablation studies to validate the rationality of each component, and sensitivity analysis to confirm the robustness and adaptability of our proposed method. The code is available at https://github.com/zsl-whu/csc-mvs.

Most of the existing deep learning-based hyperspectral image (HSI) classification algorithms are based on supervised learning, where large number of annotated labels with high acquisition cost are required. Self-supervised learning (SSL) methods can learn abundant representations using large amount of unlabeled data, thereby reducing the reliability of labels. Particularly, SSL based on Masked Image Modeling (MIM) can extract fine-grained features, which is well-suited for HSI classification as a pixel-level interpretation task. However, MIM has scarcely been investigated in HSI classification. Current algorithms lack a comprehensive consideration of the multiscale spectral-spatial characteristics of HSI when constructing the pre-training task, and there exists high computational cost and redundancy when applied to large-scale HSI. Therefore, this paper develops an SSL framework based on Spectral-Spatial Hierarchical Masked Modeling (S2HM2) for large-scale HSI classification. Considering the spectral-spatial characteristics of HSI, 3D masking strategy and spectral-spatial consistency loss are proposed to construct MIM task. To fully exploit features at each scale, hierarchical 3D Feature Pyramid Network (3D-FPN) is designed as decoder for both pre-text and downstream tasks in a “pixel-to-pixel” manner. In addition, Multi-Scale Masked Feature Modeling (MS-MFM) task is proposed to further facilitate the multiscale feature learning. The SSL pre-training is guided by both MIM and MS-MFM. Experimental results on two large-scale hyperspectral datasets, i.e., WHU-OHS and WHU-H2SR, demonstrate the superiority of the proposed method. Furthermore, transfer learning experiments are conducted on a variety of hyperspectral datasets, where classification accuracies are boosted in most of the scenarios. Source code will be made available at https://github.com/tulilin/S2HM2.

Obtaining timely and reliable built-up area (BUA) information across extensive geographical zones holds crucial significance for understanding environmental change and human activities. BUAs often exhibit detailed textures and structures in high-resolution imagery but also present strong heterogeneity. Current methods for BUA extraction primarily relied on planar information from single-view imagery, struggling to effectively capture the 3-D attributes of urban landscapes. Therefore, to address this challenge, this article proposes a cross-angle propagation network (CAPNet) based on multiview remote sensing stereo observation imagery. Our contributions are threefold: 1) we propose the cross-angle fusion module (CAFM) to exploit BUA’s complementary spatial-spectral-angular context across different viewing angles. This module leverages attention mechanisms for the automated acquisition of multiangle feature representation learning from diverse angle combinations. 2) We propose a multiangular propagation decoder (MAPD) that pioneers the exploration of gradually propagating multiangle disparity information through bidirectional-adjacent feature fusion across hierarchical levels. 3) We construct a large-scale, high-resolution multiview BUA (MVBA) dataset over China’s 41 major cities based on the ZY-3 satellites. Extensive experiment results on MVBA and the public WV-3 multiview semantic stereo datasets verify CAPNet’s superiority to existing state-of-the-art (SOTA) models, on preserving overall BUA shape, edge, and internal structures. The dataset and the source code of CAPNet will be publicly available athttps://github.com/zuo-ux/Cross-Angle-Propagation-Network.

Multiview stereo (MVS) aerial image depth estimation is a research frontier in the remote sensing field. Recent deep learning-based advances in close-range object reconstruction have suggested the great potential of this approach. Meanwhile, the deformation problem and the scale variation issue are also worthy of attention. These characteristics of aerial images limit the applicability of the current methods for aerial image depth estimation. Moreover, there are few available benchmark datasets for aerial image depth estimation. In this regard, this article describes a new benchmark dataset called the LuoJia-MVS dataset (https://irsip.whu.edu.cn/resources/resources_en_v2.php), as well as a new deep neural network known as the hierarchical deformable cascade MVS network (HDC-MVSNet). The LuoJia-MVS dataset contains 7972 five-view images with a spatial resolution of 10 cm, pixel-wise depths, and precise camera parameters, and was generated from an accurate digital surface model (DSM) built from thousands of stereo aerial images. In the HDC-MVSNet network, a new full-scale feature pyramid extraction module, a hierarchical set of 3-D convolutional blocks, and “true 3-D” deformable 3-D convolutional layers are specifically designed by considering the aforementioned characteristics of aerial images. Overall and ablation experiments on the WHU and LuoJia-MVS datasets validated the superiority of HDC-MVSNet over the current state-of-the-art MVS depth estimation methods and confirmed that the newly built dataset can provide an effective benchmark.

Change detection of remote sensing images has a wide range of applications in many fields. In recent years, deep learning has become one of the most powerful tools for remote sensing change detection, thanks to its excellent feature learning capability. However, most deep learning methods require a lot of labeled data for the training, which is time-consuming and labor-intensive. Recently, a new learning paradigm—self-supervised learning—has become one of the hot topics in the field of change detection due to its ability to learn feature representations by training with a large amount of unlabeled data and without a large number of sample annotations. However, the existing methods for self-supervised learning are usually designed for natural image processing and are less considered for change detection in more complex scenes (e.g., remote sensing imagery). Therefore, in this paper, we propose a novel network by fusing temporal and differential information for self-supervised contrastive learning change detection, namely TD-SSCD. Specifically, TD-SSCD aims to mine information from the bi-temporal images and their differential images in a self-supervised learning framework, and it gradually learns the potential correlations between them through an alternating iteration learning strategy. The experimental results based on the OSCD and SZTAKI datasets show that the proposed method outperforms the current state-of-the-art unsupervised and self-supervised change detection methods. Benefiting from pre-training on unlabeled samples, the method closes the gap between unsupervised and supervised change detection.

In this article, a deep neural network for semantic segmentation of high-resolution remote sensing images is proposed for urban construction land classification. The network follows a high-resolution network (HRNet) architecture. Specifically, a directional self-attention on the paths of different resolutions is proposed, aiming to correct the directional bias caused by the attention of strip windows during the model learning, while also reducing the computational complexity, and allowing the model to improve both the accuracy and the speed. At the end of the network, a distributed alignment module with spatial information is constructed to train additional learnable parameters, to adjust the biased decision boundaries through a two-stage learning strategy, and alleviate the problem of accuracy degradation due to the unbalanced training data. We tested the proposed method and compared it with the current state-of-the-art (SOTA) semantic segmentation methods on the Luojia-fine-grained land cover (FGLC) dataset and the Wuhan Dense Labeling Dataset (WHDLD), and the proposed one obtained the best performance. We also verified the effectiveness of each component of the network through ablation experiments. The code and model will be available athttps://github.com/Zhzhyd/DWin-HRFormer.

Change detection (CD) has received raising attention for its broad application value. However, traditional fully supervised CD methods have a huge demand for pixel-level annotations, which are laborious and even impossible in some few-shot scenarios. Recently, several semisupervised CD (SSCD) methods have been proposed to utilize numerous unlabeled remote sensing image (RSI) pairs, which can largely reduce the annotation dependence. These methods are mainly based on: 1) adversarial learning, whose optimization direction is difficult to control as a black-box method, or 2) feature-consistency learning, which has no explicit physical meaning. To deal with these difficulties, we propose a novel progressive SSCD framework in this article, termed feature-prediction alignment (FPA). FPA can efficiently utilize unlabeled RSI pairs for training by two alignment strategies. First, a class-aware feature alignment (FA) strategy is designed to align the area-level change/no-change feature extracted from different unlabeled RSI pairs (i.e., across regions) with the awareness of their locations, in order to reduce the feature difference within the same classes. Second, a pixelwise prediction alignment (PA) is devised to align the pixel-level change prediction of strongly augmented unlabeled RSI pairs to the pseudo-labels calculated from the corresponding weakly augmented counterparts, in order to reduce the prediction uncertainty of various RSI transformations with physical meaning. Experiments are carried out on four widely used CD benchmarks, including Learning, Vision and Remote Sensing Laboratory (LEVIR-CD), Wuhan University building CD (WHU-CD), CDD, and GZ-CD, and our FPA achieves the state-of-the-art performance. The experimental results demonstrate the superiority of our method in both effectiveness and generalization. Our code is available athttps://github.com/zxt9/FPA-SSCD.

Change detection (CD) is an important Earth observation task that can monitor change areas at two times from the view of space. However, fully-supervised CD has a heavy dependence on numerous manually-labeled data, limiting their applications in practice. Beyond the fully-supervised setting, semi-supervised change detection (SSCD), which uses a few labeled data to guide the unsupervised learning of dominant unlabeled data, has attracted increasing attention for its significant advantage in alleviating the demand for annotations. To this end, in this paper we propose a joint self-training and rebalanced consistency learning (ST-RCL) framework for SSCD, which consists of a basic supervised branch for the labeled data and a novel unsupervised branch for the unlabeled data. To make full use of the unlabeled data, the unsupervised branch generates pseudo-labels from weakly-augmented unlabeled remote sensing image (RSI) pairs to supervise the CD of two strongly-augmented counterparts, including an unrotated version and a rotated version. On one hand, the unrotated unlabeled RSI pairs are pseudo-supervised with the pseudo-labels by confidence-based self-training. On the other hand, to further enhance model robustness to rotation non-equivariance and imbalanced distribution, the predictions of rotated unlabeled RSI pairs are aligned to the pseudo-labels by a well-designed rebalanced consistency learning strategy based on uncertainty-based class weighting. Extensive experiments are performed on four widely-used CD datasets, and the proposed ST-RCL yields new state-of-the-art results on all these datasets in comparison with some other SSCD methods, demonstrating its effectiveness and generalization. Our code will be available at https://github.com/zxt9/STRCL-SSCD.

The accurate extraction of residential area is of great significance to disaster assessment, urban management, and climate change research. Deep learning-based methods are limited by annotations acquisition and single data source. In this paper, a dual-branch encoder is proposed to extract the features of different inputs, and a multi-attention fusion module is proposed to effectively fuse dual-branch features. Furthermore, we propose a novel encoder-decoder architecture, called the dual-branch fusion network (DBNet). In addition, we propose a large-scale residential area extraction data set (ZRA) containing 43 Ziyuan-3 (ZY3) multi-spectral (MS) and multi-view (MV) images. Experiments on ZRA, our DBNet achieve state-of-the-art performance with a F1 score of 84.6% and an IoU of 73.3%.

Deep convolutional neural networks have been dominating in the field of hyperspectral image (HSI) classification. However, single convolutional kernels can limit the receptive field and fail to capture the sequential properties of data. Self-Attention-based Transformer can build global sequence information, among which, the Swin Transformer (SwinT) integrates sequence modeling capability and priori information of the visual signals (e.g., locality and translation invariance). Based on SwinT, we propose a 3D Swin Transformer (3DSwinT) to accommodate the 3D properties of HSI and capture the rich spatial-spectral information of HSI. Currently, supervised learning is still the most commonly used method for remote sensing image interpretation. However, pixel-by-pixel HSI classification demands a large number of high-quality labeled samples, which are time-consuming and costly to collect. As an unsupervised learning, self-supervised learning (SSL), especially contrastive learning, can learn semantic representations from unlabeled data, and hence, is becoming a potential alternative to supervised learning. On the other hand, current contrastive learning methods are all single-level or single-scale, which do not consider complex and variable multi-scale features of objects. Therefore, this paper proposes a novel 3DSwinT-based hierarchical contrastive learning method (3DSwinT-HCL), which can fully exploit multi-scale semantic representations of images. Besides, we propose a multi-scale local contrastive learning (MS-LCL) module to mine the pixel-level representations in order to adapt to downstream dense prediction tasks. A series of experiments verify the great potential and superiority of 3DSwinT-HCL.

Classification and change detection of high -resolution remote sensing images using low-resolution labels is a challenging issue in remote sensing community. In this paper, we propose a semi-supervised method based on deep learning to generate high-resolution change maps of Maryland using low-resolution NLCD labels in the Multitemporal Semantic change Detection challenge track (Track MSD) of the 2021 IEEE Data Fusion Contest. Firstly, we refined the NLCD labels using five global land cover products. Subsequently, Fully Convolutional Network (FCN) was trained for the classification of NAIP images with the refined NLCD labels and then the network training was continued with the pseudo-labels from the previous classification results and Modified Normalized Difference Water Index (MNDWI) extracted from Landsat-8 images as the new features. Finally, after the decision-level fusion of the two training periods, change detection results were generated from the bi-temporal classification maps with a post-processing to improve the performance. This algorithm achieved an average Intersection-over-Union (IoU) of 0.6657 on the test dataset and won the 2ndplace of the contest.

The recent availability of high-resolution multiview ZY-3 satellite images, with angular information, can provide an opportunity to capture 3-D structural features for classification. In high-resolution image classification over urban areas, objects with diverse vertical structures make urban landscape more heterogeneous in 3-D space and consequently can make the classification challenging. In this article, a novel multiangle gray-level cooccurrence tensor feature is proposed based on the multiview bands of the ZY-3 imagery, namely, GLCMMA–T. The GLCMMA–Tfeature captures the distributions of the gray-level spatial variation under different viewing angles, which can depict the 3-D textures and structures of urban objects. The spectral and GLCMMA–Ttensor features are interpreted by two 3-D convolutional neural network (CNN) streams and then concatenated as the input to the fully connected layer. This novel multispectral and multiangle 3-D convolutional neural network (M2-3-DCNN) combines the spectral and angular information, and the fused feature has the potential to provide a comprehensive description of urban objects with complex vertical structures. The experimental results on ZY-3 multiview images from four test areas indicate that the proposed method can significantly improve the classification accuracy when compared with several state-of-the-art multiangle features and deep-learning-based image classification methods.

Satellite-derived land surface temperatures (LSTs) are a critical parameter in various fields. Unfortunately, there are numerous gaps in LST products due to cloud contamination and orbital gaps. In previous studies, various gapfilling methods have been developed. However, most of those methods use only spatiotemporal information to fill gaps. In this study, a gapfilling method called the enhanced hybrid (EH) method that integrates spatiotemporal information and information from other similar LST products was proposed. The accuracy of the EH method was compared with the accuracies of three other gapfilling methods that only use spatiotemporal information: Remotely Sensed DAily land Surface Temperature reconstruction (RSDAST), interpolation of the mean anomalies (IMAs), and Gapfill. It was found that the correlations between the four LST products were strong, indicating that using information from other products may improve the accuracy of gapfilling. On average, the mean absolute errors (MAEs) of the data filled using the EH method were 23.7%–52.7% lower than those of RSDAST, 35.4%–38.7% lower than those of IMA, and 38.5%–46.9% lower than those of the Gapfill method. The usage of information from other similar LST products was the main reason for the high accuracy observed for the EH method. In addition, the LST images filled using the RSDAST and IMA methods had some outliers, while there were fewer obvious outliers in the LST images filled with the EH method. It was concluded that the EH method is a robust gapfilling method with a high accuracy.

Building footprint maps are vital to many remote sensing (RS) applications, such as 3-D building modeling, urban planning, and disaster management. Due to the complexity of buildings, the accurate and reliable generation of the building footprint from RS imagery is still a challenging task. In this article, an end-to-end building footprint generation approach that integrates convolution neural network (CNN) and graph model is proposed. CNN serves as the feature extractor, while the graph model can take spatial correlation into consideration. Moreover, we propose to implement the feature pairwise conditional random field (FPCRF) as a graph model to preserve sharp boundaries and fine-grained segmentation. Experiments are conducted on four different data sets: 1) Planetscope satellite imagery of the cities of Munich, Paris, Rome, and Zurich; 2) ISPRS Benchmark data from the city of Potsdam; 3) Dstl Kaggle data set; and 4) Inria Aerial Image Labeling data of Austin, Chicago, Kitsap County, Western Tyrol, and Vienna. It is found that the proposed end-to-end building footprint generation framework with the FPCRF as the graph model can further improve the accuracy of building footprint generation by using only CNN, which is the current state of the art.

This paper presents a new semi-supervised method for land cover classification using Sentinel-2 time-series images, which can deal with the problem of unclear observations. First, the MCCR method, which is constituted by the matrix completion (MC) of unclear observations and feature-adaptive collaborative representation (CR) based classifier, is adopted to handle the data quality problem. Second, by fusing RF, AdaBoost, and MCCR, a tri-training process is proposed to iteratively select the semi-labeled samples, considering the difference of classification certainty in different classifiers and classes. Experiments on two sets of Sentinel-2 images are conducted to validate the effectiveness of the proposed semi-supervised method.

Land-cover mapping over urban areas using Landsat imagery has attracted considerable attention in recent years as it can promptly and accurately reflect the biophysical composition status of the urban landscape and allow further applications such as urban planning and risk management. However, due to the large diversity across different urban landscapes, adequate training sample collection for urban area mapping is both challenging and time-consuming. In this paper, we propose a novel unsupervised sample collection method for mapping urban areas using Landsat imagery. Specifically, the idea is to select reliable, representative, and diverse training samples from the images in a two-stage and iterative manner, based on a set of spectral indices (vegetation, impervious surface, soil, water). To validate the effectiveness and robustness of the proposed method, a synthetic data set was designed and a series of Landsat images over 39 representative cities from different biomes across the world was employed. The effectiveness of the proposed algorithm was quantitatively validated by assessing the quality of the automatically collected samples and the accuracy of the mapping results. In terms of the mapping performance, the proposed automatic approach can achieve a comparable mapping accuracy to supervised classification with manually collected samples. On the basis of the freely accessed Landsat data, the proposed approach demonstrates a promising potential for automatic large-scale (i.e., global) mapping over urban areas.

This paper presents a new building indicator based on the multi-angle images, the angular difference feature (ADF), which characterizes angular properties from high-resolution ZY-3 multi-view images. The method for detecting buildings based on ADF consists of two main steps: ADF feature extraction and a post-processing step to refine the results by simultaneously incorporating the spectral and geometrical information. Experiments are conducted with three ZY-3 images acquired over Chinese cities. The proposed ADF achieves promising building detection performance over both highly dense urban areas and suburban areas, with an overall accuracy of better than 89% for all the three data sets.

In this article, we propose an automatic method for building area extraction from optical high-resolution imagery by using the recently developed morphological building index (MBI). First, the original MBI feature is calculated to highlight the potential buildings in the imagery. Second, a post-processing framework is used to remove false alarms by taking advantages of the spectral, shadow and shape information. Third, an intensity feature of building area is generated from the buildings and finally the building area result is obtained. Experiments on two high-resolution images are conducted to validate the effectiveness and robustness of the proposed method.

In hyperspectral remote sensing data mining, it is important to take into account of both spectral and spatial information, such as the spectral signature, texture feature, and morphological property, to improve the performances, e.g., the image classification accuracy. In a feature representation point of view, a nature approach to handle this situation is to concatenate the spectral and spatial features into a single but high dimensional vector and then apply a certain dimension reduction technique directly on that concatenated vector before feed it into the subsequent classifier. However, multiple features from various domains definitely have different physical meanings and statistical properties, and thus such concatenation has not efficiently explore the complementary properties among different features, which should benefit for boost the feature discriminability. Furthermore, it is also difficult to interpret the transformed results of the concatenated vector. Consequently, finding a physically meaningful consensus low dimensional feature representation of original multiple features is still a challenging task. In order to address these issues, we propose a novel feature learning framework, i.e., the simultaneous spectral-spatial feature selection and extraction algorithm, for hyperspectral images spectral-spatial feature representation and classification. Specifically, the proposed method learns a latent low dimensional subspace by projecting the spectral-spatial feature into a common feature space, where the complementary information has been effectively exploited, and simultaneously, only the most significant original features have been transformed. Encouraging experimental results on three public available hyperspectral remote sensing datasets confirm that our proposed method is effective and efficient.

The mapping of urban areas at regional to global scales is a crucial task due to its value for environmental monitoring, habitat and biodiversity conservation, and decision-making. In most current applications, two techniques (i.e., supervised classification and data fusion) are widely applied in large-scale urban mapping. However, the costly training sample collection, inadequate data-source descriptions, and diverse urban characteristics (e.g., shape, size, socioeconomic status, and physical environment) are challenging problems for the urban mapping approaches. In this context, aiming at effectively deriving accurate urban areas at a large scale, we propose a novel ensemble support vector machine (SVM) method which consists of three steps: 1) the automatic generation of training data to reduce labor costs; 2) the construction of an ensemble SVM model to effectively combine the multisource data (including remote sensing and socioeconomic data); and 3) an adaptive patch-based thresholding technique to tackle the diverse urban characteristics. The proposed method is employed to map urban areas of China in 2005 and 2010, and the resulting maps are compared with the existing urban maps for 287 prefecture-level cities. It is found that our results present a satisfactory superiority, especially in challenging small cities, with a significant improvement in median Kappa (0.174 for 2005 and 0.203 for 2010). When incorporating moderate-resolution imaging spectroradiometer multispectral data as an additional source, the Kappa coefficient can be further raised by 0.028 for 2010. In general, the proposed method shows great potential for accurately mapping urban areas at regional, continental, or even global scales in a cost-effective manner.

Due to the urgent demand for remote sensing big data analysis, large-scale remote sensing image retrieval (LSRSIR) attracts increasing attention from researchers. Generally, LSRSIR can be divided into two categories as follows: uni-source LSRSIR (US-LSRSIR) and cross-source LSRSIR (CS-LSRSIR). More specifically, US-LSRSIR means the inquiry remote sensing image and images in the searching data set come from the same remote sensing data source, whereas CS-LSRSIR is designed to retrieve remote sensing images with a similar content to the inquiry remote sensing image that are from a different remote sensing data source. In the literature, US-LSRSIR has been widely exploited, but CS-LSRSIR is rarely discussed. In practical situations, remote sensing images from different kinds of remote sensing data sources are continually increasing, so there is a great motivation to exploit CS-LSRSIR. Therefore, this paper focuses on CS-LSRSIR. To cope with CS-LSRSIR, this paper proposes source-invariant deep hashing convolutional neural networks (SIDHCNNs), which can be optimized in an end-to-end manner using a series of well-designed optimization constraints. To quantitatively evaluate the proposed SIDHCNNs, we construct a dual-source remote sensing image data set that contains eight typical land-cover categories and 10 000 dual samples in each category. Extensive experiments show that the proposed SIDHCNNs can yield substantial improvements over several baselines involving the most recent techniques.

As one of the most challenging tasks of remote sensing big data mining, large-scale remote sensing image retrieval has attracted increasing attention from researchers. Existing large-scale remote sensing image retrieval approaches are generally implemented by using hashing learning methods, which take handcrafted features as inputs and map the high-dimensional feature vector to the low-dimensional binary feature vector to reduce feature-searching complexity levels. As a means of applying the merits of deep learning, this paper proposes a novel large-scale remote sensing image retrieval approach based on deep hashing neural networks (DHNNs). More specifically, DHNNs are composed of deep feature learning neural networks and hashing learning neural networks and can be optimized in an end-to-end manner. Rather than requiring to dedicate expertise and effort to the design of feature descriptors, we can automatically learn good feature extraction operations and feature hashing mapping under the supervision of labeled samples. To broaden the application field, DHNNs are evaluated under two representative remote sensing cases: scarce and sufficient labeled samples. To make up for a lack of labeled samples, DHNNs can be trained via transfer learning for the former case. For the latter case, DHNNs can be trained via supervised learning from scratch with the aid of a vast number of labeled samples. Extensive experiments on one public remote sensing image data set with a limited number of labeled samples and on another public data set with plenty of labeled samples show that the proposed remote sensing image retrieval approach based on DHNNs can remarkably outperform state-of-the-art methods under both of the examined conditions.

Classification is an important technique for remote sensing data interpretation. In order to enhance the performance of a supervised classifier and ensure the lowest possible cost of the training samples used in the process, active learning (AL) can be used to optimize the training sample set. At the same time, integrating spatial information can help to enhance the separability between similar classes, which can in turn reduce the need for training samples in AL. To effectively integrate spatial information into the AL framework, this paper proposes a new active relearning (ARL) model for remote sensing image classification. In particular, our model is used to relearn the spatial features on the classification map, which contributes significantly to enhancing the performance of the classifier. We integrate the relearning model into the AL framework, with the aim to accelerate the convergence of AL and further reduce the labeling cost. Under the newly developed ARL framework, we propose two spatial–spectral uncertainty criteria to optimize the procedure for selecting new training samples. Furthermore, an adaptive multiwindow ARL model is also introduced in this paper. Our experiments with two hyperspectral images and two very high resolution images indicate that the ARL model exhibits faster convergence speed with fewer samples than traditional AL methods. Our results also suggest that the proposed spatial–spectral uncertainty criteria and the multiwindow version can further improve the performance when implementing ARL.

In this article, we proposed a novel semi-supervised sparse representation classification for high resolution remote sensing image. First, collaborative representation mechanism that exploits the help from whole training information rather than from only the potential associated class can enhance the class recognition ability. Second, by taking advantage of spatial occurrence and alignment of class label, the adoption of the relearning can gradually learn the flexible class-oriented spatial pattern from the label space with alleviated computational complexity to enhance the original spectral characteristics. Third, inspired by the spatial smoothing phenomenon when spatial feature stacked, a novel stable self-learning method can be designed to automatically select informative unlabeled sample to help the limited supervised set. Experiments on two hyperspectral and high-spatial resolution images validated the effectiveness and robustness of the proposed algorithm.

Recent advances in airborne light detection and ranging (LiDAR) technology allow us to rapid measure the topographical information over large areas. LiDAR remote sensed data has been widely used in many applications, e.g. forest management, urban planning, disaster predictions, etc. However, extracting useful information from LiDAR data remains challenging, especially in the urban remote sensing, where many objects have the same elevation and are connected, such as road and parking lots, trees and buildings. In this work, we present a new method to extract geometric and textural information from LiDAR data by using attribute filters with partial reconstruction. The proposed method can separate the connected objects and better model the geometric and textural information than traditional connected filters (e.g. attribute filters). Experimental results on LiDAR data from the 2013 IEEE GRSS Data Fusion Contest demonstrate effectiveness of the proposed method. Compared to the methods using original LiDAR data or attribute profiles computed by traditional attribute filters, with the proposed method, overall classification accuracies were improved by 35% and 12%, respectively.

Urban villages (UVs) are a special type of urban settlements in China. Their spatial evolution in recent years has a close relationship with urban planning and economic development. However, the remote sensing community pays little attention to UVs. This paper presents a new semantic scene model based on primitive indexes for detecting UVs using high resolution remotely sensed imagery. The model represents scenes as characteristics histograms of primitive objects. In the experiments, UVs in the main urban areas of Shenzhen over 2003–2012 were mapped. The proposed model outperformed conventional scene models quantitatively and visually, and showed good transferability across multitemporal images.

In the last few years, integrating spatial information into active learning framework has been gaining growing interest in the remote sensing community to optimize the collection of training sample set for supervised image classification. We address this problem from two directions. One of the directions focus on improving the classifier's performance to reduce the need for training samples. For this purpose, relearning model is introduced to combine the active learning framework to form mutually reinforcing process. In the meantime, another direction focus on the way to select most informative samples. For this purpose, new uncertainty criterion is proposed to favor the selection of samples not only with most spectral uncertainty, but also located in most uncertain spatial regions. Experiments on hyperspectral image show the effectiveness of proposed active learning framework.

Markov random field (MRF)based spatial regularizing methodology can improve the maps by imposing a spatial smoothness prior on the image grid, but also leads to oversmoothing at image boundary areas. This problem is caused by the reason that classic isotropous smoothness prior cannot take local discontinuities into account. In this context, this paper proposes a novel two-step MRF regularization algorithm, which addresses the problem by combining both spectral and class cost in spatial modules. The developed MRF method first establishes a spatial energy function integrating local spectral dissimilarity to smooth the initial classification map while preserving object boundaries. Second, a new anisotropic spatial energy function integrating the class co-occurrence dependency is constructed to regularize pixels around object boundaries. The effectiveness of the proposed MRF method is validated by a series of remote sensing data sets. The obtained results indicate that the method can significantly improve the classification accuracy with regards to traditional MRF classification models.

For a long time, the available global products of the urban area extent were limited to a coarse spatial resolution, e.g., Moderate Resolution Imaging Spectroradiometer (MODIS) global land cover (GLC) 500-m data and European Space Agency GlobCover 300-m data. This limitation was broken by the GlobeLand30 data, which is the world's first 30-m resolution GLC data set. However, detection accuracies of urban areas for the GlobeLand30 data (i.e., artificial surfaces) are not satisfactory. Therefore, in order to refine the detection accuracy of urban areas on the basis of the GlobeLand30 data, we propose a novel framework for urban area delineation by combining a set of remote sensing images and a geographical information system database, including the GlobeLand30 data, the National Land Cover Database (NLCD), the Land Use Interpretation Map (LUIM) of China, and Landsat images. First, the GlobeLand30 and land use/land cover products (e.g., NLCD or LUIM) are overlapped, and the study area is then separated into reliable and unreliable areas with a majority voting rule. Finally, the unreliable areas are confirmed by use of the Landsat data with a multiclassifier system. Experiments were conducted over two study areas that, respectively, represent typical patterns of American and Chinese urban areas: 1) the states of Utah, Mississippi, and Pennsylvania in the U.S. and 2) the provinces of Ningxia, Fujian, and Jilin in China. The results show that the accuracy of the GlobeLand30 data for urban area delineation can be significantly improved by integrating the multisource data and using the multiclassifier system.

The spatial information has been proved to be effective in improving the performance of spectral-based classification. However, it is difficult to describe different image scenes by using monofeature owing to complexity of the geospatial scenes. In this letter, a novel framework is developed to combine the multiple spectral and spatial features based on the Markov random field (MRF). Specifically, the pixels in an image are separated into reliable and unreliable ones according to the decision of multifeature classifications. The labels of the reliable pixels can be conveniently determined, but the unreliable pixels are then classified by fusing the multifeature classification results and reducing the classification uncertainties based on the MRF optimization. Experiments are conducted on three multispectral high-resolution images to verify the effectiveness of the proposed method. Several state-of-the-art multifeature classification methods are also achieved for the purpose of comparison. Moreover, three classifiers (i.e., multinomial logistic regression, support vector machines, and random forest) are used to test the performance of the proposed framework. It is shown that the proposed method can effectively integrate multiple features, yield promising results, and outperform other approaches compared.

This letter proposes an efficient framework for building detection from coarse to fine using morphological technique for high-resolution optical satellite imagery over urban areas. First, the preliminary result of building regions is obtained by the recently developed morphological building index (MBI) method, which is able to detect potential building structures. However, the raw results derived from the MBI can be subject to a number of false alarms, which are caused by bright soil, roads, and open areas. In this letter, we propose to use morphological spatial pattern analysis as a postprocessing to further optimize the MBI result and remove the commission errors. The original MBI result is then separated into seven mutually exclusive categories-core, islet, loop, bridge, perforation, edge, and branch-by applying a series of morphological transformations such as erosions, geodesic dilation, reconstruction by dilation, anchored skeletonization, etc. The objects corresponding to the generic categories are then analyzed, and the categories corresponding to building parts are maintained, while the others are abandoned. After this postprocessing, the small noisy patches and narrow roads, which were wrongly extracted by the MBI, can be removed. In addition, the shape of the buildings can also be regularized by removing the branches, and the holes contained in the building objects can be identified and filled. Extensive experiments performed on GeoEye-1 and WorldView-2 images confirm the effectiveness and robustness of the proposed morphological building detection framework.

In recent years, the support vector machines (SVMs) have been very successful in remote sensing image classification, particularly when dealing with high-dimensional data and limited training samples. Nevertheless, the vector-based feature alignment of the SVM can lead to an information loss in representation of hyperspectral images, which intrinsically have a tensor-based data structure. In this paper, a new multiclass support tensor machine (STM) is specifically developed for hyperspectral image classification. Our newly proposed STM processes the hyperspectral image as a data cube and then identifies the information classes in tensor space. The multiclass STM is developed from a set of binary STM classifiers using the one-against-one parallel strategy. As a part of our tensor-based processing chain, a multilinear principal component analysis (MPCA) is used for preprocessing, in order to reduce the tensorial data redundancy and, at the same time, preserve the tensorial structure information in sparse and high-order subspaces. As a result, the contributions of this work are twofold: a new multiclass STM model for hyperspectral image classification is developed, and a tensorial image interpretation framework is constructed, which provides a system consisting of tensor-based feature representation, feature extraction, and classification. Experiments with four hyperspectral data sets, covering agricultural and urban areas, are conducted to validate the effectiveness of the proposed framework. Our experimental results show that the proposed STM and MPCA-STM can achieve better results than traditional SVM-based classifiers.

The new generation of Earth observation sensors with high spatial resolution can provide detailed information for change detection. The widely used methods for high-resolution image change detection rely on textural/structural features. However, these spatial features always produce high-dimensional data space since they are related to a series of parameters, e.g., window sizes and directions. Machine learning methods are also commonly employed, but their performances are subject to the quantity and quality of the training samples, and hence, much effort should be made to collect the high-quality samples. To address these problems, in this study, a novel multiindex automatic change detection method is proposed for the high-resolution imagery. The notable advantages of the proposed model include the following: 1) Complicated urban scenes are represented by a set of low dimensional but semantic information indexes, replacing the high-dimensional but low-level features (e.g., textural and structural features), and 2) the change detection model is carried out automatically without using training samples since the information indexes can directly indicate the primitive urban classes. The multiindex representation refers to the enhanced vegetation index, the water index, and the recently developed morphological building index. Experiments were conducted on the multitemporal WorldView-2 images over Shenzhen City (south of China) and Kuala Lumpur (the capital of Malaysia), where promising results were achieved by the proposed method. Moreover, the traditional methods based on the state-of-the-art textural/morphological features were also implemented for the purpose of comparison, which further validates the advantages of our proposed model.

Remote sensing images exhibit significant contrast and intensity regions and edges, which makes them highly suitable for using different texture features to properly represent and classify the objects that they contain. In this paper, we present a new technique based on multiple morphological component analysis (MMCA) that exploits multiple textural features for decomposition of remote sensing images. The proposed MMCA framework separates a given image into multiple pairs of morphological components (MCs) based on different textural features, with the ultimate goal of improving the signal-to-noise level and the data separability. A distinguishing feature of our proposed approach is the possibility to retrieve detailed image texture information, rather than using a single spatial characteristic of the texture. In this paper, four textural features: content, coarseness, contrast, and directionality (including horizontal and vertical), are considered for generating the MCs. In order to evaluate the obtained MCs, we conduct classification by using both remotely sensed hyperspectral and polarimetric synthetic aperture radar (SAR) scenes, showing the capacity of the proposed method to deal with different kinds of remotely sensed images. The obtained results indicate that the proposed MMCA framework can lead to very good classification performances in different analysis scenarios with limited training samples.

To improve the accuracy of the traditional NASA Team (NT) sea ice concentration (SIC) algorithm, a new SIC estimation method is proposed by combining the NT algorithm and a numerical optimization technique with Special Sensor Microwave/Imager (SSM/I) data. In this method, the noise is taken into consideration to improve the SIC estimation equation, and then, the least squares method is used to further optimize the estimation results from the improved equation. Validation was performed using a comparison between the results from the SSM/I-based SICs (the proposed method, the NT algorithm, and the bootstrap algorithm) and in situ data. The quantitative results show that the proposed method generates a more accurate SIC with smaller bias (-3.2-2.8) and root-mean-square error (7.7-18.4) than the other two algorithms.

A wavelength-dependent light detection and ranging (LiDAR) backscatter intensity calibration method was developed to maximize the advantages of a multispectral LiDAR system. We established a spectral ratio calibration method for multispectral LiDAR and investigated the effective calibration procedure for the mixed measurement of the effect of incident angle and surface roughness. Experiment results showed that the proposed LiDAR spectral ratio is insensitive to sensor-related factors and advantageous in calibrating the effect of incidence angle and surface roughness. As the product of the LiDAR calibration procedure based on spectral ratio, extended vegetation indexes significantly improve the classification accuracy.

Due to the rapid urbanization of China, many villages in the urban fringe are enveloped by ever-expanding cities and become so-called urban villages (UVs) with substandard living conditions. Despite physical similarities to informal settlements in other countries (e.g., slums in India), UVs have access to basic public services, and more importantly, villagers own the land legitimately. The resulting socio-economic impact on urban development attracts increasing interest. However, the identification of UVs in previous studies relies on fieldwork, leading to late and incomplete analyses. In this paper, we present three scene-based methods for detecting UVs using high-resolution remotely sensed imagery based on a novel multi-index scene model and two popular scene models, i.e., bag-of-visual-words and supervised latent Dirichlet allocation. In the experiments, our index-based approach produced Kappa values around 0.82 and outperformed conventional models both quantitatively and visually. Moreover, we performed multitemporal classification to evaluate the transferability of training samples across multitemporal images with respect to three methods, and the index-based approach yielded best results again. Finally, using the detection results, we conducted a systematic spatiotemporal analysis of UVs in Shenzhen and Wuhan, two mega cities of China. At the city level, we observe the decline of UVs in urban areas over the recent years. At the block level, we characterize UVs quantitatively from physical and geometrical perspectives and investigate the relationships between UVs and other geographic features. In both levels, the comparison between UVs in Shenzhen and Wuhan is made, and the variations within and across cities are revealed.

Abstract—Hyperspectral image classification has been an active topic of research in recent years. In the past, many different types of features have been extracted (using both linear and nonlinear strategies) for classification problems. On the one hand, some approaches have exploited the original spectral information or other features linearly derived from such information in order to have classes which are linearly separable. On the other hand, other techniques have exploited features obtained through nonlinear transformations intended to reduce data dimensionality, to better model the inherent nonlinearity of the original data (e.g., kernels) or to adequately exploit the spatial information contained in the scene (e.g., using morphological analysis). Special attention has been given to techniques able to exploit a single kind of features, such as composite kernel learning or multiple kernel learning, developed in order to deal with multiple kernels. However, few

In this paper, we proposed a relearning method for classification post-processing (CPP). CPP can be viewed as a label refinement method to improve the classification accuracy. The proposed approach considers the frequency and spatial arrangements of the labels to enhance the classification performance by iteratively learning the classification map. Experiments conducted on a series of images obtained by different sensors show that, the proposed relearning approach present promising performances compared to the state-of-the-art techniques such as filtering, Markov random field (MRF) and object-based voting.

In this paper, a multi-index learning (MIL) approach is proposed to represent and classify the complex urban scenes using a set of low-dimension information indices instead of the traditional high-dimensional spatial features. Specifically, two categories of indices are proposed: 1) Primitive indices (PI), involving a series of basic urban primitives, e.g., buildings, shadow, vegetation; and 2) Variation indices (VI), describing the spectral and spatial variation of the urban scenes. Experiments conducted on a large-scale image (260 km2) captured by the ZY3 satellite (the first Chinese civilian high-resolution satellite) show that the proposed MIL approach can provide promising accuracies even though the complicated urban landscape is represented via low-dimensional feature space. The satisfactory results achieved by the MIL can be attributed to the low-dimensional but high-level semantic information considered.

A 3-D wavelet-transform-based texture feature extraction algorithm for the classification of urban multi/hyperspectral imagery is investigated in this study. It is widely agreed that it is necessary to simultaneously exploit the spectral and spatial information for image classification. In this context, the 3-D discrete wavelet transform (3-D DWT) is studied since it considers the local imagery patch as a cube and, hence, is capable of representing the imagery information in both spectral and spatial domains. The notable characteristic of the 3-D DWT is the ability to decompose an image into a set of spectral-spatial components. Specifically, we propose three approaches for 3-D DWT texture extraction, namely, pixelwise, non-overlapping, and overlapping cube. Experiments conducted on AVIRIS hyperspectral and WorldView-2 multispectral images revealed that the 3-D DWT textures achieved much better results than the widely used spectral-spatial classification methods.

The first results of multispectral (MS) and panchromatic (PAN) image fusion for the ZiYuan-3 (ZY-3) satellite, which is China's first civilian high-resolution satellite, are announced in this study. To this end, the various commonly used image fusion (pan-sharpening) techniques are tested. However, traditionally, image fusion quality is assessed by measuring the spectral distortion between the original and the fused MS images. The traditional methods focus on the spectral information at the data level but fail to indicate the image content at the information level, which is more important for specific remote sensing applications. In this context, we propose an information-based approach for assessing the fused image quality by the use of a set of primitive indices which can be calculated automatically without a requirement for training samples or machine learning. Experiments are conducted using ZY-3 PAN and MS images from Wuhan, central China. One of the objectives of the experiments is to investigate the appropriate image fusion strategies for the ZY-3 satellite at both the data and information levels. On the other hand, the experiments also aim to reveal the inadequacy of the traditional image quality indices and the advantages of the proposed information indices for describing image content. It is suggested that an appropriate image quality index should take into account the global and local image features at both the data and information levels.

This paper develops several new strategies for remote sensing image classification postprocessing (CPP) and conducts a systematic study in this area. CPP is defined as a refinement of the labeling in a classified image in order to enhance its original classification accuracy. The current mainstream classification methods (preprocessing) extract additional spatial features in order to complement spectral information and enhance classification using spectral responses alone. On the other hand, however, the CPP methods, providing a new solution to improve classification accuracy by refining the initial result, have not received sufficient attention. They have potential for achieving comparable accuracy to the preprocessing methods but in a more direct and succinct way. In this paper, we consider four groups of CPP strategies: filtering; random field; object-based voting; and relearning. In addition to the state-of-the-art CPP algorithms, we also propose a series of new ones, e.g., anisotropic probability diffusion and primitive cooccurrence matrix. In experiments, a number of multisource remote sensing data sets are used for evaluation of the considered CPP algorithms. It is shown that all the CPP strategies are capable of providing more accurate results than the raw classification. Among them, the relearning approaches achieve the best results. In addition, our relearning algorithms are compared with the state-of-the-art spectral-spatial classification. The results obtained further verify the effectiveness of CPP in different remote sensing applications.

Target detection is one of the most important applications in hyperspectral remote sensing image analysis. However, the state-of-the-art machine-learning-based algorithms for hyperspectral target detection cannot perform well when the training samples, especially for the target samples, are limited in number. This is because the training data and test data are drawn from different distributions in practice and given a small-size training set in a high-dimensional space, traditional learning models without the sparse constraint face the over-fitting problem. Therefore, in this paper, we introduce a novel feature extraction algorithm named sparse transfer manifold embedding (STME), which can effectively and efficiently encode the discriminative information from limited training data and the sample distribution information from unlimited test data to find a low-dimensional feature embedding by a sparse transformation. Technically speaking, STME is particularly designed for hyperspectral target detection by introducing sparse and transfer constraints. As a result of this, it can avoid over-fitting when only very few training samples are provided. The proposed feature extraction algorithm was applied to extensive experiments to detect targets of interest, and STME showed the outstanding detection performance on most of the hyperspectral datasets.

The detection and identification of target pixels such as certain minerals and man-made objects from hyperspectral remote sensing images is of great interest for both civilian and military applications. However, due to the restriction in the spatial resolution of most airborne or satellite hyperspectral sensors, the targets often appear as subpixels in the hyperspectral image (HSI). The observed spectral feature of the desired target pixel (positive sample) is therefore a mixed signature of the reference target spectrum and the background pixels spectra (negative samples), which belong to various land cover classes. In this paper, we propose a novel supervised metric learning (SML) algorithm, which can effectively learn a distance metric for hyperspectral target detection, by which target pixels are easily detected in positive space while the background pixels are pushed into negative space as far as possible. The proposed SML algorithm first maximizes the distance between the positive and negative samples by an objective function of the supervised distance maximization. Then, by considering the variety of the background spectral features, we put a similarity propagation constraint into the SML to simultaneously link the target pixels with positive samples, as well as the background pixels with negative samples, which helps to reject false alarms in the target detection. Finally, a manifold smoothness regularization is imposed on the positive samples to preserve their local geometry in the obtained metric. Based on the public data sets of mineral detection in an Airborne Visible/Infrared Imaging Spectrometer image and fabric and vehicle detection in a Hyperspectral Mapper image, quantitative comparisons of several HSI target detection methods, as well as some state-of-the-art metric learning algorithms, were performed. All the experimental results demonstrate the effectiveness of the proposed SML algorithm for hyperspectral target detection.

This paper addresses the problem of automatic extraction of built-up areas from high-resolution remote sensing images. We propose a new building presence index from the point view of perception. We argue that built-up areas usually result in significant corners and junctions in high-resolution satellite images, due to the man-made structures and occlusion, and thus can be measured by the geometrical structures they contained. More precisely, we first detect corners and junctions by relying on a perception-inspired corner detector, called an a-contrario junction detector. Each detected corner is associated with a perceptual significance, which measures the structural saliency of the corner in the image and is independent of the contrast and scale. All these detected corners together with their significance are then used to compute the building index. The proposed approach is evaluated on a high-resolution satellite image set, including 15 big images from GeoEye-1, QuickBird and IKONOS. The results demonstrated that our method achieves the state-of-the-art results and can be used in practical applications.

This letter proposes a novel change detection model, focusing on building change information extraction from urban high-resolution imagery. It consists of two blocks: 1) building interest-point detection, using the morphological building index (MBI) and the Harris detector; and 2) multitemporal building interest-point matching and the fault-tolerant change detection. The proposed method is insensitive to the geometrical differences of buildings caused by different imaging conditions in the multitemporal high-resolution imagery and is able to significantly reduce false alarms. Experiments showed that the proposed method was effective for building change detection from multitemporal urban high-resolution images. Moreover, the effectiveness of the algorithm was validated by comparing with the morphological change vector analysis (CVA), parcel-based CVA, and MBI-based CVA.

An energy-driven total variation (TV) formulation is proposed for the segmentation of high spatial resolution remote-sensing imagery. The TV model is an effective tool for image processing operations such as restoration, enhancement, reconstruction, and diffusion. Due to the relationship between the TV model and the segmentation problem, in this letter, a TV-based approach is investigated for segmentation of high-spatial-resolution remote-sensing imagery. Subsequently, an object-based classification method, i.e., majority voting, is used to classify the segmented results. In experiments, the proposed TV-based method is compared with the widely used fractal net evolution approach and the clustering segmentation methods such as the expectation–maximization and$k$-means. The performances of the segmentation and the classification are evaluated based on both thematic and geometric indices.

In recent years, the resolution of remotely sensed imagery has become increasingly high in both the spectral and spatial domains, which simultaneously provides more plentiful spectral and spatial information. Accordingly, the accurate interpretation of high-resolution imagery depends on effective integration of the spectral, structural and semantic features contained in the images. In this paper, we propose a new multifeature model, aiming to construct a support vector machine (SVM) ensemble combining multiple spectral and spatial features at both pixel and object levels. The features employed in this study include a gray-level co-occurrence matrix, differential morphological profiles, and an urban complexity index. Subsequently, three algorithms are proposed to integrate the multifeature SVMs: certainty voting, probabilistic fusion, and an object-based semantic approach, respectively. The proposed algorithms are compared with other multifeature SVM methods including the vector stacking, feature selection, and composite kernels. Experiments are conducted on the hyperspectral digital imagery collection experiment DC Mall data set and two WorldView-2 data sets. It is found that the multifeature model with semantic-based postprocessing provides more accurate classification results (an accuracy improvement of 1-4% for the three experimental data sets) compared to the voting and probabilistic models.

In this paper, we propose a method for the dimensionality reduction (DR) of spectral-spatial features in hyperspectral images (HSIs), under the umbrella of multilinear algebra, i.e., the algebra of tensors. The proposed approach is a tensor extension of conventional supervised manifold-learning-based DR. In particular, we define a tensor organization scheme for representing a pixel's spectral-spatial feature and develop tensor discriminative locality alignment (TDLA) for removing redundant information for subsequent classification. The optimal solution of TDLA is obtained by alternately optimizing each mode of the input tensors. The methods are tested on three public real HSI data sets collected by hyperspectral digital imagery collection experiment, reflective optics system imaging spectrometer, and airborne visible/infrared imaging spectrometer. The classification results show significant improvements in classification accuracies while using a small number of features.

The sub-pixel mapping technique, which can provide a fine-resolution map of class labels, has attracted more and more attention in recent years. Generally speaking, there are two kinds of methods used to realize the sub-pixel labeling. The first kind are image reconstruction based methods, which first improve the spatial resolution of an image by the super-resolution technique, and then perform a hard classification on the super-resolved image. The second kind are pixel unmixing based methods, where the sub-pixel mapping is implemented based on the results of image unmixing. In this paper, we present a sparse representation method and a back-propagation (BP) neural network method for image reconstruction based and pixel unmixing based mapping, respectively. The advantages and disadvantages of both kinds of methods are analyzed and discussed.

In hyperspectral remote sensing image classification, multiple features, e.g., spectral, texture, and shape features, are employed to represent pixels from different perspectives. It has been widely acknowledged that properly combining multiple features always results in good classification performance. In this paper, we introduce the patch alignment framework to linearly combine multiple features in the optimal way and obtain a unified low-dimensional representation of these multiple features for subsequent classification. Each feature has its particular contribution to the unified representation determined by simultaneously optimizing the weights in the objective function. This scheme considers the specific statistical properties of each feature to achieve a physically meaningful unified low-dimensional representation of multiple features. Experiments on the classification of the hyperspectral digital imagery collection experiment and reflective optics system imaging spectrometer hyperspectral data sets suggest that this scheme is effective.

In remote-sensing image target recognition, the target or background object is usually transformed to a feature vector, such as a spectral feature vector. However, this kind of vector represents only one pixel of a remote-sensing image that considers the spectral information but ignores the spatial relationship of neighboring pixels (i.e., the local texture and structure). In this letter, we propose a new way to represent an image object as a multifeature tensor that encodes both the spectral and textural information (Gabor function) and then apply the support tensor machine for target recognition. A range of experiments demonstrates that the effectiveness of the proposed method can deliver a high and correct recognition rate with a small number of training samples.

The objective of this letter is to integrate multiscale information for urban mapping using very high resolution (VHR) imagery. Three multiscale fusion methods were presented: 1) vector stacking (VS); 2) multiple support vector machines (multi-SVMs) fuzzy output; and 3) multi-SVMs voting. Two kinds of spatial features were used to obtain multiscale representations of VHR images: morphological structural features and object-based approaches. In experiments, the Reflective Optics System Imaging Spectrometer-03 Pavia Center and University, the Hyperspectral Digital Imagery Collection Experiment Washington DC Mall, and the Quickbird Beijing data sets were used for algorithm validation. The experimental results revealed that, in most cases, the VS fusion outperformed other methods because it was able to create a new high-dimensional multiscale feature space and enhance the class separability. It was also shown that the multi-SVMs fuzzy fusion could optimize and reorganize the multiscale information effectively. Furthermore, multi-SVMs fuzzy output was better than multi-SVMs voting because the former was able to exploit the probabilistic output, while the latter only considered the crisp classification label. In addition, it is suggested that VS fusion is suitable for morphological features; however, for the object-based classification, the multiscale fusion methods do not necessarily yield better results than the single-scale classification in terms of accuracies.

The extraction of spatial details is crucial for fusion quality. An efficient way is to exploit the difference between high-resolution panchromatic (Pan) images and low-resolution Pan (LRP), which is to be simulated by weighted average value from low-resolution multispectral images. To obtain the weighting coefficients with multivariate linear regression, three issues were discussed, and corresponding solutions were proposed in this letter. The proposed method consists of separating high-frequency pixels from low-frequency pixels using support vector machine and selecting observations that are evenly distributed by a bucketing technique and forcing coefficients to be sound physically by constrained least squares. Validation experiments are undertaken using three IKONOS data sets, and fusion results are compared against four popular methods. The results show that the proposed method can simulate LRP soundly and therefore achieve a better fusion quality.

This letter aims to exploit morphological textures in discriminating three mangrove species and surrounding environment with multispectral IKONOS imagery in a study area on the Caribbean coast of Panama. Morphological texture features are utilized to distinguish red (Rhizophora mangle), white (Laguncularia racemosa), and black (Avicennia germinans) mangroves and rainforest regions. Meanwhile, two fusion methods are presented, i.e., vector stacking and support vector machine (SVM) output fusion, for integrating the hybrid spectral-textural features. For comparison purposes, the object-based analysis and the gray-level co-occurrence matrix (GLCM) textures are adopted. Results revealed that the morphological feature opening by reconstruction (OBR) followed by closing by reconstruction (CBR) and its dual operator CBR followed by OBR gave very promising accuracies for both mangrove discrimination (89.1% and 91.1%, respectively) and forest mapping (91.4% and 93.7%, respectively), compared with the object-based analysis (80.5% for mangrove discrimination and 82.9% for forest mapping) and the GLCM method (81.9% and 87.2%, respectively). With respect to the spectral-textural information fusion algorithms, experiments showed that the SVM output fusion could obtain an additional 2.0% accuracy improvement than the vector-stacking approach.

In this paper, an adaptive mean-shift (MS) analysis framework is proposed for object extraction and classification of hyperspectral imagery over urban areas. The basic idea is to apply an MS to obtain an object-oriented representation of hyperspectral data and then use support vector machine to interpret the feature set. In order to employ MS for hyperspectral data effectively, a feature-extraction algorithm, nonnegative matrix factorization, is utilized to reduce the high-dimensional feature space. Furthermore, two bandwidth-selection algorithms are proposed for the MS procedure. One is based on the local structures, and the other exploits separability analysis. Experiments are conducted on two hyperspectral data sets, the DC Mall hyperspectral digital-imagery collection experiment and the Purdue campus hyperspectral mapper images. We evaluate and compare the proposed approach with the well-known commercial software eCognition (object-based analysis approach) and an effective spectral/spatial classifier for hyperspectral data, namely, the derivative of the morphological profile. Experimental results show that the proposed MS-based analysis system is robust and obviously outperforms the other methods.

Classification and extraction of spatial features are investigated in urban areas from high spatial resolution multispectral imagery. The proposed approach consists of three steps. First, as an extension of our previous work [pixel shape index (PSI)], a structural feature set (SFS) is proposed to extract the statistical features of the direction-lines histogram. Second, some methods of dimension reduction, including independent component analysis, decision boundary feature extraction, and the similarity-index feature selection, are implemented for the proposed SFS to reduce information redundancy. Third, four classifiers, the maximum-likelihood classifier, backpropagation neural network, probability neural network based on expectation-maximization training, and support vector machine, are compared to assess SFS and other spatial feature sets. We evaluate the proposed approach on two QuickBird datasets, and the results show that the new set of reduced spatial features has better performance than the existing length-width extraction algorithm and PSI

An adaptive multiscale information fusion algorithm is proposed to extract the spatial features and classify IKONOS multispectral imagery. It is well known that combining spectral and spatial information can improve land use classification of very high resolution data. However, many spatial measures refer to the window size problem, and the success of the classification procedure using spatial features depends largely on the window size that was selected. In this letter, we first propose an optimal window selection method, based on the spectral and edge information in a local region, for choosing the suitable window size adaptively; second, the multiscale information is fused based on the selected optimal window size. In order to evaluate the effectiveness of the proposed multiscale feature fusion approach, the spatial features that were extracted by the gray-level cooccurrence matrix are utilized for multispectral IKONOS data. The results show that the proposed algorithm can select and fuse the multiscale features effectively and, at the same time, increase the classification accuracy.

Shape and spectra are both important features of high spatial resolution remotely sensed (HSRRS) imagery, and they are concrete manifestation of textures on such imagery. This paper presents a spatial feature index, pixel shape index (PSI), to describe the shape feature in a local area surrounding a pixel. PSI is a pixel-based feature which measures the gray similarity distance in every direction. As merely the shape feature is inadequate for classifying HSRRS imagery, a transformed spectral feature extracted by independent component analysis is added to the input vectors of our classifier, and this replaces the original multispectral bands. Meanwhile, a fast fusion algorithm that integrates both shape and spectral features using the support vector machine has been developed to interpret the complex input vectors. The results by PSI are compared with some spatial features extracted using wavelet transform, gray level co-occurrence matrix, and the length-width extraction algorithm to test its effectiveness. The experiments demonstrate that PSI is capable of describing shape features effectively and result in more accurate classifications than other methods. While it is found that spectral and shape features can complement each other and their integration can improve classification accuracy, the transformed spectral components are also found to be more suitable for classification