Core image processing tasks, such as super-resolution, denoising, deblurring, pansharpening, and atmospheric correction, underpin all optical remote sensing (RS) pipelines. Errors at this stage propagate through downstream applications, distorting land-cover maps, change detection, and climate records. Classical physics-based models capture sensor optics, radiometry, and geometry but struggle with complex noise and scene variability. In contrast, deep learning (DL) methods offer powerful data-driven solutions yet often act as closed boxes, ignoring physical constraints and overfitting to spurious patterns. Hybrid DL (HDL) approaches bridge this gap by integrating physical models with neural architectures, combining interpretability and data adaptivity. This article surveys the emerging landscape of HDL methods in RS image processing, outlining their theoretical foundations, motivations, and design philosophies. We categorize fusion strategies, from model-embedded schemes (e.g., plug-and-play (PnP) and unrolling) to model-guided learning (e.g., deep image prior (DIP) and unsupervised frameworks), and discuss how they enhance trust, robustness, and physical consistency in RS image analysis.

Multiple object tracking (MOT) from unmanned aerial vehicle (UAV) platforms requires efficient motion modeling. This is because UAV-MOT faces both local object motion and global camera motion. Motion blur also increases the difficulty of detecting large moving objects. Previous UAV motion modeling approaches either focus only on local motion or ignore motion blurring effects, thus limiting their tracking performance and speed. To address these issues, we propose the Motion Mamba Module, which explores both local and global motion features through cross-correlation and bi-directional Mamba Modules for better motion modeling. To address the detection difficulties caused by motion blur, we also design motion margin loss to effectively improve the detection accuracy of motion blurred objects. Based on the Motion Mamba module and motion margin loss, our proposed MM-Tracker surpasses the state-of-the-art in two widely open-source UAV-MOT datasets.

Achieving land cover change detection (LCCD) through remotely sensed images (RSIs) is important in the observation of the changes on the Earth’s surface. In such detection, spectral-reflectance noise and the uncertainty of the imaging external conditions for the bitemporal RSIs usually cause some salt-and-pepper noisy pixels in the results and reduce the change detection accuracy. In this article, a hierarchical feature-fusion triple network (HFTN) is proposed to improve the performance of LCCD with RSIs. Overall, the proposed HFTN aims to learn representative features to improve change detection performance via two feature learning enhancement strategies and a hierarchical feature-fusion mechanism. First, an image feature difference model is proposed to generate the input feature for the middle branch and guide the learning performance. Second, a progressive denoising module (PDM) is proposed and applied to each temporal image to reduce the noise before feeding the features into the backbone of the proposed HFTN. Finally, a hierarchical feature-fusion module (HFFM) is proposed to fuse the learned deep feature for generating a change-magnitude image. Additionally, multiscale convolution, cross-scale fusion, and a shared weight are adopted in the backbone of the proposed HFTN to further enhance the feature learning performance. Compared with eight state-of-the-art methods, experimental results verified the feasibility and superiority of the proposed HFTN for LCCD with RSIs. For example, the proposed HFTN achieved improvement rates of approximately 0.43%–11.83% for overall accuracy (OA) and 0.11%–4.81% for false alarms (FAs) across six pairs of real RSIs. The code can be available athttps://github.com/ImgSciGroup/HFTN-NET.git.

This study examines the microwave emission properties of snow-covered sea ice, modeled as a layer with rough top and bottom boundaries. The emission model is based on the first-order solution to the radiative transfer equation (RTE). This equation describes the brightness temperatures that propagate both upward and downward in the layer, and it is solved numerically using the eigenvalue method. Additionally, the model takes into account contributions from irregular boundaries as well as surface and volume scattering interactions. The study looks at the variability of brightness temperature and the growth of different ice types. It uses modulation theory to consider the roughness of the boundaries, showing how this affects the emission from various types of snow-covered sea ice. We used polarization ratio-gradient ratio (PR-GR) space to analyze ice-grown transitions quantitatively. The thresholds of PR were found to be sensitive to changes in ice concentration. It was observed that the threshold of PR did not change with the roughness of the top boundary. We also found that the roughness of the ice surface has a more significant impact on emission than that of the snow-covered ice. It also observed that roughness effects are more potent in H-polarization than in V-polarization for each of the seven ice types. The roughness effects are weaker at large look angles, indicating stronger volume scattering. The study concludes that the presence of roughness in the ice layer’s boundaries leads to noticeable variations in brightness temperature.

Supervised classification with hyperspectral remote-sensing images (HRSIs) plays an important role in practical applications. However, labeling samples with HRSIs for supervised classification is time-consuming and labor-intensive. In this letter, we propose a new sample enhancement approach to improve the classification performance of HRSIs. First, the uncertainty and representativeness of the sample are defined to achieve sample possibility measurement for each pixel, and some pixels with high possibility can be selected as candidate samples. Then, two rules related to label correlation analysis and spectral similarity are defined to further refine the candidate samples used for generating the final sample set. Finally, the above-mentioned steps are fused into an iterative algorithm to enhance and balance the training samples for each class. The feasibility of the proposed approach was verified by applying it to classification with two real HRSIs. A comparison with some typical traditional sample enhancement methods and widely used few-shot deep-learning methods indicated the advantages of the proposed approach for improving classification accuracies. The improvement achieved by our proposed approach is about 0.79% ~ 2.31% in terms of the overall accuracy (OA). The code of the proposed approach is available athttps://github.com/ImgSciGroup/2023-GRSL-SIEA.

The foundation model has recently garnered significant attention due to its potential to revolutionize the field of visual representation learning in a self-supervised manner. While most foundation models are tailored to effectively process RGB images for various visual tasks, there is a noticeable gap in research focused on spectral data, which offers valuable information for scene understanding, especially in remote sensing (RS) applications. To fill this gap, we created for the first time a universal RS foundation model, named SpectralGPT, which is purpose-built to handle spectral RS images using a novel 3D generative pretrained transformer (GPT). Compared to existing foundation models, SpectralGPT 1) accommodates input images with varying sizes, resolutions, time series, and regions in a progressive training fashion, enabling full utilization of extensive RS Big Data; 2) leverages 3D token generation for spatial-spectral coupling; 3) captures spectrally sequential patterns via multi-target reconstruction; and 4) trains on one million spectral RS images, yielding models with over 600 million parameters. Our evaluation highlights significant performance improvements with pretrained SpectralGPT models, signifying substantial potential in advancing spectral RS Big Data applications within the field of geoscience across four downstream tasks: single/multi-label scene classification, semantic segmentation, and change detection.

Sample augmentation is effective for improving the supervised performance of land-cover classification with hyperspectral remote sensed image (HRSI) when the training samples are limited. However, numerous existing methods have neglected, considering the interclass-imbalance problem in the process of sample augmentation. In this work, new sample generation and sample balance strategies were promoted and simultaneously combined into an iteration for balancing and improving classification performance with HRSI. First, a sample augmentation with superpixel’s constraint (SASC) is designed to augment the initial training samples set to avoid the overfitting of a sample generation neural network. Second, sample generation based on generative adversarial network (SGGAN) was proposed to generate samples for each class. Then, the proposed SASC, SGGAN, and a pattern recognition neural network named 3 dimensions-convolutional neural network (3-D-CNN) are combined into an iterative classification process called iterative sample generation and balance (ISGB) for balancing the user’s accuracy for each class and optimizing the classification performance. Experiments on four widely used HRSIs are performed. The results when compared with eight state-of-the-art methods based on few-shot learning and generative adversarial network (GAN) efficiently demonstrate the feasibility and superiorities of the proposed approach for improving land-cover classification performance when the initial samples are limited. Moreover, the comparisons of the standard deviation of the user’s accuracies (SDUA) demonstrated the balancing ability of the proposed approach. The code of the proposed approach is available athttps://github.com/ImgSciGroup/ISGBA.

Very limited training samples pose significant challenges for hyperspectral image (HSI) classification. To address this issue, small-sample learning methods based on classical machine learning or deep learning offer promising solutions. In this article, a novel two-stage learning-based small-sample classification framework is proposed for HSIs, termed edge-preserving features-based smooth ordering (EPFSO). In the proposed EPFSO, a self-training approach and two screening mechanisms are designed to iteratively learn newly labeled samples from a vast pool of unlabeled samples, thereby enhancing classification accuracies by incorporating these additional samples into the training set. The preprocessing step involves using edge-preserving filters to extract key features and generate low-dimensional feature images. Subsequently, all samples are ordered based on spectral similarity and spatial proximity, resulting in a smooth 1-D signal. In the case of limited labeled samples, a specialized self-training approach based on linear interpolation is utilized to iteratively learn newly labeled samples from unlabeled samples. This process continues until no further labeled samples are introduced, enabling gradual improvement in classification performance. In addition, two screening mechanisms are designed into the self-training process to strike a balance between the reliability and quantity of newly labeled samples. Finally, once a sufficient number of training samples are available, a majority voting mechanism is employed to efficiently classify the remaining samples. Experimental results on three open HSI datasets demonstrate that the proposed EPFSO framework outperforms several state-of-the-art methods, including six deep learning approaches. This validates the attractiveness of using EPFSO to address the challenges associated with limited labeled samples.

A comprehensive exploration of the transformer and dual-receiver system-based direction-of-arrival (DOA) estimation is presented in the context of sea surface scattering, particularly under varying sea conditions. A bidirectional encoder representation from transformer (BERT) with a physics-based loss function is utilized to process two individual channel radars. The datasets are the radar scattering coefficients of sea surface simulated at C-band for copolarizations and cross-polarizations. Through detailed analysis of simulated datasets and root mean square error (RMSE) evaluations, the model’s performance is investigated across different observation modes, namely, the co-polar (CP), co-azimuth (CA), full-bistatic (FB), and Beaufort wind scale from 3 to 5. Our study demonstrates that the bidirectional encoder representation from transformer model, employing a physics-based loss function, outperforms the baseline long short-term memory (LSTM) model, especially under high noise levels and with larger datasets. Significant correlations between wind conditions and DOA accuracy are observed, highlighting the bidirectional encoder representation from transformer model’s adaptability to dynamic environmental factors, particularly under increased wind scales. The choice of observation mode, with CP and FB consistently outperforming CA, proves pivotal. Precise simulation of speckle variations and optimized observation mode selection are identified as crucial avenues for enhancing the model’s practical utility.

Labeled samples are important in achieving land cover change detection (LCCD) tasks via deep learning techniques with remote sensing images. However, labeling samples for change detection with bitemporal remote sensing images is labor-intensive and time-consuming. Moreover, manually labeling samples between bitemporal images requires professional knowledge for practitioners. To address this problem in this article, an iterative training sample augmentation (ITSA) strategy to couple with a deep learning neural network for improving LCCD performance is proposed here. In the proposed ITSA, we start by measuring the similarity between an initial sample and its four-quarter-overlapped neighboring blocks. If the similarity satisfies a predefined constraint, then a neighboring block will be selected as the potential sample. Next, a neural network is trained with renewed samples and used to predict an intermediate result. Finally, these operations are fused into an iterative algorithm to achieve the training and prediction of a neural network. The performance of the proposed ITSA strategy is verified with some widely used change detection deep learning networks using seven pairs of real remote sensing images. The excellent visual performance and quantitative comparisons from the experiments clearly indicate that detection accuracies of LCCD can be effectively improved when a deep learning network is coupled with the proposed ITSA. For example, compared with some state-of-the-art methods, the quantitative improvement is 0.38%-7.53% in terms of overall accuracy. Moreover, the improvement is robust, generic to both homogeneous and heterogeneous images, and universally adaptive to various neural networks of LCCD. The code will be available at https://github.com/ImgSciGroup/ITSA.

A Bayesian deep restricted Boltzmann-Kohonen architecture for data clustering termed deep restricted Boltzmann machine (DRBM)-ClustNet is proposed. This core-clustering engine consists of a DRBM for processing unlabeled data by creating new features that are uncorrelated and have large variance with each other. Next, the number of clusters is predicted using the Bayesian information criterion (BIC), followed by a Kohonen network (KN)-based clustering layer. The processing of unlabeled data is done in three stages for efficient clustering of the nonlinearly separable datasets. In the first stage, DRBM performs nonlinear feature extraction by capturing the highly complex data representation by projecting the feature vectors of d dimensions into n dimensions. Most clustering algorithms require the number of clusters to be decided a priori; hence, here, to automate the number of clusters in the second stage, we use BIC. In the third stage, the number of clusters derived from BIC forms the input for the KN, which performs clustering of the feature-extracted data obtained from the DRBM. This method overcomes the general disadvantages of clustering algorithms, such as the prior specification of the number of clusters, convergence to local optima, and poor clustering accuracy on nonlinear datasets. In this research, we use two synthetic datasets, 15 benchmark datasets from the UCI Machine Learning repository, and four image datasets to analyze the DRBM-ClustNet. The proposed framework is evaluated based on clustering accuracy and ranked against other state-of-the-art clustering methods. The obtained results demonstrate that the DRBM-ClustNet outperforms state-of-the-art clustering algorithms.

Land cover change detection (LCCD) with bitemporal remote sensing images has been widely used in practical applications. However, when the bitemporal images are multimodal remote sensing images (MRSIs) which are acquired with different sensors, the change detection performance may be unsatisfactory, because MRSIs cannot be compared directly to generate a change magnitude and obtain a change detection map. Here a novel approach is proposed to overcome this problem, i.e., the Enhanced UNet (E-UNet) which learns deep shared features from MRSIs to achieve change detection with MRSIs. First, apre-event image to post-eventimage (P2P) transformation module based on classical Cycle-consistent Generative Adversarial Network (CGAN) is suggested to embed at the head of the proposed E-UNet to translate the pre-event image to a post-event image one. Then, multi-scale convolutions are added at each encoding layer to capture the various shapes and sizes of ground targets. Finally, a Polarized Self-Attention (PSA) module is employed before beginning the decoding progress of E-UNet with an aim to pay extra attention to changed areas. Compared with five typical state-of-the-art methods, experimental results based on two pairs of MRSIs well demonstrated the feasibility and advantages of the proposed E-UNet for LCCD with MRSIs in terms of visual observations and quantitative evaluations. For example, the improvement is 4.19% and 4.75% in terms of the overall accuracy for the Sardinia dataset and California dataset, respectively. The code of the proposed approach can be found at https://github.com/ImgSciGroup/E-UNet.

In the last decade, deep learning (DL) has significantly impacted industry and science. Initially largely motivated by computer vision tasks in 2-D imagery, the focus has shifted toward 3-D data analysis. In particular, 3-D surface reconstruction, i.e., reconstructing a 3-D shape from sparse input, is of great interest to a large variety of application fields. DL-based approaches show promising quantitative and qualitative surface reconstruction performance compared to traditional computer vision and geometric algorithms. This survey provides a comprehensive overview of these DL-based methods for 3-D surface reconstruction. To this end, we will first discuss input data modalities, such as volumetric data, point clouds, and RGB, single-view, multiview, and depth images, along with corresponding acquisition technologies and common benchmark datasets. For practical purposes, we also discuss evaluation metrics enabling us to judge the reconstructive performance of different methods. The main part of the document will introduce a methodological taxonomy ranging from point-and mesh-based techniques to volumetric and implicit neural approaches. Recent research trends, both methodological and for applications, are highlighted, pointing toward future developments.

Deep learning techniques have become popular in land cover change detection (LCCD) with remote sensing images (RSIs). However, many existing networks mostly concentrate on learning deep features but without considering the effect of different features’ attention and fusion strategy on detection performance. In this paper, a novel hierarchical attention feature fusion (HAFF)-based network for LCCD with RSIs is proposed. In the proposed HAFF-based network, novel multi-scale convolution fusion filters (MCFFs) explore the global semantic feature of the interested targets from multi-perspectives ways. To achieve that objective, the proposed MCFFs are composed by a well-known position attention module (PAM) and a novel multi-perspectives feature filter block with different kernel sizes. In addition, a compound loss function was proposed for balancing the impact from the features at different levels in terms of backpropagation error. Experiments conducted on six pairs of real RSIs, including three pairs of homogeneous images and three pairs of heterogeneous images, confirmed the superiority of the proposed HAFF network over other cognate methods. Moreover, the ablation experiments further confirmed the feasibility and superiority of the proposed MCFFs, whereas quantitative observations indicated that competitive improvements are achieved by the proposed MCFFs in terms of all the evaluation indicators. The code for the proposed approach will be available at https://github.com/ImgSciGroup/HAFF.

Samples play a crucial role in the supervised classification of remote sensing images. However, labeling large samples for training a classifier or deep learning network is not only time-consuming but also labor-intensive. In this paper, a novel land cover classification with nonparametric sample augmentation is proposed to improve the performance of hyperspectral remote sensing images (HRSIs) classification. First, initial samples with limited quantity are selected randomly from the ground truth map. Second, based on the gray image, a nonparametric adaptive region generation (NARG) algorithm is developed for utilizing the contextual information around each sample. Then, an nonparametric sample augmentation algorithm is developed with NARG to explore reliable samples iteratively around each initial sample. Finally, the above steps are fused into an iterative progress to obtain the final classification map. Compared with some typical traditional methods and some widely used deep learning methods based on four real HRSIs, our proposed approach exhibits some advantages in improving the visual performance and quantitative accuracies of HRSIs classification, such as the improvement is about 2.0% ~ 10.34% for four real HRSIs in term of the overall accuracy.

Land cover change detection (LCCD) using bitemporal remote sensing images is a crucial task for identifying the change areas on the Earth’s surface. However, the utilization of hyperspectral remote sensing images (HRSIs) introduces challenges as the detection performance is affected by the spectral noise and deducing change detection accuracies. In this work, we concentrated on utilizing spatial-contextual information to improve the change detection performance while using HRSIs. First, a band selection approach is used to minimize the spectral redundancy of HRSIs. Second, an iterative spatial-adaptive filter is proposed to smooth the noise of HRSIs. Thereafter, the change magnitude between bitemporal HRSIs is measured by coupling change vector analysis and the adaptive region around each pixel, resulting in a change magnitude image (CMI). Subsequently, the CMI is divided into a binary change detection map by using an Ostu threshold method. The experimental results on three pairs of real HRSIs efficiently demonstrated the feasibility and superiorities of the proposed approach compared with six state-of-art methods. For example, the improvement rates are approximately 0.43%-11.83% and 1.05%-15.41% for overall accuracy and average accuracy, respectively. The code of our proposed approach will be available at: https://github.com/ImgSciGroup/2023-HSICD.

Spectral-spatial features are important for ground target identification and classification with High Spatial Resolution Remotely Sensed (HSRRS) Imagery. In this paper, two novel features, named the Gaussian-Weighting Spectral (GWS) feature and the Area Shape Index (ASI) feature, are proposed to complement the deficiency of the basic image feature for land cover classification with HSRRS imagery. The proposed GWS feature is an adaptive region-based feature that aims to improve the spectral homogeneity of a local area surrounding a pixel. Additionally, it is well known that the spectral feature is inadequate for classifying HSRRS imagery. Therefore, one spatial feature called the ASI feature is proposed here to describe the relationship between the area and shape for an adaptive region around each pixel. The proposed GWS and ASI features coupled with the basic red-green-blue feature are fed into a supervised classifier to obtain the final classification map. Experiments based on four real HSRRS images demonstrate that the proposed GWS and ASI features are capable of improving classification accuracies compared with some cognate state of the art methods. Moreover, the experiments also reveal that the proposed spectral-spatial features can complement each other for enhancing the classification performance with HSRRS images.

Land cover change detection (LCCD) with very high-resolution remote-sensing images (VHR_RSIs) is important in observing surface change on Earth. However, pseudo changes usually reduces the accuracy of the detection map. In this paper, novel piecewise distance based on adaptive region key-points extraction called sparse key-point distance (SKPD) is developed to measure the change magnitude between the bitemporal VHR_RSIs for LCCD. The proposed approach consists of three steps. First, an adaptive region generation algorithm is promoted for exploring spatial-contextual information. Then, the adaptive region around each pixel is sparsely represented with the box-whisker plot theory and the adaptive region is converted into a sparse key point vector. Finally, a piecewise distance is defined to measure the change magnitude between the bi-temporal images. While the entire VHR_RSIs are scanned and the proposed SKPD method proceeds on a pixel by pixel basis, a change magnitude image (CMI) can be generated and a binary threshold method can be applied on the CMI to obtain a change detection map. Experimental results based on four pairs of real VHR_RSIs and four state-of-the-art methods effectively demonstrated the superiority of the proposed approach for achieving LCCD with VHR_RSIs, such as the improvements for the four datasets are 5.25%, 14.76%, 18.13%, and 22.24%, respectively in terms of overall accuracy.

Change detection with heterogeneous remote sensing images (HRSIs) is attractive for observing the Earth’s surface when homogeneous images are unavailable. However, HRSIs cannot be compared directly because the imaging mechanisms for bitemporal HRSIs are different, and detecting change with HRSIs is challenging. In this letter, a simple yet effective deep learning approach based on the classical UNet is proposed. First, a pair of image patches are concatenated together to learn a shared abstract feature in both image patch domains. Then, a multiscale convolution module is embedded in a UNet backbone to cover the various sizes and shapes of ground targets in an image scene. Finally, a combined loss function, which incorporates the focal and dice losses with an adjustable parameter, was incorporated to alleviate the effect of the imbalanced quantity of positive and negative samples in the training progress. By comparisons with five state-of-the-art methods in three pairs of real HRSIs, the experimental results achieved by our proposed approach have the best overall accuracy (OA), average accuracy (AA), recall (RC), and F-Score that are more than 95%, 79%, 60%, and 61%, respectively. The quantitative results and visual performance indicated the feasibility and superiority of the proposed approach for detecting land cover change with HRSIs.

Landslide inventory mapping (LIM) plays an important role in landslide susceptibility analysis. Many LIM approaches based on change detection techniques have been proposed, but with various drawbacks. For example, existing approaches have limited capability to capture the objects of varying shapes/sizes present in an area impacted by landslide. Many existing approaches are supervised and require parameter tuning. Moreover, some methods are prone to salt-and-pepper noise. To overcome these limitations, in this letter, an algorithm based on automatic adaptive region extension using very-high-resolution remote sensing images is developed. First, a simple yet effective k-means clustering method is used to generate training samples for landslide and nonlandslide classes, which refer to changed and unchanged areas, respectively. Second, an automatic adaptive region extension algorithm is developed and applied to each pixel of the postevent image, and the label of an extended region around a pixel is determined by the nearest distance between the central pixel and the changed or unchanged samples. Finally, the labels of a pixel are recorded because a pixel in different adaptive regions may be reassigned dissimilar labels, and the final label of the pixel is consistent with its maximum assigned label. To verify the performance of the proposed approach, we conducted experiments on two different landslide sites with VHR remote sensing images in Lantau Island, Hong Kong, China. Experimental results clearly demonstrate that the proposed approach has several advantages in improving the performance of LIM with VHR remote sensing images.

Training samples are usually required to train a classifier for supervised classification of very high spatial resolution (VHR) remote sensing images. However, labeling samples is often a labor-intensive and time-consuming task. To solve this problem, this study integrates histogram distribution analysis, double-window flexible pace search (DFPS), and box–whisker plot (BP) techniques into an iterative algorithm to enrich training samples. The major steps of the proposed algorithm are given as follows. First, to acquire the feature distribution of a class, a histogram of each class (HOC) based on the raw classification map is generated. Second, to cover the spectral heterogeneity of an intraclass, some pixel points in each bin of HOC are selected as the coarse training sample set (CTS). Third, to further purify the CTS, DFPS, and BP techniques are adopted to exclude outlier samples and select the representative samples to signify the corresponding class. Finally, the refined training samples are used to retrain the classifier, and the preceding steps are constructed as an iterative algorithm. Experiments were performed on three real VHR remote sensing images to demonstrate the superiorities of the proposed approach in improving classification performance with respect to the maps obtained directly by the initial training set. In addition, compared with cognate state-of-the-art methods, the proposed approach achieved an approximately 2%–13% improvement in classification accuracy. Code available here:https://github.com/ImgSciGroup/IEEE-GRSL-GSEA-Code.

Many land cover change detection (LCCD) approaches applied on very high resolution (VHR) remote sensing images utilize spatial information by using a regular window or strict mathematical model. However, regular shape or strict models cannot fit the various shapes and sizes of the ground targets. In this article, a novel LCCD approach without the parameter is proposed to detect land cover change with VHR remote sensing images. First, an adaptive spatial-context extraction algorithm is applied to explore contextual information around a pixel. Second, the change magnitude between pairwise pixels is quantitatively measured by computing the band-to-band distance which is defined by the pairwise adaptive regions around the corresponding pixels. Finally, after the generation of a change magnitude image (CMI), a binary threshold method called double-window flexible pace search (DFPS) is adopted to divide CMI into a binary change detection map. The performance of the proposed approach is verified by comparing it with five state-of-the-art methods with three pairs of VHR images. The comparisons demonstrated that the proposed approach achieved the improved detected results comparing with state-of-the-art LCCD methods. The code of the proposed approach is available athttps://github.com/TongfeiLiu/ASEA-CD.

Land cover change detection (LCCD) with very high-resolution (VHR) remote sensing images has been widely used in various applications. However, pseudo-changes and noise usually affect the performance of detection map. In this letter, an object-oriented sorted-histogram similarity measurement (OSSM) is proposed for measuring the change magnitude between bi-temporal remote sensing images. First, multi-scale objects are acquired for the post-event image using a multi-scale segmentation algorithm, and then the pixels within each object are considered to construct the pairwise histograms and the bin of each histogram is sorted in descending order. Second, a bin-to-bin (B2B) distance is defined to measure the change magnitude between the pairwise object-based histograms, and the change magnitude image (CMI) is generated after all the bi-temporal images are scanned object by object. Finally, a simple yet effective method called Otsu is used to divide the CMI into binary change detection maps. The experiments on three pairs of VHR images produced promising results compared with five popular LCCD approaches, for example, the improvement is about 2.5% for F-score.

With the fast development of remote sensing platforms and sensors technology, change detection with heterogeneous remote sensing images (Hete-CD) has become an attractive topic in recent years and plays a vital role in land cover change detection for responding to natural disaster emergencies when homogeneous images are unavailable. Although Hete-CD has been developed for about three decades, and various related methods have been developed and applied successfully in practice, a systematic and comprehensive review of the current achievements regarding Hete-CD remains lacking. Therefore, in this article, we first present an overview of Hete-CD in terms of the related literature. Second, the major techniques of Hete-CD are reviewed in terms of publicly available datasets, the taxonomy of major techniques, results, performance, and quantitative evaluation. Then, some classical methods are selected for comparison and discussion. Finally, based on the discussion and literature review, challenges, opportunities, and future directions for Hete-CD are concluded. The review aims to provide a “one-stop-shop” understanding of the problems with the categories of existing approaches, open opportunities and challenges, and potential future directions for Hete-CD.

Land cover change detection (LCCD) using remote sensing images (RSIs) plays an important role in natural disaster evaluation, forest deformation monitoring, and wildfire destruction detection. However, bitemporal images are usually acquired at different atmospheric conditions, such as sun height and soil moisture, which usually cause pseudo and noise change into the change detection map. Changed areas on the ground also generally have various shapes and sizes, consequently making the utilization of spatial contextual information a challenging task. In this paper, we design a novel neural network with spatial-spectral attention mechanism and multi-scale dilation convolution modules. This work is based on the previously demonstrated promising performance of convolutional neural network for LCCD with RSIs and attempts to capture more positive changes and further enhance the detection accuracies. The learning of the proposed neural network is guided with a change magnitude image. The performance and feasibility of the proposed network are validated with four pairs of RSIs that depict real land cover change events on the Earth’s surface. Comparison of the performance of the proposed approach with that of five state-of-art methods indicates the superiority of the proposed network in terms of 10 quantitative evaluation metrics and visual performance. Such as, the proposed network achieved an improvement about 0.08%~14.87% in terms of OA for Dataset-A.

Landslide inventory mapping (LIM) is an important application in remote sensing for assisting in the relief of landslide geohazards. However, while conducting LIM tasks performing change detection analysis using bi-temporal very high-resolution (VHR) remote sensing images, due to landslide usually occurred in a mountain area, the phenological difference and outcrop rock may bring pseudo-changes to LIM results. In this paper, a novel change detection approach based on Adaptive Region Shape Similarity (ARSS) is proposed for LIM with VHR remote sensing images to improve detection performance. First, an adaptive region around each pixel is extended to explore the contextual information. Then, direction lines within an adaptive region are defined to describe the shape of the adaptive region. Finally, the pixels located on each direction line are taken into account to build the corresponding histogram. The shape similarity between the pairwise histogram curves is measured by using the Discrete Frchet Distance (DFD). Once the bi-temporal images are processed by using the abovementioned steps, a change magnitude image (CMI) is generated, while a threshold is then used to obtain a final binary change map. The proposed approach is applied to three pairs of landslide sites images acquired with aerial plane and one land use change dataset acquired by Quick Bird Satellite. Compared with ten state-of-the-art methods, the proposed approach achieved LIMs and detection results with higher accuracies and better performance.

In this article, we propose a bilevel segmentation framework with metacognitive learning (BS-McL) to detect power lines with an RGB camera mounted on an unmanned aerial vehicle (UAV) platform. The proposed framework consists of two levels based on spectral and spatial techniques. In the first level, spectral classification is carried out using the McL method, which is an evolving online learning neural network architecture. Due to similarities in spectral intensities, few nonpower line pixels are grouped along with power line pixels. The nonpower line pixels are removed by spatial segmentation in the second level. The second level includes morphological operations such as geometric features (shape and density indices), which are applied to detect the power lines. The processing steps of BS-McL are illustrated using a synthetic image of size$9 \times 6$pixels. Also, two datasets consisting of 64 images with varying backgrounds, different locations, and dimensions of power lines are used to demonstrate the performance of the proposed BS-McL. The obtained results for BS-McL are compared with five commonly used methods. For both datasets, the efficiency of the BS-McL for power line extraction is better than for the methods used for comparison. Furthermore, the trained knowledge from our experimental set-up (Dataset 1: suburban scene) can be transferred to another dataset that is available publicly (Dataset 2: urban and mountain scenes) if the power line spectral values are in relevance with the distribution in the training dataset. The proposed approach BS-McL is based on online learning with a self-adaptive architecture, which provides improved generalization ability.

Remote sensing image retrieval (RSIR), aiming at searching for a set of similar items to a given query image, is a very important task in remote sensing applications. Deep hashing learning as the current mainstream method has achieved satisfactory retrieval performance. On one hand, various deep neural networks are used to extract semantic features of remote sensing images. On the other hand, the hashing techniques are subsequently adopted to map the high-dimensional deep features to the low-dimensional binary codes. This kind of method attempts to learn one hash function for both the query and database samples in a symmetric way. However, with the number of database samples increasing, it is typically time-consuming to generate the hash codes of large-scale database images. In this article, we propose a novel deep hashing method, named asymmetric hash code learning (AHCL), for RSIR. The proposed AHCL generates the hash codes of query and database images in an asymmetric way. In more detail, the hash codes of query images are obtained by binarizing the output of the network, while the hash codes of database images are directly learned by solving the designed objective function. In addition, we combine the semantic information of each image and the similarity information of pairs of images as supervised information to train a deep hashing network, which improves the representation ability of deep features and hash codes. The experimental results on three public datasets demonstrate that the proposed method outperforms symmetric methods in terms of retrieval accuracy and efficiency. The source code is available athttps://github.com/weiweisong415/Demo_AHCL_for_TGRS2022.

Using computationally efficient techniques for transforming the massive amount of Remote Sensing (RS) data into scientific understanding is critical for Earth science. The utilization of efficient techniques through innovative computing systems in RS applications has become more widespread in recent years. The continuously increased use of Deep Learning (DL) as a specific type of Machine Learning (ML) for data-intensive problems (i.e., ‘big data’) requires powerful computing resources with equally increasing performance. This paper reviews recent advances in High-Performance Computing (HPC), Cloud Computing (CC), and Quantum Computing (QC) applied to RS problems. It thus represents a snapshot of the state-of-the-art in ML in the context of the most recent developments in those computing areas, including our lessons learned over the last years. Our paper also includes some recent challenges and good experiences by using Europeans fastest supercomputer for hyper-spectral and multi-spectral image analysis with state-of-the-art data analysis tools. It offers a thoughtful perspective of the potential and emerging challenges of applying innovative computing paradigms to RS problems.

The majority of the change detection (CD) methods consider spatial information by using a regular window or strict mathematical model. Moreover, these methods use the spectra directly to measure the change magnitude between bitemporal images. To solve this problem, local histogram-based analysis (LHBA) is proposed for detecting a land cover change in this letter. This new approach aims to inhibit the pseudo change by defining the local histogram trend (LHT) in an adaptive manner instead of using spectral values to measure change magnitude directly. In the proposed approach, the spatial information around each pixel is first exploited by defining an adaptive local histogram. The LHT distance between the pairwise local histograms is then developed to measure the change magnitude between the pairwise pixels of bitemporal images. Finally, the change magnitude image is generated, and a binary CD is achieved by a threshold method. Experiments based on two pairs of very high-resolution remote sensing images, which refer to land use change and landslides events, demonstrate the advantages and performance of the proposed approach.

This special section investigates the state-of-the-art in the analysis and processing of remotely sensed big data employing distributed computing architectures.

The large data volume and high algorithm complexity of hyperspectral image (HSI) problems have posed big challenges for efficient classification of massive HSI data repositories. Recently, cloud computing architectures have become more relevant to address the big computational challenges introduced in the HSI field. This article proposes an acceleration method for HSI classification that relies on scheduling metaheuristics to automatically and optimally distribute the workload of HSI applications across multiple computing resources on a cloud platform. By analyzing the procedure of a representative classification method, we first develop its distributed and parallel implementation based on the MapReduce mechanism on Apache Spark. The subtasks of the processing flow that can be processed in a distributed way are identified as divisible tasks. The optimal execution of this application on Spark is further formulated as a divisible scheduling framework that takes into account both task execution precedences and task divisibility when allocating the divisible and indivisible subtasks onto computing nodes. The formulated scheduling framework is an optimization procedure that searches for optimized task assignments and partition counts for divisible tasks. Two metaheuristic algorithms are developed to solve this divisible scheduling problem. The scheduling results provide an optimized solution to the automatic processing of HSI big data on clouds, improving the computational efficiency of HSI classification by exploring the parallelism during the parallel processing flow. Experimental results demonstrate that our scheduling-guided approach achieves remarkable speedups by facilitating the automatic processing of HSI classification on Spark, and is scalable to the increasing HSI data volume.

The recent availability of high-resolution multiview ZY-3 satellite images, with angular information, can provide an opportunity to capture 3-D structural features for classification. In high-resolution image classification over urban areas, objects with diverse vertical structures make urban landscape more heterogeneous in 3-D space and consequently can make the classification challenging. In this article, a novel multiangle gray-level cooccurrence tensor feature is proposed based on the multiview bands of the ZY-3 imagery, namely, GLCMMA–T. The GLCMMA–Tfeature captures the distributions of the gray-level spatial variation under different viewing angles, which can depict the 3-D textures and structures of urban objects. The spectral and GLCMMA–Ttensor features are interpreted by two 3-D convolutional neural network (CNN) streams and then concatenated as the input to the fully connected layer. This novel multispectral and multiangle 3-D convolutional neural network (M2-3-DCNN) combines the spectral and angular information, and the fused feature has the potential to provide a comprehensive description of urban objects with complex vertical structures. The experimental results on ZY-3 multiview images from four test areas indicate that the proposed method can significantly improve the classification accuracy when compared with several state-of-the-art multiangle features and deep-learning-based image classification methods.

Imbalanced training sets are known to produce suboptimal maps for supervised classification. Therefore, one challenge in mapping land cover is acquiring training data that will allow classification with high overall accuracy (OA) in which each class is also mapped onto similar user's accuracy. To solve this problem, we integrated local adaptive region and box-and-whisker plot (BP) techniques into an iterative algorithm to expand the size of the training sample for selected classes in this article. The major steps of the proposed algorithm are as follows. First, a very small initial training sample (ITS) for each class set is labeled manually. Second, potential new training samples are found within an adaptive region by conducting local spectral variation analysis. Lastly, three new training samples are acquired to capture information regarding intraclass variation; these samples lie in the lower, median, and upper quartiles of BP. After adding these new training samples to the ITS, classification is retrained and the process is continued iteratively until termination. The proposed approach was applied to three very high-resolution (VHR) remote-sensing images and compared with a set of cognate methods. The comparison demonstrated that the proposed approach produced the best result in terms of OA and exhibited superiority in balancing user's accuracy. For example, the proposed approach was typically 2%-10% more accurate than the compared methods in terms of OA and it generally yielded the most balanced classification.

Driven by the urgent demand for managing remote sensing big data, large-scale remote sensing image retrieval (RSIR) attracts increasing attention in the remote sensing field. In general, existing retrieval methods can be regarded as visual-based retrieval approaches that search and return a set of similar images to a given query image from a database. Although these retrieval methods have delivered good results, there is still a question that needs to be addressed: can we obtain the accurate semantic labels of the returned similar images to further help analyzing and processing imagery? To this end, in this article, we redefine the image retrieval problem as visual and semantic retrieval of images. Especially, we propose a novel deep hashing convolutional neural network (DHCNN) to retrieve similar images and classify their semantic labels simultaneously in a unified framework. In more detail, a convolutional neural network (CNN) is used to extract high-dimensional deep features. Then, a hash layer is perfectly inserted into the network to transfer the deep features into compact hash codes. In addition, a fully connected layer with a softmax function is performed on the hash layer to generate the probability distribution of each class. Finally, a loss function is elaborately designed to consider the label loss of each image and similarity loss of pairs of images simultaneously. Experimental results on three remote sensing data sets demonstrate that the proposed method can achieve state-of-art retrieval and classification performance.

A functional feature extraction method based on rational function approximation for hyperspectral image (HSI) classification is proposed. In digital imagery, the spectral information of a pixel can be regarded as a 1-D signal. An HSI is composed of these 1-D signals arranged in a certain spatial structure. According to the functional characteristic of hyperspectral data, 1-D signals can be approximated by a linear combination of basis functions. Thus, a joint rational basis function system (JRBFS) based on class adaptivity is here first built for an HSI by adaptive Fourier decomposition (AFD). Second, the functional representations (FRs) and corresponding reconstructed spectral curves are obtained by decomposing the original spectral information in a JRBFS. Furthermore, the functional spectral-spatial features are extracted on the basis of FRs by an edge-preserving filtering method, FR-EPFs. Finally, the functional spectral-spatial features are used for HSI classification by SVM. Experimental results for five commonly used HSI data sets demonstrate the effectiveness and advantages of the proposed method FR-EPFs.

Hyperspectral images (HSIs) are often degraded by different noise types such as Gaussian and sparse noise. In this letter, a hyperspectral mixed Gaussian and sparse noise reduction technique, the HyMiNoR, is proposed. The proposed technique, hierarchically, removes the mixed noise. First, the Gaussian noise is removed using a recently developed automatic hyperspectral noise removal technique called hyperspectral restoration (HyRes). Then, we develop a novel sparse noise removal technique to remove the sparse noise, including salt and pepper noise, missing pixels, and missing lines. The performance of the proposed approach has been validated using both real and simulated data sets. Results on the simulated data set confirm considerable improvements in terms of signal-to-noise ratio and singular angle distance compared to the state-of-the-art techniques used in the experiments. In addition, visual improvements can be clearly observed in the case of real data set experiments.

Very high-resolution (VHR) remote sensing images can geometrically depict ground targets in detail but are usually insufficient in the spectral domain. This characteristic leads to a considerable amount of noise and pseudo change in the produced binary change detection maps (BCDMs) when VHR remote sensing images are used for change detection. Here, to solve the aforementioned problem, an object-oriented key point vector distance (KPVD) is proposed to measure the change magnitude between bitemporal VHR images when land cover changes are detected. The proposed KPVD-based change detection approach comprises the following major steps. First, multiscale objects based on a postevent image are extracted by the fractional net evaluation segmentation approach, and then, the segments are taken as the unit for measuring the change magnitude between bitemporal images. Second, key points and the corresponding vector are defined to describe the object feature instead of using the total pixels within the object. Finally, KPVD is proposed to measure the change magnitude between the local areas referenced to the object in the bitemporal images. The change magnitude image (CMI) between the bitemporal images is generated while the entire images are scanned and processed object by object. A well-known automatic binary method, the Otsu approach, is employed in this article to divide CMI into a BCDM. Experimental results conducted on four real data sets demonstrate the feasibility and outperformance of the proposed KPVD-based change detection approach compared with five state-of-the-art methods in terms of visual performance and quantitative measurements.

This article proposes a functional data discriminant analysis (FDDA) method for hyperspectral image (HSI) classification. This method analyzes and processes the HSI data from a functional point of view, which is a novel perspective in HSI processing. The classical methods achieve dimensionality reduction by directly eliminating the redundancy of the HSI data. However, the proposed method extracts the functional features by utilizing the redundancy of the HSI data. Functional features can effectively reveal inherent characteristics of the HSI data with the change in the wavelengths. Based on this, a regularized weighted fitting model is first built for converting a spectral vector into a spectral curve. Second, an FDDA method defined in the function field is presented for extracting the functional features of the spectral curves. Finally, a novel spectral-spatial framework is designed for classification tasks of HSI data sets. Experimental results in three commonly used HSI data sets indicate that the proposed method is effective and leads to promising classification results compared with some benchmarking methods. More importantly, the work tries to diversify and develop the existing theory and methods of HSI classification from discrete (vector) data learning methods to continuous (functional) data learning methods.

Answering to metrological constraints typically required in the context of industrial and medical applications, a spectral difference space is introduced in this work. In this space, an acquired hyperspectral data is treated as measurements. Then, modelling the spectral difference space as multivariate Normal laws, a Gaussian mixture model is used in a classification task of remote sensing images. An encouraging result is obtained, comparing the proposed space with a data-driven one. Moreover, it offers a starting point in developing a directly interpretable spectral analysis tools.

This paper presents a new semi-supervised method for land cover classification using Sentinel-2 time-series images, which can deal with the problem of unclear observations. First, the MCCR method, which is constituted by the matrix completion (MC) of unclear observations and feature-adaptive collaborative representation (CR) based classifier, is adopted to handle the data quality problem. Second, by fusing RF, AdaBoost, and MCCR, a tri-training process is proposed to iteratively select the semi-labeled samples, considering the difference of classification certainty in different classifiers and classes. Experiments on two sets of Sentinel-2 images are conducted to validate the effectiveness of the proposed semi-supervised method.

In this letter, we first propose multisensor composite kernel (MCK) extreme learning machines to fuse hyperspectral and light detection and ranging (LiDAR) features effectively. Then, based on the MCK, we develop a fully automatic fusion framework. In the proposed framework, spatial and elevation features of hyperspectral and LiDAR data are first extracted using extinction profiles. Then, hyperspectral Stein's unbiased risk estimator is utilized to extract the subspace (informative features) of spectral, spatial, and elevation features. The obtained results indicate that the proposed approach can successfully integrate and classify hyperspectral and LiDAR images to provide accurate classification results classification accuracies in an automatic manner.

Since the 1960s, remote sensing (as an innovative, comprehensive, and interdisciplinary academic area) has been adopted in a wide range of disciplines related to Earth observation, including hydrology, ecology, oceanography, glaciology, geology, military, intelligence, business, economy, and planning [1]-[3]. The constant development of the remote sensing image acquisition technology now allows for the collection of a wide variety of images with different characteristics and resolutions, obtained by remote sensing instruments mounted on spacecraft or aircraft platforms. These images record some type of signal or energy measured from the Earth's surface, which depends on the type of sensor used.

Remote sensing has evolved into a multidisciplinary field involving many different areas such as sensor technology, computing, and advanced applications. Information extraction now plays a fundamental role in the exploitation of the massive amount of data collected by earth observation instruments. In this Point of View, the authors analyze the evolution of this field, identifying three main phases in its development. The first period, which was marked by advances in digital signal processing, led to a significant development of statistical processing methods. The second phase was based on advances in physical models and brought an era of quantitative remote sensing which lasted until the first decade of this century. In the third and current period, information extraction techniques are gradually adopting advanced artificial intelligence models in an effort to cope with the tremendous increase in data volume. This article describes some of these recent advances and addresses challenges caused by the 4Vs (volume, velocity, variety, and veracity) of big data. Finally, the authors offer insight into future directions in this multidisciplinary field.

Hyperspectral image (HSI) classification is a core task in the remote sensing community, and recently, deep learning-based methods have shown their capability of accurate classification of HSIs. Among the deep learning-based methods, deep convolutional neural networks (CNNs) have been widely used for the HSI classification. In order to obtain a good classification performance, substantial efforts are required to design a proper deep learning architecture. Furthermore, the manually designed architecture may not fit a specific data set very well. In this paper, the idea of automatic CNN for the HSI classification is proposed for the first time. First, a number of operations, including convolution, pooling, identity, and batch normalization, are selected. Then, a gradient descent-based search algorithm is used to effectively find the optimal deep architecture that is evaluated on the validation data set. After that, the best CNN architecture is selected as the model for the HSI classification. Specifically, the automatic 1-D Auto-CNN and 3-D Auto-CNN are used as spectral and spectral-spatial HSI classifiers, respectively. Furthermore, the cutout is introduced as a regularization technique for the HSI spectral-spatial classification to further improve the classification accuracy. The experiments on four widely used hyperspectral data sets (i.e., Salinas, Pavia University, Kennedy Space Center, and Indiana Pines) show that the automatically designed data-dependent CNNs obtain competitive classification accuracy compared with the state-of-the-art methods. In addition, the automatic design of the deep learning architecture opens a new window for future research, showing the huge potential of using neural architectures' optimization capabilities for the accurate HSI classification.

In this article, a novel hyperspectral image (HSI) classification method based on fusing multiple edge-preserving operations (EPOs) is proposed, which consists of the following steps. First, the edge-preserving features are obtained by performing different types of EPOs, i.e., local edge-preserving filtering and global edge-preserving smoothing on the dimension-reduced HSI. Then, with the assistance of a superpixel segmentation method, the edge-preserving features are further improved by considering the inter and intra spectral properties of superpixels. Finally, the spectral and edge-preserving features are fused to form one composite kernel, which is fed into the support vector machine (SVM) followed by a majority voting fusion scheme. Experimental results on three data sets demonstrate the superiority of the proposed method over several state-of-the-art classification approaches, especially when the training sample size is limited. Furthermore, 21 well-known methods, including mathematical morphology-based approaches, sparse representation models, and deep learning-based classifiers, are adopted to be compared with the proposed method on Houston data set with standard sets of training and test samples released during 2013 Data Fusion Contest, which also shows the effectiveness of the proposed method.

A convolutional neural network (CNN) has recently demonstrated its outstanding capability for the classification of hyperspectral images (HSIs). Typical CNN-based methods usually adopt image patches as inputs to the network. However, a fixed-size image patch in HSI with complex spatial contexts may contain multiple ground objects of different classes, which will deteriorate the classification performance of the CNN. In addition, traditional convolutional layers adopted in the CNN have a huge amount of parameters needed to be tuned, which will cause high computational cost. To address the above-mentioned issues, a novel squeeze multibias network (SMBN) is proposed for HSI classification. Specifically, the proposed SMBN first introduces the multibias module (MBM), which incorporates multibias into the rectified linear unit layers. The MBM can decouple the feature maps of input patches into multiple response maps (corresponding to different ground objects) and adaptively select the meaningful maps for classification. Furthermore, the proposed SMBN replaces the traditional convolutional layer with a squeeze convolution module, which can greatly reduce the number of parameters in the network, thus saving the running time, while still maintaining high classification accuracy. Experimental results on three real HSIs demonstrate the superiority of the proposed SMBN method over several state-of-the-art classification approaches.

Land-cover mapping over urban areas using Landsat imagery has attracted considerable attention in recent years as it can promptly and accurately reflect the biophysical composition status of the urban landscape and allow further applications such as urban planning and risk management. However, due to the large diversity across different urban landscapes, adequate training sample collection for urban area mapping is both challenging and time-consuming. In this paper, we propose a novel unsupervised sample collection method for mapping urban areas using Landsat imagery. Specifically, the idea is to select reliable, representative, and diverse training samples from the images in a two-stage and iterative manner, based on a set of spectral indices (vegetation, impervious surface, soil, water). To validate the effectiveness and robustness of the proposed method, a synthetic data set was designed and a series of Landsat images over 39 representative cities from different biomes across the world was employed. The effectiveness of the proposed algorithm was quantitatively validated by assessing the quality of the automatically collected samples and the accuracy of the mapping results. In terms of the mapping performance, the proposed automatic approach can achieve a comparable mapping accuracy to supervised classification with manually collected samples. On the basis of the freely accessed Landsat data, the proposed approach demonstrates a promising potential for automatic large-scale (i.e., global) mapping over urban areas.

Hyperspectral image (HSI) classification has become a hot topic in the field of remote sensing. In general, the complex characteristics of hyperspectral data make the accurate classification of such data challenging for traditional machine learning methods. In addition, hyperspectral imaging often deals with an inherently nonlinear relation between the captured spectral information and the corresponding materials. In recent years, deep learning has been recognized as a powerful feature-extraction tool to effectively address nonlinear problems and widely used in a number of image processing tasks. Motivated by those successful applications, deep learning has also been introduced to classify HSIs and demonstrated good performance. This survey paper presents a systematic review of deep learning-based HSI classification literatures and compares several strategies for this topic. Specifically, we first summarize the main challenges of HSI classification which cannot be effectively overcome by traditional machine learning methods, and also introduce the advantages of deep learning to handle these problems. Then, we build a framework that divides the corresponding works into spectral-feature networks, spatial-feature networks, and spectral-spatial-feature networks to systematically review the recent achievements in deep learning-based HSI classification. In addition, considering the fact that available training samples in the remote sensing field are usually very limited and training deep networks require a large number of samples, we include some strategies to improve classification performance, which can provide some guidelines for future studies on this topic. Finally, several representative deep learning-based classification methods are conducted on real HSIs in our experiments.

Detecting land cover change through very-high-resolution (VHR) remote sensing images is helpful in supporting urban sustainable development, natural disaster evaluation, and environmental assessment. However, the intraclass spectral variance in VHR remote sensing images is usually larger than that of median-low remote sensing images. Furthermore, the bitemporal images are usually acquired under different atmospheric conditions, sun height, soil moisture, and other factors. Consequently, in practical applications, many pseudo changes are presented in the detected map. In this paper, an adaptive histogram trend (AHT) similarity approach is promoted to quantitatively measure the magnitude between the corresponding pixels in bitemporal images in terms of change semantic. In the proposed approach, to reduce the phenological effect on the bitemporal images of land cover change detection (LCCD), we first define the quantitative description of AHT. Second, the change magnitudes between pairwise pixels are quantitatively measured by an improved bin-to-bin (B2B) distance between the corresponding AHTs. Then, the change magnitudes between two entire bitemporal images are measured AHT-by-AHT. Finally, binary threshold methods, such as the Otsu method or the double-window flexible pace search (DFPS) method, are used to divide the change magnitude image into binary change detection maps and obtain the final change detection map. The performance of the AHT-based LCCD approach is verified by four pairs of VHR remote-sensing images that correspond to two types of real land cover change cases. The detected results based on the four pairs of bitemporal VHR images outperformed the compared state-of-the-art LCCD methods.

The “noisy label” problem is one of the major challenges in hyperspectral image (HSI) classification. In order to address this problem, a spatial density peak (SDP) clustering-based method is proposed to detect mislabeled samples in the training set. Specifically, the proposed methods consist of the following steps: first, the correlation coefficients among the training samples in each class are estimated. In this step, instead of measuring the correlation coefficients by considering individual samples, all neighbor samples or K representative neighbor samples in a local window surrounding each training sample are considered. By this way, the spatial contextual information could be used, and two versions of the proposed method, i.e., measuring the correlation coefficients using all neighbor samples or K representative samples, are referred as SDP and K-SDP, respectively. Second, with the correlation coefficients calculated above, the local density of each training sample can be obtained by the DP clustering algorithm. Finally, those mislabeled samples which usually have lower local densities in each class are able to be identified by a defined decision function. The effectiveness of the proposed detection method is evaluated using a series of spectral and spectral-spatial classification methods on several real hyperspectral data sets.

In this paper, a novel multi-scale structure extraction based spectral-spatial hyperspectral image classification method is proposed, which consists of the following steps. First, the spectral dimension of the hyperspectral image is reduced by averaging adjacent spectral bands. Then, in order to extract the multi-scale significant structural features (MSFs) which are insensitive to image noise and texture, a relative total variation based structure extraction method is applied on the dimension reduced hyperspectral image. Finally, the MSFs are fused together with the kernel principal component analysis (KPCA), so as to obtain the kernel PCA fused multi-scale structural features (KPCA-MSFs) for classification. Experiments conducted on a real hyperspectral image demonstrate the outstanding performance of the proposed approach over several state-of-the-art spectral-spatial classifiers, especially when the image is corrupted by serious scene noise.

Although extreme learning machine (ELM) has successfully been applied to a number of pattern recognition problems, only with the original ELM it can hardly yield high accuracy for the classification of hyperspectral images (HSIs) due to two main drawbacks. The first is due to the randomly generated initial weights and bias, which cannot guarantee optimal output of ELM. The second is the lack of spatial information in the classifier as the conventional ELM only utilizes spectral information for classification of HSI. To tackle these two problems, a new framework for ELM-based spectral-spatial classification of HSI is proposed, where probabilistic modeling with sparse representation and weighted composite features (WCFs) is employed to derive the optimized output weights and extract spatial features. First, ELM is represented as a concave logarithmic-likelihood function under statistical modeling using the maximum a posteriori estimator. Second, sparse representation is applied to the Laplacian prior to efficiently determine a logarithmic posterior with a unique maximum in order to solve the ill-posed problem of ELM. The variable splitting and the augmented Lagrangian are subsequently used to further reduce the computation complexity of the proposed algorithm. Third, the spatial information is extracted using the WCFs to construct the spectral-spatial classification framework. In addition, the lower bound of the proposed method is derived by a rigorous mathematical proof. Experimental results on three publicly available HSI data sets demonstrate that the proposed methodology outperforms ELM and also a number of state-of-the-art approaches.

An extinction profile (EP) is an effective spatial-spectral feature extraction method for hyperspectral images (HSIs), which has recently drawn much attention. However, the existing methods utilize the EPs in a stacking way, which is hard to fully explore the information in EPs for HSI classification. In this paper, a novel fusion framework termed EPs-fusion (EPs-F) is proposed to exploit the information within and among EPs for HSI classification. In general, EPs-F includes the following two stages. In the first stage, by extracting the EPs from three independent components of an HSI, three complementary groups of EPs can be constructed. For each EP, an adaptive superpixel-based composite kernel strategy is proposed to explore the spatial information within an EP. The weights to create the composite kernel and the number of superpixels are automatically determined based on the spatial information of each EP. In the second stage, since the different EPs contain highly complementary information, a simple yet effective decision fusion method is further applied to obtain the final classification result. Experiments on three real HSI data sets verify the qualitative and quantitative superiority of the proposed EPs-F method over several state-of-the-art HSI classifiers.

Sparse representation (SR) has been successfully used in the classification of hyperspectral images (HSIs) by representing HSI pixels over a dictionary and yielding discriminative sparse coefficients. Most of SR-based classification methods construct the dictionary by directly using some labeled pixels as atoms. Such dictionary can lead to inefficient SR for large-sized HSIs, and may be incomplete when the number of labeled pixels is less than the number of spectral bands. This paper proposes a contextual online dictionary learning (DL) method for HSIs classification, which learns a dictionary over the whole image rather than few labeled pixels. The proposed method can effectively and efficiently improve the adaptive representation capability of different pixels with an online learning mechanism. Specifically, the contextual characteristics of the HSI are integrated with discriminative spectral information for online DL, i.e., pushing similar pixels in neighborhood to share similar sparse coefficients with respect to the well-learned dictionary. By this way, the obtained sparse coefficients are structured and discriminative. Finally, a traditional classifier, i.e., the linear support vector machine, is applied to the sparse coefficients, and the final classification results are obtained. Experimental results on real HSIs show the effectiveness of the proposed method.

The mapping of urban areas at regional to global scales is a crucial task due to its value for environmental monitoring, habitat and biodiversity conservation, and decision-making. In most current applications, two techniques (i.e., supervised classification and data fusion) are widely applied in large-scale urban mapping. However, the costly training sample collection, inadequate data-source descriptions, and diverse urban characteristics (e.g., shape, size, socioeconomic status, and physical environment) are challenging problems for the urban mapping approaches. In this context, aiming at effectively deriving accurate urban areas at a large scale, we propose a novel ensemble support vector machine (SVM) method which consists of three steps: 1) the automatic generation of training data to reduce labor costs; 2) the construction of an ensemble SVM model to effectively combine the multisource data (including remote sensing and socioeconomic data); and 3) an adaptive patch-based thresholding technique to tackle the diverse urban characteristics. The proposed method is employed to map urban areas of China in 2005 and 2010, and the resulting maps are compared with the existing urban maps for 287 prefecture-level cities. It is found that our results present a satisfactory superiority, especially in challenging small cities, with a significant improvement in median Kappa (0.174 for 2005 and 0.203 for 2010). When incorporating moderate-resolution imaging spectroradiometer multispectral data as an additional source, the Kappa coefficient can be further raised by 0.028 for 2010. In general, the proposed method shows great potential for accurately mapping urban areas at regional, continental, or even global scales in a cost-effective manner.

Image decolorization is known to be an effective way in transferring a color image into a gray one while well preserving the major information of all three bands. In this paper, a simple yet effective hyperspectral image visualization framework based on decolorization, named decolorization based hyperspectral visualization, is proposed, which enables us to fully exploit the benefits of decolorization technique. The proposed framework consists of the following two main steps. First, the hyperspectral image is partitioned into nine subsets of adjacent hyperspectral bands and the averaged band of each subset is calculated. Then, the dimension reduced image is further divided into three groups of adjacent bands, and the bands in each group are fused by using an image decolorization method. The main contribution of this paper is that the strong correlations in two different fields, i.e., image decolorization and hyperspectral image visualization, are first built. Experiments performed on several real hyperspectral data sets demonstrate that the proposed framework can obtain outstanding visualization performance in terms of both subjective and objective evaluations.

In this paper, a novel method is introduced to detect and correct mislabeled training samples for hyperspectral image classification. First, domain transform recursive filtering-based feature extraction is used to improve the separability of the training samples. Then, constrained energy minimization-based object detection is performed on the training set with each training sample serving as the object spectrum. Finally, the label of each training sample is verified or corrected based on the averaged detection probabilities of different classes. Experiments performed on real hyperspectral data sets demonstrate the effectiveness of the proposed method in improving classification performance with respect to the classifier trained with the original training set that contains a number of mislabeled samples.

The existence of shadows in very high resolution satellite images obstructs image interpretation and the following applications, such as target detection and recognition. Traditional shadow detection methods consider only the pixel-level properties, such as color and intensity of image pixels, and thus, may produce errors around object boundaries. To overcome this problem, a novel shadow detection algorithm based on extended random walker (ERW) is proposed by jointly integrating both shadow property and spatial correlations among adjacent pixels. First, a set of training samples is automatically generated via an improved Otsu-based thresholding method. Then, the support vector machine is applied to obtain an initial detection map, which categorizes all the pixels in the scene into shadow and nonshadow. Finally, the initial detection map is refined with the ERW model, which can simultaneously characterize the shadow property and spatial information in satellite images to further improve shadow detection accuracy. Experiments performed on five real remote sensing images demonstrate the superiority of the proposed method over several state-of-the-art methods in terms of detection accuracy.

A generative adversarial network (GAN) usually contains a generative network and a discriminative network in competition with each other. The GAN has shown its capability in a variety of applications. In this paper, the usefulness and effectiveness of GAN for classification of hyperspectral images (HSIs) are explored for the first time. In the proposed GAN, a convolutional neural network (CNN) is designed to discriminate the inputs and another CNN is used to generate so-called fake inputs. The aforementioned CNNs are trained together: the generative CNN tries to generate fake inputs that are as real as possible, and the discriminative CNN tries to classify the real and fake inputs. This kind of adversarial training improves the generalization capability of the discriminative CNN, which is really important when the training samples are limited. Specifically, we propose two schemes: 1) a well-designed 1D-GAN as a spectral classifier and 2) a robust 3D-GAN as a spectral-spatial classifier. Furthermore, the generated adversarial samples are used with real training samples to fine-tune the discriminative CNN, which improves the final classification performance. The proposed classifiers are carried out on three widely used hyperspectral data sets: Salinas, Indiana Pines, and Kennedy Space Center. The obtained results reveal that the proposed models provide competitive results compared to the state-of-the-art methods. In addition, the proposed GANs open new opportunities in the remote sensing community for the challenging task of HSI classification and also reveal the huge potential of GAN-based methods for the analysis of such complex and inherently nonlinear data.

Self-Dual Attribute Profiles (SDAPs) have proven to be an effective method for extracting spatial features able to improve scene classification of remote sensing images with very high spatial resolution. An SDAP is a multilevel decomposition of an image obtained with a sequence of transformations performed by attribute filters over the Tree of Shapes (ToS). One of the main issues with this technique is the identification of the filter thresholds generating a SDAP composed of features that should be relevant for the classification problem. This paper proposes a tree-based supervised feature extraction strategy, which is based on Fisher's linear discriminant analysis relying on the available class information. The exploitation of the ToS structure in the threshold selection procedure allows one to avoid any prior full image filtering, as in other related techniques. Furthermore, the ToS automates and optimizes the whole process by decreasing the computational time and overcoming the conventional selection procedure based on trial and error attempts. The proposed automatic spatial feature extraction technique has been tested in the classification of a very high resolution image proving its effectiveness with respect to a conventional selection strategy.

In this work, we develop a new framework to combine ensemble learning and composite kernel learning for hyperspectral image classification. We refer it as the multiple composite kernel learning, which is based on an iterative architecture. More specifically, in each iteration, we use the rotation-based ensemble to create rotation matrix, which is used to generate rotated features for both spectral and spatial information (e.g., extinction profiles). Then, the new spectral and spatial features are integrated into the composite kernels based on support vector machines classifier. Different rotation matrices will lead to obtaining various newly spectral and spatial characteristics, thereby they further increase the diversity and the classification performance. Experimental results on Indian Pines benchmark hyperspectral dataset demonstrate the excellent performance of the proposed method.

Sparse representation (SR) based hyperspectral image (HSI) classification is a rapidly evolving research topic. How to construct an optimized dictionary to better characterize spectral-spatial features of HSI is an important problem. In this paper, a novel spectral-spatial online dictionary learning (SSODL) method for HSI classification is proposed. The main idea is to learn a complete and discriminative dictionary by exploiting both spatial and spectral information all over the whole image. Rather than only using training samples for dictionary construction, the online dictionary learning (ODL) mechanism can effectively improve the adaptive representation capability of different pixels. Specifically, the contextual characteristics of HSI are integrated with discriminative spectral information for the ODL, i.e., pushing similar pixels in neighborhood to share similar sparse coefficients w.r.t. the well learnt dictionary. By this way, the yielding sparse coefficients are structured and discriminative. Finally, a traditional classifier, i.e., linear support vector mechine (SVM), is applied to the sparse coefficients and the final classification results are obtained. Experimental results on real HSIs show the effectiveness of the proposed method.

Extinction profile (EP) is an effective feature extraction method which can well preserve the geometrical characteristics of a hyperspectral image (HSI) and by extracting the EP from first three independent components (ICs) of an HSI, three correlated and complementary groups of EP features can be constructed. In this paper, an EPs fusion (EPs-F) strategy is proposed for HSI classification by exploring spatial-spectral information within and among three EP features. In general, the EPs-F method includes two stages. In the first stage, within each EP feature, a superpixel-based composite kernel strategy is proposed to adaptively fuse the spatial information of EP and the spectral feature of HSI. Then, the obtained adaptive composite kernel is used to create a classification map for each EP. In the second stage, decision fusion is further applied on different classification maps to create the final classification result. Experiments on two real HSIs verify the effectiveness of the proposed EPs-F algorithm.

Hyperspectral image classification, an astonishing tool to distinguish the land covers in remote sensed hyperspectral images, has been investigated by multiple disciplines such as geoscience, environmental science, mathematics, and computer vision. Following early machine learning (e.g., support vector machines and neural networks) and feature extraction theories (e.g., principal component analysis), hundreds of hyperspectral image classification algorithms have been proposed in order to further improve the classification accuracies. However, it is still unclear what are the real improvements of the newly proposed methods in this field or we are just fitting models to some specific data sets? To address this problem, this paper aims at discussing the major motivations and ideas in conducting a comprehensive benchmark analysis for hyperspectral image classification. The benchmark should not only allows researchers to compare their models with other algorithms but also helps identify the chief factors affecting the performance of their classification methods.

The empirical line (EL) calibration is commonly used for atmospheric correction of remotely sensed spectral images and recovery of surface reflectance. Current methods for EL calibration are applied to single image using two (or more) reference targets. Considering cases with large number of (partially overlapped) images, only few scenes will include reference targets. Moreover, applying the estimated calibration coefficients of one image to other images can cause wrong results. Accordingly, the use of EL calibration is impractical for these cases. In this paper, we present a novel method for a simultaneous calibration of multiple images, which is called multiple image constrained empirical line (MIcEL). We present a generalized EL model that provide constrained results and is adaptable for large number of images. Given a set of images, we use available reference targets and tie points between overlapping images to calibrate all the images in the set simultaneously. Tie points are automatically extracted using scale-invariant feature transform (SIFT) method. Accuracy assessment of the MIcEL was carried out using real hyperspectral images and field measurements. The performance of MIcEL was compared to the quick atmospheric correction (QUAC) method. the results show that (comparable with respect to QUAC) the absolute accuracy of the MIcEL, with respect to filed measurements, is ~ ± 11%.

In this paper, a novel iterative clustering based active learning (ICAL) method for hyperspectral image classification is proposed. On the one hand, the extreme learning machine is combined with the Markov random field (ELM-MRF) for label assignment, to exploit both spectral and spatial information to boost classification result. On the other hand, an iterative clustering based sample selection strategy is introduced to optimally choose the most informative training sample set. This strategy first selects a candidate set of samples, according to the differential map that is obtained by comparing the ELM-MRF based classification results in adjacent iterations. Then, all the pixels in the candidate set are clustered according to spectral characteristics. Finally, from each cluster, the one sample with the highest uncertainty is added to the new training sample set. By this sample selection strategy, the diversity and uncertainty of training samples can be maximized, which can further contribute to the improvement of classification performance. Experimental results show that the proposed ICAL method can achieve competitive classification results even with a limited number of labeled samples.

The optimal selection of chemical features (molecular descriptors) is an essential pre-processing step for the efficient application of computational intelligence techniques in virtual screening for identification of bioactive molecules in drug discovery . The selection of molecular descriptors has key influence in the accuracy of affinity prediction. In order to improve this prediction, we examined a Random Forest (RF)-based approach to automatically select molecular descriptors of training data for ligands of kinases, nuclear hormone receptors , and other enzymes . The reduction of features to use during prediction dramatically reduces the computing time over existing approaches and consequently permits the exploration of much larger sets of experimental data. To test the validity of the method, we compared the results of our approach with the ones obtained using manual feature selection in our previous study (Perez-Sanchez, Cano, and Garcia-Rodriguez, 2014).The main novelty of this work in the field of drug discovery is the use of RF in two different ways: feature ranking and dimensionality reduction, and classification using the automatically selected feature subset. Our RF-based method outperforms classification results provided by Support Vector Machine (SVM) and Neural Networks (NN) approaches.

Oil spill monitoring in optical remote sensing (RS) images is a challenging task due to the complexity of target discrimination in an oil spill scenario. Differently from traditional oil spill detection methods that are mainly carried out in a monotemporal image, in this letter, a novel solution is given in a multitemporal domain by investigating potential capability of change detection (CD) techniques, and it mainly contributes to an unsupervised, semiautomatic, and efficient approach. It opens a new perspective for solving an oil spill detection problem. In particular, a coarse-to-fine multitemporal change analysis procedure is designed to investigate the spectral–temporal variation of change targets present in the scenario. Changes relevant and irrelevant to suspected oil spills are identified and discriminated according to a binary and a multiple CD process, respectively. The proposed approach provides a quick yet effective oil spill detection solution, which is valuable and important in practical applications. The proposed method was validated on two real multitemporal RS data sets presenting the oil spill event in northern Gulf of Mexico in 2010. Experimental results confirmed its effectiveness.

With the rapid development of the internet and popularization of intelligent mobile devices, social media is evolving fast and contains rich spatial information, such as geolocated posts, tweets, photos, video, and audio. Those location-based social media data have offered new opportunities for hazards and disaster identification or tracking, recommendations for locations, friends or tags, pay-per-click advertising, etc. Meanwhile, a massive amount of remote sensing (RS) data can be easily acquired in both high temporal and spatial resolution with a multiple satellite system, if RS maps can be provided, to possibly enable the monitoring of our location-based living environments with some devices like charge-coupled device (CCD) cameras but on a much larger scale. To generate the classification maps, usually, labeled RS image pixels should be provided by RS experts to train a classification system. Traditionally, labeled samples are obtained according to ground surveys, image photo interpretation or a combination of the aforementioned strategies. All the strategies should be taken care of by domain experts, in a means which is costly, time consuming, and sometimes of a low quality due to reasons such as photo interpretation based on RS images only. These practices and constraints make it more challenging to classify land-cover RS images using big RS data. In this paper, a new methodology is proposed to classify urban RS images by exploiting the semantics of location-based social media photos (SMPs). To validate the effectiveness of this methodology, an automatic classification system is developed based on RS images as well as SMPs via big data analysis techniques including active learning, crowdsourcing, shallow machine learning, and deep learning. As the labels of RS training data are given by ordinary people with a crowdsourcing technique, the developed system is named Crowd4RS. The quantitative and qualitative experiments confirm the effectiveness of the proposed Crowd4RS system as well as the proposed methodology for automatically generating RS image maps in terms of classification results based on big RS data made up of multispectral RS images in a high spatial resolution and a large amount of photos from social media sites, such as Flickr and Panoramio.

The convergence of remote sensing technologies with social media, coupled with advances in other location-aware technologies such as WiFi and smartphones, is moving us on a fast track to a situation in which we can readily know, at any time, where everybody and everything are located on the surface of the Earth, and to exploit the power of social media in different contexts. Remote sensing technology involves the use of systems and algorithms to record information about the surface of the Earth from a remote location [1]. Although reliable as a data source, remote sensing data may not always be available. However, these data can be complemented by other sources of data, such as geographic information systems (GIS) and social media [2], in order to address time-critical applications. For instance, relating publicly available social media information with remote sensing or GIS data can lead to a more efficient management of emergency response (which refers to applications in which real/near-real-time response is needed, such as natural disasters). Social media are now playing a more relevant role in our daily lives and provide a unique opportunity to gain valuable insight on information flow and social networking within the society. As a result, the integration of social media data with other consolidated technologies such as remote sensing or GIS is of great importance.

The rapid development of social media data and the associated growth in volume, velocity, and variety has fostered the idea of using these data to guide traditional remote sensing image retrieval and information extraction tasks. Although important progress has been made in recent years in harvesting spatial and temporal data from social media, the exploitation of these data for decision making still needs further investigation, particularly in the context of its integration with remote sensing and geographic information systems. In this paper, we first discuss the relation between localization techniques and spatial technologies, pointing out their similarities and differences. Then, we provide a discussion on location analysis of social media data, and the fusion of multiple data sources, with specific attention to the integration of social media content (including localization) with remote sensing-based spatial technologies. Next, we provide specific examples addressing the use of social media data to perform information extraction from large remote sensing data repositories. Although significant possibilities for the integration of localization and spatial technologies can be seen in the examples provided, our survey suggests that the convergence of remote sensing and social media data will continue to deeply transform these technologies.

The significant development of social media over the past decade has been complemented by the rise of spatial technologies to provide new mapping mechanisms that allow users engage with online information services and also with each other in an unprecedented way. Users of these technologies now provide a comprehensive geosocial overlay of the physical environment of the planet.

Sparse representation (SR) can transform spectral signatures of hyperspectral pixels into sparse coefficients with very few nonzero entries, which can efficiently be used for compression. In this paper, a spectral-spatial adaptive SR (SSASR) method is proposed for hyperspectral image (HSI) compression by taking advantage of the spectral and spatial information of HSIs. First, we construct superpixels, i.e., homogeneous regions with adaptive sizes and shapes, to describe HSIs. Since homogeneous regions usually consist of similar pixels, pixels within each superpixel will be similar and share similar spectral signatures. Then, the spectral signatures of each superpixel can be simultaneously coded in the SR model to exploit their joint sparsity. Since different superpixels generally have different performances of SR, their rate-distortion performances in the sparse coding will be different. To achieve the best possible overall rate-distortion performance, an adaptive coding scheme is introduced to adaptively assign distortions to superpixels. Finally, the obtained sparse coefficients are quantized and entropy coded and constitute the final bitstream with the coded superpixel map. The experimental results over several HSIs show that the proposed SSASR method outperforms some state-of-the-art HSI compression methods in terms of the rate-distortion and spectral fidelity performances.

With the rapid development of spectral imaging techniques, classification of hyperspectral images (HSIs) has attracted great attention in various applications such as land survey and resource monitoring in the field of remote sensing. A key challenge in HSI classification is how to explore effective approaches to fully use the spatial-spectral information provided by the data cube. Multiple kernel learning (MKL) has been successfully applied to HSI classification due to its capacity to handle heterogeneous fusion of both spectral and spatial features. This approach can generate an adaptive kernel as an optimally weighted sum of a few fixed kernels to model a nonlinear data structure. In this way, the difficulty of kernel selection and the limitation of a fixed kernel can be alleviated. Various MKL algorithms have been developed in recent years, such as the general MKL, the subspace MKL, the nonlinear MKL, the sparse MKL, and the ensemble MKL. The goal of this paper is to provide a systematic review of MKL methods, which have been applied to HSI classification. We also analyze and evaluate different MKL algorithms and their respective characteristics in different cases of HSI classification cases. Finally, we discuss the future direction and trends of research in this area.

Edge-preserving features (EPFs) obtained by the application of edge-preserving filters to hyperspectral images (HSIs) have been found very effective in characterizing significant spectral and spatial structures of objects in a scene. However, a direct use of the EPFs can be insufficient to provide a complete characterization of spatial information when objects of different scales are present in the considered images. Furthermore, the edge-preserving smoothing operation unavoidably decreases the spectral differences among objects of different classes, which may affect the following classification. To overcome these problems, in this paper, a novel principal component analysis (PCA)-based EPFs (PCA-EPFs) method for HSI classification is proposed, which consists of the following steps. First, the standard EPFs are constructed by applying edge-preserving filters with different parameter settings to the considered image, and the resulting EPFs are stacked together. Next, the spectral dimension of the stacked EPFs is reduced with the PCA, which not only can represent the EPFs in the mean square sense but also highlight the separability of pixels in the EPFs. Finally, the resulting PCA-EPFs are classified by a support vector machine (SVM) classifier. Experiments performed on several real hyperspectral data sets show the effectiveness of the proposed PCA-EPFs, which sharply improves the accuracy of the SVM classifier with respect to the standard edge-preserving filtering-based feature extraction method, and other widely used spectral-spatial classifiers.

A novel method for anomaly detection in hyperspectral images is proposed. The method is based on two ideas. First, compared with the surrounding background, objects with anomalies usually appear with small areas and distinct spectral signatures. Second, for both the background and the objects with anomalies, pixels in the same class are usually highly correlated in the spatial domain. In this paper, the pixels with specific area property and distinct spectral signatures are first detected with attribute filtering and a Boolean map-based fusion approach in order to obtain an initial pixel-wise detection result. Then, the initial detection result is refined with edge-preserving filtering to make full use of the spatial correlations among adjacent pixels. Compared with other widely used anomaly detection methods, the experimental results obtained on real hyperspectral data sets including airport, beach, and urban scenes demonstrate that the performance of the proposed method is quite competitive in terms of computing time and detection accuracy.

We present, in this paper, a new methodology for spectral unmixing, where a vector of fractions, corresponding to a set of endmembers (EMs), is estimated for each pixel in the image. The process first provides an initial estimate of the fraction vector, followed by an iterative procedure that converges to an optimal solution. Specifically, projected gradient descent (PGD) optimization is applied to (a variant of) the spectral angle mapper objective function, so as to significantly reduce the estimation error due to amplitude (i.e., magnitude) variations in EM spectra, caused by the illumination change effect. To improve the computational efficiency of our method over a commonly used gradient descent technique, we have analytically derived the objective function's gradient and the optimal step size (used in each iteration). To gain further improvement, we have implemented our unmixing module via code vectorization, where the entire process is “folded” into a single loop, and the fractions for all of the pixels are solved simultaneously. We call this new parallel scheme vectorized code PGD unmixing (VPGDU). VPGDU has the advantage of solving (simultaneously) an independent optimization problem per image pixel, exactly as other pixelwise algorithms, but significantly faster. Its performance was compared with the commonly used fully constrained least squares unmixing (FCLSU), the generalized bilinear model (GBM) method for hyperspectral unmixng, and the fast state-of-the-art methods, sparse unmixing by variable splitting and augmented Lagrangian (SUnSAL) and collaborative SUnSAL (CLSUnSAL) based on the alternating direction method of multipliers. Considering all of the prospective EMs of a scene at each pixel (i.e., without a priori knowledge which/how many EMs are actually present in a given pixel), we demonstrate that the accuracy due to VPGDU is considerably higher than that obtained by FCLSU, GBM, SUnSAL, and CLSUnSAL under varying illumination, and is, otherwise, comparable with respect to these methods. However, while our method is significantly faster than FCLSU and GBM, it is slower than SUnSAL and CLSUnSAL by roughly an order of magnitude.

Supervised classification of hyperspectral images (HSI) is a very challenging task due to the existence of noisy and mixed spectral characteristics. Recently, the widely developed spectral unmixing techniques offer the possibility to extract spectral mixture information at a subpixel level, which can contribute to the categorization of seriously mixed spectral pixels. Besides, it has been demonstrated that the discrimination between different materials will be improved by integrating the geometry and structure information, which can be derived from the variance between neighboring pixels. Furthermore, by incorporating the spatial context, the superpixel-based spectral-spatial similarity information can be used to smooth classification results in homogeneous regions. Therefore, a novel fusion framework for HSI classification that combines subpixel, pixel, and superpixel-based complementary information is proposed in this paper. Here, both feature fusion and decision fusion schemes are introduced. For the feature fusion scheme, the first step is to extract subpixel-level, pixel-level, and superpixel-level features from HSI, respectively. Then, the multiple feature-induced kernels are fused to form one composite kernel, which is incorporated with a support vector machine (SVM) classifier for label assignment. For the decision fusion scheme, class probabilities based on three different features are estimated by the probabilistic SVM classifier first. Then, the class probabilities are adaptively fused to form a probabilistic decision rule for classification. Experimental results tested on different real HSI images can demonstrate the effectiveness of the proposed fusion schemes in improving discrimination capability, when compared with the classification results relied on each individual feature.

Hyperspectral imaging (HSI) classification has become a popular research topic in recent years, and effective feature extraction is an important step before the classification task. Traditionally, spectral feature extraction techniques are applied to the HSI data cube directly. This paper presents a novel algorithm for HSI feature extraction by exploiting the curvelet-transformed domain via a relatively new spectral feature processing technique—singular spectrum analysis (SSA). Although the wavelet transform has been widely applied for HSI data analysis, the curvelet transform is employed in this paper since it is able to separate image geometric details and background noise effectively. Using the support vector machine classifier, experimental results have shown that features extracted by SSA on curvelet coefficients have better performance in terms of classification accuracy over features extracted on wavelet coefficients. Since the proposed approach mainly relies on SSA for feature extraction on the spectral dimension, it actually belongs to the spectral feature extraction category. Therefore, the proposed method has also been compared with some state-of-the-art spectral feature extraction techniques to show its efficacy. In addition, it has been proven that the proposed method is able to remove the undesirable artifacts introduced during the data acquisition process. By adding an extra spatial postprocessing step to the classified map achieved using the proposed approach, we have shown that the classification performance is comparable with several recent spectral–spatial classification methods.

Active learning (AL) and semisupervised learning (SSL) are both promising solutions to hyperspectral image classification. Given a few initial labeled samples, this work combines AL and SSL in a novel manner, aiming to obtain more manually labeled and pseudolabeled samples and use them together with the initial labeled samples to improve the classification performance. First, based on a comparison of the segmentation and spectral-spatial classification results obtained by random walker (RW) and extended RW (ERW) algorithms, the unlabeled samples are separated into two different sets, i.e., low- and high-confidence unlabeled data sets. For the high-confidence unlabeled data, pseudolabeling is performed, which can ensure the correctness and informativeness of the pseudolabeled samples. For the low-confidence unlabeled data, AL is used to select samples. In this way, the samples which are more effective for improvement of classification performance can be labeled in only a few iterations. Finally, with the learned training set and the original hyperspectral image as inputs, the ERW classifier is used to obtain the final classification result. Experiments performed on three real hyperspectral data sets show that the proposed method can achieve competitive classification accuracy even with a very limited number of manually labeled samples.

This paper proposes a spectral-spatial classification algorithm based on principal components (PCs)-based smooth ordering and multiple 1-D interpolation, which can alleviate the general classification problems effectively. Because of the characteristics of hyperspectral image, there always exist easily separable samples (ESSs) and difficultly separable samples (DSSs) in view of the different sets of labeled samples. In this paper, the PC analysis is first used for reducing features and extracting the few first PCs of a hyperspectral image. Then, PC-based smooth ordering is designed for the separation of ESSs and DSSs, and multiple 1-D interpolation is used for the accurate classification of the ESSs. Next, the highly confident samples are selected from the ESSs by the spatial neighborhood information, which are added into the training set for the classification of DSSs. In the case of sufficient training samples, a supervised spectral-spatial method is used for classifying the DSSs by combining the spatial information built with popular extended multiattribute profiles. The proposed algorithm is compared with some state-of-the-art methods on three hyperspectral data sets. The results demonstrate that the presented algorithm achieves much better classification performance in terms of the accuracy and the computation time.

Morphological attribute profiles are multilevel decompositions of images obtained with a sequence of transformations performed by connected operators. They have been extensively employed in performing multi-scale and region-based analysis in a large number of applications. One main, still unresolved, issue is the selection of filter parameters able to provide representative and non-redundant threshold decomposition of the image. This paper presents a framework for the automatic selection of filter thresholds based on Granulometric Characteristic Functions (GCFs). GCFs describe the way that non-linear morphological filters simplify a scene according to a given measure. Since attribute filters rely on a hierarchical representation of an image (e.g., the Tree of Shapes) for their implementation, GCFs can be efficiently computed by taking advantage of the tree representation. Eventually, the study of the GCFs allows the identification of a meaningful set of thresholds. Therefore, a trial and error approach is not necessary for the threshold selection, automating the process and in turn decreasing the computational time. It is shown that the redundant information is reduced within the resulting profiles (a problem of high occurrence, as regards manual selection). The proposed approach is tested on two real remote sensing data sets, and the classification results are compared with strategies present in the literature.

Satellite image classification is a key task used in remote sensing for the automatic interpretation of a large amount of information. Today there exist many types of classification algorithms using advanced image processing methods enhancing the classification accuracy rate. One of the best state-of-the-art methods which improves significantly the classification of complex scenes relies on Self-Dual Attribute Profiles (SDAPs). In this approach, the underlying representation of an image is the Tree of Shapes, which encodes the inclusion of connected components of the image. The SDAP computes for each pixel a vector of attributes providing a local multiscale representation of the information and hence leading to a fine description of the local structures of the image. Instead of performing a pixel-wise classification on features extracted from the Tree of Shapes, it is proposed to directly classify its nodes. Extending a specific interactive segmentation algorithm enables it to deal with the multi-class classification problem. The method does not involve any statistical learning and it is based entirely on morphological information related to the tree. Consequently, a very simple and effective region-based classifier relying on basic attributes is presented.

A novel unsupervised change detection approach for multi-spectral remote sensing data based on morphological transformation is presented. Profiles obtained by attribute filters can provide a rich multi-level analysis of the contextual information. The proposed method is based on the assumption that pixels belonging to changed areas exhibit profiles with significant differences due to a variation in their geometry, whereas pixels within unchanged areas result in similar profiles due to their similar spatial characteristics. The extension to the multi-spectral scenario is performed by applying the morphological analysis on the available bands that compose a given data set. In such scenario radiometric normalization results mandatory in order to minimize the effect due to different acquisition's conditions. To this purpose, IR-MAD is performed as pre-processing. In the paper, preliminary results obtained considering a multi-temporal Landsat ETM+ data set acquired over an agriculture area are shown.

This paper presents a novel approach named extinction profiles to model the spatial information of remote sensing images. Then, the output of the extinction profile is fed to a grid-search random forest classification method. Results indicate that the proposed approach can effectively extract spatial information from remote sensing gray scale images and provide high classification accuracies in an automatic way.

Active learning usually is conducted in an iterative way. In the paper, a Computational Efficiency Active Learning (CEAL) algorithm is proposed to address this problem based on diversity measurement for classification of hyperspectral images. In particular, each unlabeled sample is pre-assigned a group label, which can be carried out by such as a clustering algorithm. After that, candidate patterns are selected from each group to satisfy the diversity assumption in each round. The proposed CEAL algorithm is validated by real hyperspectral images. Experimental results show that the proposed CEAL algorithm can obtain not only high classification accuracies but also yield a two to four order of magnitude increase in computational efficiency.

This letter proposes a new approach for the spectral-spatial classification of hyperspectral images, which is based on a novel extrema-oriented connected filtering technique, entitled as extended extinction profiles. The proposed approach progressively simplifies the first informative features extracted from hyperspectral data considering different attributes. Then, the classification approach is applied on two well-known hyperspectral data sets, i.e., Pavia University and Indian Pines, and compared with one of the most powerful filtering approaches in the literature, i.e., extended attribute profiles. Results indicate that the proposed approach is able to efficiently extract spatial information for the classification of hyperspectral images automatically and swiftly. In addition, an array-based node-oriented max-tree representation was carried out to efficiently implement the proposed approach.

Among the multiple advantages and applications of remote sensing, one of the most important uses is to solve the problem of crop classification, i.e., differentiating between various crop types. Satellite images are a reliable source for investigating the temporal changes in crop cultivated areas. In this letter, we propose a novel bat algorithm (BA)-based clustering approach for solving crop type classification problems using a multispectral satellite image. The proposed partitional clustering algorithm is used to extract information in the form of optimal cluster centers from training samples. The extracted cluster centers are then validated on test samples. A real-time multispectral satellite image and one benchmark data set from the University of California, Irvine (UCI) repository are used to demonstrate the robustness of the proposed algorithm. The performance of the BA is compared with two other nature-inspired metaheuristic techniques, namely, genetic algorithm and particle swarm optimization. The performance is also compared with the existing hybrid approach such as the BA with K-means. From the results obtained, it can be concluded that the BA can be successfully applied to solve crop type classification problems.

In this letter, we propose a new version of the rotation forest (RoF) method for the pixelwise classification of hyperspectral images. RoF, which is an ensemble of decision tree classifiers, uses random feature selection and data transformation techniques (i.e., principal component analysis) to improve both the accuracy of base classifiers and the diversity within the ensemble. Traditional RoF performs data transformation on the training samples of each subset. In order to further improve the performance of RoF, the data transformation is separately performed on each class, extracting sets of transformation matrices that are strictly dependent on the training samples of each single class. The approach, namely, class-separation-based RoF (RoFCS), is experimentally investigated on a hyperspectral image collected by Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) sensor. Experimental results demonstrate that the proposed methodology achieves excellent performances, in comparison with random forest and RoF classifiers.

Every day a large number of Earth observation (EO) spaceborne and airborne sensors from many different countries provide a massive amount of remotely sensed data. Those data are used for different applications, such as natural hazard monitoring, global climate change, urban planning, etc. The applications are data driven and mostly interdisciplinary. Based on this it can truly be stated that we are now living in the age of big remote sensing data. Furthermore, these data are becoming an economic asset and a new important resource in many applications. In this paper, we specifically analyze the challenges and opportunities that big data bring in the context of remote sensing applications. Our focus is to analyze what exactly does big data mean in remote sensing applications and how can big data provide added value in this context. Furthermore, this paper describes the most challenging issues in managing, processing, and efficient exploitation of big data for remote sensing problems. In order to illustrate the aforementioned aspects, two case studies discussing the use of big data in remote sensing are demonstrated. In the first test case, big data are used to automatically detect marine oil spills using a large archive of remote sensing data. In the second test case, content-based information retrieval is performed using high-performance computing (HPC) to extract information from a large database of remote sensing images, collected after the terrorist attack to the World Trade Center in New York City. Both cases are used to illustrate the significant challenges and opportunities brought by the use of big data in remote sensing applications.

The papers in this special issue focus on the topic of Big Data. This is a second in a series of papers on the topic of Big Data. This first part in this series (IEEE Proceedings, January 2016) consist of papers that are devoted to the theoretical aspects of big data. This second in a two-part series focuses on the practical applications of Big Data.

Remotely sensed images with very high spatial resolution provide a detailed representation of the surveyed scene with a geometrical resolution that, at the present, can be up to 30 cm (WorldView-3). A set of powerful image processing operators have been defined in the mathematical morphology framework. Among those, connected operators [e.g., attribute filters (AFs)] have proven their effectiveness in processing very high resolution images. AFs are based on attributes which can be efficiently implemented on tree-based image representations. In this paper, we considered the definition of min, max, direct, and subtractive filter rules for the computation of AFs over the tree-of-shapes representation. We study their performance on the classification of remotely sensed images. We compare the classification results over the tree of shapes with the results obtained when the same rules are applied on the component trees. The random forest is used as a baseline classifier, and the experiments are conducted using multispectral data sets acquired by QuickBird and IKONOS sensors over urban areas.

With respect to recent advances in remote sensing technologies, the spatial resolution of airborne and spaceborne sensors is getting finer, which enables us to precisely analyze even small objects on the Earth. This fact has made the research area of developing efficient approaches to extract spatial and contextual information highly active. Among the existing approaches, morphological profile and attribute profile (AP) have gained great attention due to their ability to classify remote sensing data. This paper proposes a novel approach that makes it possible to precisely extract spatial and contextual information from remote sensing images. The proposed approach is based on extinction filters, which are used here for the first time in the remote sensing community. Then, the approach is carried out on two well-known high-resolution panchromatic data sets captured over Rome, Italy, and Reykjavik, Iceland. In order to prove the capabilities of the proposed approach, the obtained results are compared with the results from one of the strongest approaches in the literature, i.e., APs, using different points of view such as classification accuracies, simplification rate, and complexity analysis. Results indicate that the proposed approach can significantly outperform its alternative in terms of classification accuracies. In addition, based on our implementation, profiles can be generated in a very short processing time. It should be noted that the proposed approach is fully automatic.

In this paper, we propose a novel multiple kernel learning (MKL) framework to incorporate both spectral and spatial features for hyperspectral image classification, which is called multiple-structure-element nonlinear MKL (MultiSE-NMKL). In the proposed framework, multiple structure elements (MultiSEs) are employed to generate extended morphological profiles (EMPs) to present spatial-spectral information. In order to better mine interscale and interstructure similarity among EMPs, a nonlinear MKL (NMKL) is introduced to learn an optimal combined kernel from the predefined linear base kernels. We integrate this NMKL with support vector machines (SVMs) and reduce the min-max problem to a simple minimization problem. The optimal weight for each kernel matrix is then solved by a projection-based gradient descent algorithm. The advantages of using nonlinear combination of base kernels and multiSE-based EMP are that similarity information generated from the nonlinear interaction of different kernels is fully exploited, and the discriminability of the classes of interest is deeply enhanced. Experiments are conducted on three real hyperspectral data sets. The experimental results show that the proposed method achieves better performance for hyperspectral image classification, compared with several state-of-the-art algorithms. The MultiSE EMPs can provide much higher classification accuracy than using a single-SE EMP.

In recent years, the support vector machines (SVMs) have been very successful in remote sensing image classification, particularly when dealing with high-dimensional data and limited training samples. Nevertheless, the vector-based feature alignment of the SVM can lead to an information loss in representation of hyperspectral images, which intrinsically have a tensor-based data structure. In this paper, a new multiclass support tensor machine (STM) is specifically developed for hyperspectral image classification. Our newly proposed STM processes the hyperspectral image as a data cube and then identifies the information classes in tensor space. The multiclass STM is developed from a set of binary STM classifiers using the one-against-one parallel strategy. As a part of our tensor-based processing chain, a multilinear principal component analysis (MPCA) is used for preprocessing, in order to reduce the tensorial data redundancy and, at the same time, preserve the tensorial structure information in sparse and high-order subspaces. As a result, the contributions of this work are twofold: a new multiclass STM model for hyperspectral image classification is developed, and a tensorial image interpretation framework is constructed, which provides a system consisting of tensor-based feature representation, feature extraction, and classification. Experiments with four hyperspectral data sets, covering agricultural and urban areas, are conducted to validate the effectiveness of the proposed framework. Our experimental results show that the proposed STM and MPCA-STM can achieve better results than traditional SVM-based classifiers.

A novel hyperspectral image (HSI) classification method by the probabilistic fusion of pixel-level and superpixel-level classifiers is proposed. Generally, pixel-level classifiers based on spectral information only may generate “salt and pepper” result in the classification map since spatial correlation is not considered. By incorporating spatial information in homogeneous regions, the superpixel-level classifiers can effectively eliminate the noisy appearance. However, the classification accuracy will be deteriorated if undersegmentation cannot be fully avoided in superpixel-based approaches. Therefore, it is proposed to adaptively combine both the pixel-level and superpixel-level classifiers, to improve the classification performance in both homogenous and structural areas. In the proposed method, a support vector machine classifier is first applied to estimate the pixel-level class probabilities. Then, superpixel-level class probabilities are estimated based on a joint sparse representation. Finally, the two levels of class probabilities are adaptively combined in a maximum a posteriori estimation model, and the classification map is obtained by solving the maximum optimization problem. Experimental results on real HSI images demonstrate the superiority of the proposed method over several well-known classification approaches in terms of classification accuracy.

In recent years, many studies on hyperspectral image classification have shown that using multiple features can effectively improve the classification accuracy. As a very powerful means of learning, multiple kernel learning (MKL) can conveniently be embedded in a variety of characteristics. This paper proposes a class-specific sparse MKL (CS-SMKL) framework to improve the capability of hyperspectral image classification. In terms of the features, extended multiattribute profiles are adopted because it can effectively represent the spatial and spectral information of hyperspectral images. CS-SMKL classifies the hyperspectral images, simultaneously learns class-specific significant features, and selects class-specific weights. Using an $L_{1}$-norm constraint (i.e., group lasso) as the regularizer, we can enforce the sparsity at the group/feature level and automatically learn a compact feature set for the classification of any two classes. More precisely, our CS-SMKL determines the associated weights of optimal base kernels for any two classes and results in improved classification performances. The advantage of the proposed method is that only the features useful for the classification of any two classes can be retained, which leads to greatly enhanced discriminability. Experiments are conducted on three hyperspectral data sets. The experimental results show that the proposed method achieves better performances for hyperspectral image classification compared with several state-of-the-art algorithms, and the results confirm the capability of the method in selecting the useful features.

A novel set-to-set distance-based spectral-spatial classification method for hyperspectral images (HSIs) is proposed. In HSIs, the spatially connected and spectrally similar pixels within each homogeneous region can be considered as one set of test samples, i.e., a test set, which should belong to the same class. In addition, each class of labeled pixels can be regarded as one set of training samples, i.e., a training set. Therefore, it is a natural consideration in the proposed method to measure the similarity between test and training sets via specific set-based distance criteria and then decide the classification label for each test set, accordingly. Specifically, the superpixel algorithm-based oversegmentation technique jointly exploits both the spatial similarity and structural information to first divide the HSI into multiple perceptually uniform regions. As a result, each segmented region corresponds to one test set. Then, each test/training set is represented with an affine hull (AH) model, which utilizes both the similarity and variance of pixels within each set to adaptively characterize the set. Finally, the class label for each test set is determined based on the closest geometry distance between test and training AHs. Experimental results on real HSI data sets demonstrate the superiority of the proposed algorithm over several well-known classification approaches, in terms of classification accuracy and computational speed.

In this paper, a novel spectral–spatial adaptive sparse representation (SSASR) method is proposed for hyperspectral image (HSI) denoising. The proposed SSASR method aims at improving noise-free estimation for noisy HSI by making full use of highly correlated spectral information and highly similar spatial information via sparse representation, which consists of the following three steps. First, according to spectral correlation across bands, the HSI is partitioned into several nonoverlapping band subsets. Each band subset contains multiple continuous bands with highly similar spectral characteristics. Then, within each band subset, shape-adaptive local regions consisting of spatially similar pixels are searched in spatial domain. This way, spectral–spatial similar pixels can be grouped. Finally, the highly correlated and similar spectral–spatial information in each group is effectively used via the joint sparse coding, in order to generate better noise-free estimation. The proposed SSASR method is evaluated by different objective metrics in both real and simulated experiments. The numerical and visual comparison results demonstrate the effectiveness and superiority of the proposed method.

Pansharpening is the process of fusing a high-resolution panchromatic image and a low-spatial-resolution multispectral image to yield a high-spatial-resolution multispectral image. This is a typical ill-posed inverse problem, and in the past two decades, many methods have been proposed to solve it. Still, there is no general consensus on the best way to quantitatively evaluate the spectral and spatial quality of the fused image. In this paper, we compare the two most widely used and accepted methods for quality evaluation. The first method is the verification of the synthesis property which states that the fused image should be as identical as possible to the multispectral image that the sensor would observe at a higher resolution. This is impossible to verify unless the observed images are spatially degraded so that the original observed multispectral image can be used as reference. The second method is to use metrics that do not use a reference, such as the quality no reference (QNR) metrics. However, there is another property, i.e., the consistency property, which states that the fused image reduced to the resolution of the original multispectral image should be as identical to the original image as possible. This has generally been considered a necessary condition that does not have to imply correct fusion. Using real WorldView-2 and QuickBird data and a total of 18 component substitution and multiresolution analysis methods, we demonstrate that the consistency property can indeed be used to give reliable assessment of the relative performance of pansharpening methods and is superior to using the QNR metrics.

The new generation of Earth observation sensors with high spatial resolution can provide detailed information for change detection. The widely used methods for high-resolution image change detection rely on textural/structural features. However, these spatial features always produce high-dimensional data space since they are related to a series of parameters, e.g., window sizes and directions. Machine learning methods are also commonly employed, but their performances are subject to the quantity and quality of the training samples, and hence, much effort should be made to collect the high-quality samples. To address these problems, in this study, a novel multiindex automatic change detection method is proposed for the high-resolution imagery. The notable advantages of the proposed model include the following: 1) Complicated urban scenes are represented by a set of low dimensional but semantic information indexes, replacing the high-dimensional but low-level features (e.g., textural and structural features), and 2) the change detection model is carried out automatically without using training samples since the information indexes can directly indicate the primitive urban classes. The multiindex representation refers to the enhanced vegetation index, the water index, and the recently developed morphological building index. Experiments were conducted on the multitemporal WorldView-2 images over Shenzhen City (south of China) and Kuala Lumpur (the capital of Malaysia), where promising results were achieved by the proposed method. Moreover, the traditional methods based on the state-of-the-art textural/morphological features were also implemented for the purpose of comparison, which further validates the advantages of our proposed model.

Attribute profiles (APs) have increasingly been receiving more attention over the last years, as they are able to extract and model spatial information that is useful for the analysis of remote sensing images of very high spatial resolution (VHR). However, one of the major issues in employing APs is the choice of a proper range of thresholds, able to provide a representative and non-redundant multi-level image decomposition. This paper presents a novel method for the automatic selection of adequate thresholds to compute the AP. A new concept of cumulative function, which can be seen as an extension of the basic notion of granulometry, is introduced. In particular, different information on the spatial context is achieved according to the measure used for computing the cumulative function, which is computed on the AP composed by considering all possible values of the attribute. The proposed approach aims at selecting the set of thresholds that provides the best approximation of the resulting cumulative function based on the chosen measure. Experimental analysis carried out on a very high resolution image shows the effectiveness of the presented strategy in providing a set of thresholds able to retain the salient spatial structures in the scene.

Big Data Analytics methods take advantage of techniques from the fields of data mining, machine learning, or statistics with a focus on analysing large quantities of data (aka `big datasets') with modern technologies. Big data sets appear in remote sensing in the sense of large volumes, but also in the sense of an ever increasing amount of spectral bands (i.e., high-dimensional data). The remote sensing has traditionally used the above described techniques for a wide variety of application such as classification (e.g., land cover analysis using different spectral bands from satellite data), but more recently scalability challenges occur when using traditional (often serial) methods. This paper addresses observed scalability limits when using support vector machines (SVMs) for classification and discusses scalable and parallel developments used in concrete application areas of remote sensing. Different approaches that are based on massively parallel methods are discussed as well as recent developments in parallel methods.

With respect to the exponential increase in the number of available remote sensors in recent years, the possibility of having different types of data captured over the same scene, has resulted in many research works related to the joint use of passive and active sensors for the accurate classification of different materials. However, until now, there is a small number of research works related to the integration of highly valuable information obtained from the joint use of LiDAR and hyperspectral data. This paper proposes an efficient classification approach in terms of accuracies and demanded CPU processing time for integrating big data sets (e.g., LiDAR and hyperspectral) to provide land cover mapping capabilities at a range of spatial scales. In addition, the proposed approach is fully automatic and is able to efficiently handle big data containing a huge number of features with very limited number of training samples in few seconds.

This is a selection of results of the North State project, that demonstrate how innovative methods applied to the new Sentinel data streams can be combined with models to monitor carbon and water fluxes for pan-boreal Europe.

The 2014 data fusion contest organized by IGARSS 2014 has shown an example that dual images with quite different spatial resolutions may have quite different ground coverages. Specifically, due to the technology limit of optical sensor and the incomplete flight path, the high spatial resolution visible image used in the data fusion contest can only provide a sparse ground coverage. To fill the missing area in this visible image, a simple yet effective image completion method is introduced in this paper which consists of the following two steps: First, through performing patch matching on the hyperspectral image, the most common recurring offsets between patches in the known region and those in the missing region are estimated. Second, the pixels in the missing region is completed by fusing pixels of the shifted visible images (obtained using the above estimated offsets). Experimental results show that the proposed method performs much better than multiple general image completion methods for this data set.

An interactive color visualization method is proposed for hyperspectral imagery (HSI). The method visualizes complex information through different fusion results of multiple images in a color space which is under the interactive control of the observers. In order to solve the main problem of traditional visualization methods, i.e., they can at most display information from three bands in one image, this paper proposes an easy, vivid and effective method for color visualization. In the proposed approach, observers interactively control a cursor position to change the output fusion images and their fusion coefficients. In the approach, the dynamic display will include more than three bands of HSIs. The proposed method is also applicable for visualization of other multi-images, e.g., multispectral images, output images of direction filters, multi-focus images, and multi-temporal images, etc.

In the past two decades, many methods have been proposed to fuse low resolution multispectral (MS) and high resolution panchromatic (Pan) images, i.e., pansharpening. Two large families of such methods are component substitution (CS) and multiresolution analysis methods (MRA). We develop a model based method for pansharpening based on minimizing a cost function which includes a data fidelity term, a detail injection term and a total variation (TV) term. The model takes into account the modulation transfer function (MTF) and spectral response of the sensor. The resulting iterative method not only sharpens the MS image with details from the Pan image but is also able to extract important information from the MS image itself via MTF-based deconvolution. We compare the proposed method to a number of state-of-the-art CS and MRA pansharpening methods using a real WorldView-2 dataset and show that it gives excellent results with details that all the CS and MRA methods can not extract.

The fusion of low resolution multispectral (MS) images and high resolution panchromatic (PAN) images, i.e., pansharpening, is an important technique in remote sensing and has many applications where high resolution imagery is important. Component substitution (CS) and multiresolution analysis (MRA) are two large families of pansharpening methods that are fast and computationally efficient. They can be described using a general framework, where details from the PAN image are added to the upsampled and interpolated MS image. However, these methods often suffer from spectral and spatial distortions. We propose a pre-processing step, where instead of just interpolating the MS image to the resolution scale of the PAN image, we do a deconvolution of the interpolated MS image based on the sensor's modulation transfer function (MTF). This results in large improvement gains in the spectral and spatial quality of the fused image. We demonstrate our method using a real WorldView-2 dataset and show that our approach significantly improves the tested methods in both the CS and MRA families of pansharpening methods.

In this letter, we explore the use of self-dual attribute profiles (SDAPs) for the classification of hyperspectral images. The hyperspectral data are reduced into a set of components by nonparametric weighted feature extraction (NWFE), and a morphological processing is then performed by the SDAPs separately on each of the extracted components. Since the spatial information extracted by SDAPs results in a high number of features, the NWFE is applied a second time in order to extract a fixed number of features, which are finally classified. The experiments are carried out on two hyperspectral images, and the support vector machines and random forest are used as classifiers. The effectiveness of SDAPs is assessed by comparing its results against those obtained by an approach based on extended APs.

A new feature selection approach that is based on the integration of a genetic algorithm and particle swarm optimization is proposed. The overall accuracy of a support vector machine classifier on validation samples is used as a fitness value. The new approach is carried out on the well-known Indian Pines hyperspectral data set. Results confirm that the new approach is able to automatically select the most informative features in terms of classification accuracy within an acceptable CPU processing time without requiring the number of desired features to be set a priori by users. Furthermore, the usefulness of the proposed method is also tested for road detection. Results confirm that the proposed method is capable of discriminating between road and background pixels and performs better than the other approaches used for comparison in terms of performance metrics.

Remote sensing is one of the most common ways to extract relevant information about Earth and our environment. Remote sensing acquisitions can be done by both active (synthetic aperture radar, LiDAR) and passive (optical and thermal range, multispectral and hyperspectral) devices. According to the sensor, a variety of information about the Earth's surface can be obtained. The data acquired by these sensors can provide information about the structure (optical, synthetic aperture radar), elevation (LiDAR), and material content (multispectral and hyperspectral) of the objects in the image. Once considered together their complementarity can be helpful for characterizing land use (urban analysis, precision agriculture), damage detection (e.g., in natural disasters such as floods, hurricanes, earthquakes, oil spills in seas), and give insights to potential exploitation of resources (oil fields, minerals). In addition, repeated acquisitions of a scene at different times allows one to monitor natural resources and environmental variables (vegetation phenology, snow cover), anthropological effects (urban sprawl, deforestation), climate changes (desertification, coastal erosion), among others. In this paper, we sketch the current opportunities and challenges related to the exploitation of multimodal data for Earth observation. This is done by leveraging the outcomes of the data fusion contests, organized by the IEEE Geoscience and Remote Sensing Society since 2006. We will report on the outcomes of these contests, presenting the multimodal sets of data made available to the community each year, the targeted applications, and an analysis of the submitted methods and results: How was multimodality considered and integrated in the processing chain? What were the improvements/new opportunities offered by the fusion? What were the objectives to be addressed and the reported solutions? And from this, what will be the next challenges?

The availability of hyperspectral images with improved spectral and spatial resolutions provides the opportunity to obtain accurate land-cover classification. In this paper, a novel methodology that combines spectral and spatial information for supervised hyperspectral image classification is proposed. A feature reduction strategy based on independent component analysis is the main core of the spectral analysis, where the exploitation of prior information coupled to the evaluation of the reconstruction error assures the identification of the best class-informative subset of independent components. Reduced attribute profiles (APs), which are designed to address well-known issues related to information redundancy that affect the common morphological APs, are then employed for the modeling and fusion of the contextual information. Four real hyperspectral data sets, which are characterized by different spectral and spatial resolutions with a variety of scene typologies (urban, agriculture areas), have been used for assessing the accuracy and generalization capabilities of the proposed methodology. The obtained results demonstrate the classification effectiveness of the proposed approach in all different scene typologies, with respect to other state-of-the-art techniques.

For the classification of hyperspectral images (HSIs), this paper presents a novel framework to effectively utilize the spectral-spatial information of superpixels via multiple kernels, which is termed as superpixel-based classification via multiple kernels (SC-MK). In the HSI, each superpixel can be regarded as a shape-adaptive region, which consists of a number of spatial neighboring pixels with very similar spectral characteristics. First, the proposed SC-MK method adopts an oversegmentation algorithm to cluster the HSI into many superpixels. Then, three kernels are separately employed for the utilization of the spectral information, as well as spatial information, within and among superpixels. Finally, the three kernels are combined together and incorporated into a support vector machine classifier. Experimental results on three widely used real HSIs indicate that the proposed SC-MK approach outperforms several well-known classification methods.

A novel superpixel-based discriminative sparse model (SBDSM) for spectral-spatial classification of hyperspectral images (HSIs) is proposed. Here, a superpixel in a HSI is considered as a small spatial region whose size and shape can be adaptively adjusted for different spatial structures. In the proposed approach, the SBDSM first clusters the HSI into many superpixels using an efficient oversegmentation method. Then, pixels within each superpixel are jointly represented by a set of common atoms from a dictionary via a joint sparse regularization. The recovered sparse coefficients are utilized to determine the class label of the superpixel. In addition, instead of directly using a large number of sampled pixels as dictionary atoms, the SBDSM applies a discriminative K-SVD learning algorithm to simultaneously train a compact representation dictionary, as well as a discriminative classifier. Furthermore, by utilizing the class label information of training pixels and dictionary atoms, a class-labeled orthogonal matching pursuit is proposed to accelerate the K-SVD algorithm while still enforcing high discriminability on sparse coefficients when training the classifier. Experimental results on four real HSI datasets demonstrate the superiority of the proposed SBDSM algorithm over several well-known classification approaches in terms of both classification accuracies and computational speed.

A novel feature selection approach is proposed to address the curse of dimensionality and reduce the redundancy of hyperspectral data. The proposed approach is based on a new binary optimization method inspired by fractional-order Darwinian particle swarm optimization (FODPSO). The overall accuracy (OA) of a support vector machine (SVM) classifier on validation samples is used as fitness values in order to evaluate the informativity of different groups of bands. In order to show the capability of the proposed method, two different applications are considered. In the first application, the proposed feature selection approach is directly carried out on the input hyperspectral data. The most informative bands selected from this step are classified by the SVM. In the second application, the main shortcoming of using attribute profiles (APs) for spectral-spatial classification is addressed. In this case, a stacked vector of the input data and an AP with all widely used attributes are created. Then, the proposed feature selection approach automatically chooses the most informative features from the stacked vector. Experimental results successfully confirm that the proposed feature selection technique works better in terms of classification accuracies and CPU processing time than other studied methods without requiring the number of desired features to be set a priori by users.

Just over a decade has passed since the concept of morphological profile was defined for the analysis of remote sensing images. Since then, the morphological profile has largely proved to be a powerful tool able to model spatial information (e.g., contextual relations) of the image. However, due to the shortcomings of using the morphological profiles, many variants, extensions, and refinements of its definition have appeared stating that the morphological profile is still under continuous development. In this case, recently introduced theoretically sound attribute profiles (APs) can be considered as a generalization of the morphological profile, which is a powerful tool to model spatial information existing in the scene. Although the concept of the AP has been introduced in remote sensing only recently, an extensive literature on its use in different applications and on different types of data has appeared. To that end, the great amount of contributions in the literature that address the application of the AP to many tasks (e.g., classification, object detection, segmentation, change detection, etc.) and to different types of images (e.g., panchromatic, multispectral, and hyperspectral) proves how the AP is an effective and modern tool. The main objective of this survey paper is to recall the concept of the APs along with all its modifications and generalizations with special emphasis on remote sensing image classification and summarize the important aspects of its efficient utilization while also listing potential future works.

A novel multiple-kernel learning (MKL) model is proposed for urban classification to integrate heterogeneous features (HF-MKL) from two data sources, i.e., spectral images and LiDAR data. The features include spectral, spatial, and elevation attributes of urban objects from the two data sources. With these heterogeneous features (HFs), the new MKL model is designed to carry out feature fusion that is embedded in classification. First, Gaussian kernels with different bandwidths are used to measure the similarity of samples on each feature at different scales. Then, these multiscale kernels with different features are integrated using a linear combination. In the combination, the weights of the kernels with different features are determined by finding a projection based on the maximum variance. This way, the discriminative ability of the HFs is exploited at different scales and is also integrated to generate an optimal combined kernel. Finally, the optimization of the conventional support vector machine with this kernel is performed to construct a more effective classifier. Experiments are conducted on two real data sets, and the experimental results show that the HF-MKL model achieves the best performance in terms of classification accuracies in integrating the HFs for classification when compared with several state-of-the-art algorithms.

In this paper, a novel feature extraction method based on intrinsic image decomposition (IID) is proposed for hyperspectral image classification. The proposed method consists of the following steps. First, the spectral dimension of the hyperspectral image is reduced with averaging-based image fusion. Then, the dimension reduced image is partitioned into several subsets of adjacent bands. Next, the reflectance and shading components of each subset are estimated with an optimization-based IID technique. Finally, pixel-wise classification is performed only on the reflectance components, which reflect the material-dependent properties of different objects. Experimental results show that, with the proposed feature extraction method, the support vector machine classifier is able to obtain much higher classification accuracy even when the number of training samples is quite small. This demonstrates that IID is indeed an effective way for feature extraction of hyperspectral images.

This paper introduces a novel spectral-spatial classification method for hyperspectral images based on extended random walkers (ERWs), which consists of two main steps. First, a widely used pixelwise classifier, i.e., the support vector machine (SVM), is adopted to obtain classification probability maps for a hyperspectral image, which reflect the probabilities that each hyperspectral pixel belongs to different classes. Then, the obtained pixelwise probability maps are optimized with the ERW algorithm that encodes the spatial information of the hyperspectral image in a weighted graph. Specifically, the class of a test pixel is determined based on three factors, i.e., the pixelwise statistics information learned by a SVM classifier, the spatial correlation among adjacent pixels modeled by the weights of graph edges, and the connectedness between the training and test samples modeled by random walkers. Since the three factors are all well considered in the ERW-based global optimization framework, the proposed method shows very good classification performances for three widely used real hyperspectral data sets even when the number of training samples is relatively small.

Abstract—Hyperspectral image classification has been an active topic of research in recent years. In the past, many different types of features have been extracted (using both linear and nonlinear strategies) for classification problems. On the one hand, some approaches have exploited the original spectral information or other features linearly derived from such information in order to have classes which are linearly separable. On the other hand, other techniques have exploited features obtained through nonlinear transformations intended to reduce data dimensionality, to better model the inherent nonlinearity of the original data (e.g., kernels) or to adequately exploit the spatial information contained in the scene (e.g., using morphological analysis). Special attention has been given to techniques able to exploit a single kind of features, such as composite kernel learning or multiple kernel learning, developed in order to deal with multiple kernels. However, few

In remote sensing, due to cost and complexity issues, multispectral (MS) and hyperspectral (HS) sensors have significantly lower spatial resolution than panchromatic (PAN) images. Recently, the problem of fusing coregistered MS and HS images has gained some attention. In this paper, we propose a novel method for fusion of MS/HS and PAN images and of MS and HS images. MS and, more so, HS images contain spectral redundancy, which makes the dimensionality reduction of the data via principal component (PC) analysis very effective. The fusion is performed in the lower dimensional PC subspace; thus, we only need to estimate the first few PCs, instead of every spectral reflectance band, and without compromising the spectral and spatial quality. The benefits of the approach are substantially lower computational requirements and very high tolerance to noise in the observed data. Examples are presented using WorldView 2 data and a simulated data set based on a real HS image, with and without added noise.

In this paper we compare features obtained by different filtering strategies for morphological attribute filters by considering non-increasing attributes. The Attribute profiles (APs) and Self Dual Attribute Profiles (SDAPs) are obtained by sequentially applying attribute filters on tree-based image representations, such as Min- or Max-trees and Inclusion tree, respectively. This work aims to study the effects of using the filtering rules max, min, direct and subtractive, when considering the non-increasing attributes moment of inertia and standard deviation. A very high spatial resolution data set is used in the experiments, and the extracted information obtained by the profiles is analyzed. This is done by studying the effects on the classification accuracy by using the profiles as additional input features to a Random Forest classifier.

The big data analytics approach emerged that can be interpreted as extracting information from large quantities of scientific data in a systematic way. In order to have a more concrete understanding of this term we refer to its refinement as smart data analytics in order to examine large quantities of scientific data to uncover hidden patterns, unknown correlations, or to extract information in cases where there is no exact formula (e.g. known physical laws). Our concrete big data problem is the classification of classes of land cover types in image-based datasets that have been created using remote sensing technologies, because the resolution can be high (i.e. large volumes) and there are various types such as panchromatic or different used bands like red, green, blue, and nearly infrared (i.e. large variety). We investigate various smart data analytics methods that take advantage of machine learning algorithms (i.e. support vector machines) and state-of-the-art parallelization approaches in order to overcome limitations of big data processing using non-scalable serial approaches.

This article proposes a feature reduction technique for hyperspec-tral images using Independent Component Analysis (ICA). The proposed technique aims at extracting the best subset of class-informative independent components (ICs) for hyperspectral supervised classification. The selection of the most representative components is assured by the minimization of the reconstruction error, which is computed on the training samples used for the supervised classification. The searching strategy is optimized by exploiting a genetic algorithm-based approach where the fitness function is the classification accuracy obtained by using a support vector machine (SVM) classifier. The obtained results show the effectiveness of the proposed approach in providing class-informative components to improve the classification accuracy.

This paper proposes a new spectral-spatial hyperspectral classification method named the shape-adaptive sparse representation (SASR). The fixed window is not suitable for all pixels of hyperspectral image (HSI) to search local similar regions. In order to overcome the drawback, we propose to apply the shape-adaptive algorithm to exploit the contextual spatial information of HSI. Furthermore, the hyperspectral classification is implemented by incorporating the spatial contextual information of HSI into the sparse representation classification model. Experimental results demonstrate the superiority of the proposed SASR method over both classical and state-of-the-art approaches.

In this paper, the fusion of hyperspectral and Li-DAR data is taken into account in order to develop a new classification framework for the accurate analysis of urban areas. In this method, an attribute profile is considered in order to model the spatial information of LiDAR and hyper-spectral data. In parallel, in order to reduce the redundancy of the hyperspectral data and address the so-called curse of dimensionality, a supervised feature extraction technique is used. Then, the new features obtained by the attribute profile and the supervised feature extraction technique are concatenated into a stacked vector. The final classification map is achieved by using a Random Forest classifier. Results infer that the proposed method can provide very good results in terms of classification accuracy and CPU processing time in an automatic manner.

A novel spectral-spatial hyperspectral image classification is proposed based on extended random walkers. First, a widely used pixel-wise classifier, i.e., the support vector machine (SVM), is adopted to obtain probability maps for a hyper-psectral image, which measure the probabilities that a pixel belongs to different classes. Then, the initial probabilities are optimized with the extended random walkers. Finally, by assigning each pixel with the label for which the greatest probability is obtained, the classification result is obtained. Experiments show the outstanding performance of the proposed method in terms of classification accuracy especially when the number of training samples is relatively small.

This paper presents a new multiple feature learning approach for accurate spectral-spatial classification of hyperspec-tral images. The proposed method integrates multiple features based on the logarithmic opinion pool. We consider subspace multinomial logistic regression for classification as it exhibits a flexible structure for the combination of multiple features through the posterior probability. At the same time, it is able to cope with highly mixed hyperspectral data and with the presence of limited training samples. In this work, we considered lowpass filtering and morphological attribute profiles for spatial feature extraction. Our experimental results with a real hyperspectral images collected by the NASA Jet Propulsion Laboratory's Airborne Visible Infra-Red Imaging Spectrometer (AVIRIS) indicate that the proposed method exhibits state-of-the-art classification performance.

Due to cost and complexity issues, hyperspectral (HS) images have lower spatial resolution than multispectral (MS) and panchromatic (PAN) images. We present a novel method for fusing both MS and PAN images and also HS and MS images, based on their statistical properties in the wavelet domain. HS images contain spectral redundancy that makes the dimensionality reduction of the data via principal component analysis (PCA) very effective. The fusion is performed in the lower dimensional PC subspace so we only need to estimate the first few PCs, instead of every spectral reflectance band, and without compromising the spectral and spatial quality. The benefits of the approach are substantially lower computational requirements and a very high tolerance to noise in the observed data. Examples are presented using World View 2 data and also a simulated dataset based on a real HS image, with and without noise.

Over the last few years, several new strategies have been proposed for spectral-spatial classification of remotely sensed image data, for cases when high spatial and spectral resolutions are available. In this letter, we focus on the possibility of performing advanced spectral-spatial classification of remote sensing images with limited spectral resolution (often called multispectral). A new strategy is proposed, where the spectral dimensionality of the multispectral data is first expanded by using nonlinear feature extraction with kernel methods such as kernel principal component analysis. Then, extended multiattribute profiles (EMAPs), built on the expanded set of spectral features, are used to include spatial information. This strategy allows us to first decompose different spectral clusters into different spectral features and further improve the spatial discrimination. The resulting EMAPs are used for classification using advanced classifiers such as support vector machines and random forests. We test our proposed methodology with different multispectral data sets obtaining state-of-the-art classification results.

A new spectral–spatial method for classification of hyperspectral images is introduced. The proposed approach is based on two segmentation methods, fractional-order Darwinian particle swarm optimization and mean shift segmentation. The output of these two methods is classified by support vector machines. Experimental results indicate that the integration of the two segmentation methods can overcome the drawbacks of each other and increase the overall accuracy in classification.

Sparse representation has been demonstrated to be a powerful tool in classification of hyperspectral images (HSIs). The spatial context of an HSI can be exploited by first defining a local region for each test pixel and then jointly representing pixels within each region by a set of common training atoms (samples). However, the selection of the optimal region scale (size) for different HSIs with different types of structures is a nontrivial task. In this paper, considering that regions of different scales incorporate the complementary yet correlated information for classification, a multiscale adaptive sparse representation (MASR) model is proposed. The MASR effectively exploits spatial information at multiple scales via an adaptive sparse strategy. The adaptive sparse strategy not only restricts pixels from different scales to be represented by training atoms from a particular class but also allows the selected atoms for these pixels to be varied, thus providing an improved representation. Experiments on several real HSI data sets demonstrate the qualitative and quantitative superiority of the proposed MASR algorithm when compared to several well-known classifiers.

A robust framework for the classification of hyperspectral images which takes into account both spectral and spatial information is proposed. The extended multivariate attribute profile (EMAP) is used for extracting spatial information. Moreover, for solving the so-called curse of dimensionality, supervised feature extraction is carried out on both the original hyperspectral data and the output of the EMAP. After performing the dimensionality reduction, two output vectors of the original data and attributes are concatenated into one stacked vector. The final classification map is achieved by using a random-forest classifier. The main difficulties of using an EMAP is to initialize the attribute parameters. Therefore, a fully automatic scheme of the proposed method is introduced to overcome the shortcomings of using EMAP. The proposed method is tested on two widely known data sets. Experimental results confirm that the proposed method provides an accurate classification map in an acceptable CPU processing time.

Hyperspectral remote sensing technology allows one to acquire a sequence of possibly hundreds of contiguous spectral images from ultraviolet to infrared. Conventional spectral classifiers treat hyperspectral images as a list of spectral measurements and do not consider spatial dependences, which leads to a dramatic decrease in classification accuracies. In this paper, a new automatic framework for the classification of hyperspectral images is proposed. The new method is based on combining hidden Markov random field segmentation with support vector machine (SVM) classifier. In order to preserve edges in the final classification map, a gradient step is taken into account. Experiments confirm that the new spectral and spatial classification approach is able to improve results significantly in terms of classification accuracies compared to the standard SVM method and also outperforms other studied methods.

Hyperspectral remote sensing images contain hundreds of data channels. Due to the high dimensionality of the hyperspectral data, it is difficult to design accurate and efficient image segmentation algorithms for such imagery. In this paper, a new multilevel thresholding method is introduced for the segmentation of hyperspectral and multispectral images. The new method is based on fractional-order Darwinian particle swarm optimization (FODPSO) which exploits the many swarms of test solutions that may exist at any time. In addition, the concept of fractional derivative is used to control the convergence rate of particles. In this paper, the so-called Otsu problem is solved for each channel of the multispectral and hyperspectral data. Therefore, the problem of n-level thresholding is reduced to an optimization problem in order to search for the thresholds that maximize the between-class variance. Experimental results are favorable for the FODPSO when compared to other bioinspired methods for multilevel segmentation of multispectral and hyperspectral images. The FODPSO presents a statistically significant improvement in terms of both CPU time and fitness value, i.e., the approach is able to find the optimal set of thresholds with a larger between-class variance in less computational time than the other approaches. In addition, a new classification approach based on support vector machine (SVM) and FODPSO is introduced in this paper. Results confirm that the new segmentation method is able to improve upon results obtained with the standard SVM in terms of classification accuracies.

The integration of spatial context in the classification of hyperspectral images is known to be an effective way in improving classification accuracy. In this paper, a novel spectral-spatial classification framework based on edge-preserving filtering is proposed. The proposed framework consists of the following three steps. First, the hyperspectral image is classified using a pixelwise classifier, e.g., the support vector machine classifier. Then, the resulting classification map is represented as multiple probability maps, and edge-preserving filtering is conducted on each probability map, with the first principal component or the first three principal components of the hyperspectral image serving as the gray or color guidance image. Finally, according to the filtered probability maps, the class of each pixel is selected based on the maximum probability. Experimental results demonstrate that the proposed edge-preserving filtering based classification method can improve the classification accuracy significantly in a very short time. Thus, it can be easily applied in real applications.

Feature extraction is known to be an effective way in both reducing computational complexity and increasing accuracy of hyperspectral image classification. In this paper, a simple yet quite powerful feature extraction method based on image fusion and recursive filtering (IFRF) is proposed. First, the hyperspectral image is partitioned into multiple subsets of adjacent hyperspectral bands. Then, the bands in each subset are fused together by averaging, which is one of the simplest image fusion methods. Finally, the fused bands are processed with transform domain recursive filtering to get the resulting features for classification. Experiments are performed on different hyperspectral images, with the support vector machines (SVMs) serving as the classifier. By using the proposed method, the accuracy of the SVM classifier can be improved significantly. Furthermore, compared with other hyperspectral classification methods, the proposed IFRF method shows outstanding performance in terms of classification accuracy and computational efficiency.

Pansharpening aims at creating a fused image of high spatial and spectral resolutions through merging a panchromatic (PAN) image with a multispectral (MS) image. Component substitution is the most widely used pansharpening method. However, most research in this field focuses on improving the existing component substitution-based pansharpening methods, e.g., principal component substitution and intensity hue saturation transform. The major contribution of this paper is a novel component substitution framework based on an image matting model. The matting model refers to an MS image that can be decomposed into three components, i.e., alpha channel, spectral foreground, and background. Through substituting the alpha channel of the MS image with the PAN image, the high-resolution MS image is able to be reconstructed perfectly. Experiments performed on different data sets demonstrate that the proposed method outperforms several state-of-the-art pansharpening methods in terms of subjective and objective evaluation.

In recent years, sparse representations have been widely studied in the context of remote sensing image analysis. In this paper, we propose to exploit sparse representations of morphological attribute profiles for remotely sensed image classification. Specifically, we use extended multiattribute profiles (EMAPs) to integrate the spatial and spectral information contained in the data. EMAPs provide a multilevel characterization of an image created by the sequential application of morphological attribute filters that can be used to model different kinds of structural information. Although the EMAPs' feature vectors may have high dimensionality, they lie in class-dependent low-dimensional subpaces or submanifolds. In this paper, we use the sparse representation classification framework to exploit this characteristic of the EMAPs. In short, by gathering representative samples of the low-dimensional class-dependent structures, any given sample may by sparsely represented, and thus classified, with respect to the gathered samples. Our experiments reveal that the proposed approach exploits the inherent low-dimensional structure of the EMAPs to provide state-of-the-art classification results for different multi/hyperspectral data sets.

In this work, a new framework for accurate classification of hyperspectral images is proposed. The new method is based on Hidden Markov Random Field and its Expectation Maximization (HMRF-EM) and Support Vector Machine (SVM) classifier. In order to preserve edges in final map, the Sobel edge detector is used. Result confirms that the combination of the spectral and spatial information can significantly improve results compared to the standard SVM method.

A new spectral-spatial method for the classification of hyperspectral images is introduced. The proposed approach is based on two segmentation methods, Fractional-Order Darwinian Particle Swarm Optimization and Mean Shift Segmentation and one clustering method, K-means. In parallel, the input data set is classified by Support Vector Machines (SVM). Furthermore, the result of the segmentation and clustering steps are combined with the result of SVM through majority voting within each object. The final classification map is made by using majority voting between three produced classification maps. Experimental results indicate that the proposed method can significantly improve SVM and other studied methods in terms of accuracies.

A novel matting model based pansharpening (MMP) method is proposed for creating a fused image of high spatial and spectral resolutions through merging a panchromatic (PAN) image with a multi-spectral (MS) image. Matting model refers to that an MS image can be decomposed into three components, i.e., alpha channel, spectral foreground and background. Through substituting the alpha channel of the MS image with the PAN image, the edge information from the PAN image can be transferred into the MS image without causing spectral distortion. Experiments demonstrate the superiority of the proposed method by subjective and objective evaluation.

In this paper we consider pansharpening of multispectral satellite imagery based on solving an under-determined inverse problem regularized by the ℓ1-norm of the coefficients of overcomplete multi-scale transforms which all are tight-frame systems. There are two main approaches in sparsity promoting ℓ1-norm regularization, the analysis and the synthesis approach. We perform a number of experiments using two real and well known datasets where the focus is the comparison of the two approaches. One dataset includes a high resolution reference image while the other needs to be degraded prior to pansharpening in order to use the original multispectral image as the reference. Experiments are performed for a range of values for the regularization parameter, where each resulting pansharpened image is evaluated using three quality metrics. The behavior of those metrics as a function of the regularization parameter is compared for the analysis and synthesis formulations and it is shown that analysis gives better results.

This paper deals with hyperspectral image reconstruction using a new linear model and Sparse Regularization (SR). The new model is based on Principal Components (PCs) and wavelets. Since the hyperspectral PCs are not spatially sparse, wavelet is applied to get spatially sparse representation. Sparse regularization is used to recover the corrupted signal. The regularization parameter is chosen by Stein's Unbiased Risk Estimator (SURE). The results show improvements for simulated data sets compare to other denoising methods based on Signal to Noise Ratio (SNR). In addition, the methods are applied on a real noisy data set, and the results of the new method demonstrate visual improvement. The proposed approach is automatic, fast and has the ability to be applied on very large data sets.

We propose CUR methods for hyperspectral unmixing that decompose the data matrix into non-negative endmembers and abundance maps. The endmembers will be selected from a dictionary constructed from the data matrix. Each endmember will coincide with certain columns of the data matrix. By doing this we are assured that the dictionary will be physically meaningful and may be interpreted unambiguously from the data set. This assumption, that the endmembers are contained within the data, is called the pixel purity assumption. We compare two regularization terms to promote sparsity in our solutions, the first is ℓ2regularization and the second is vector ℓ0regularization. The methods are evaluated both on simulated data and a real hyperspectral image of an urban landscape.

Hyperspectral unmixing is the task of decomposing hyperspectral images into endmembers and their abundances. The endmembers are spectral signatures of specific material in the image and the abundances dictate the amount of the material found in each pixel. In this paper we present a blind signal separation method, based on the total variation penalty, that simultaneously estimates the endmembers and the abundances. We evaluate our method using both simulated and a real data set.

A new approach to change detection in very high resolution remote sensing images based on morphological attribute profiles (APs) is presented. A multiresolution contextual transformation performed by APs allows the extraction of geometrical features related to the structures within the scene at different scales. The temporal changes are detected by comparing the geometrical features extracted from the image of each date. The experiments performed on panchromatic QuickBird images related to an urban area show the effectiveness of the proposed technique in detecting changes on the basis of the spatial morphology by preserving geometrical detail.

Extended attribute profiles, which are based on attribute filters, have recently been presented as efficient tools for spectral-spatial classification of remote sensing images. However, construction of these profiles usually requires manual selection of parameters for the corresponding attribute filters. In this letter, we present a technique to automatically build the extended attribute profiles with the standard deviation attribute based on the statistics of the samples belonging to the classes of interest. The methodology is tested on two widely used hyperspectral images and the results are found to be highly accurate.

The articles in special issue focus on advancements in very high resolution remote sensing technologies and applications.

Recent advances in spectral-spatial classification of hyperspectral images are presented in this paper. Several techniques are investigated for combining both spatial and spectral information. Spatial information is extracted at the object (set of pixels) level rather than at the conventional pixel level. Mathematical morphology is first used to derive the morphological profile of the image, which includes characteristics about the size, orientation, and contrast of the spatial structures present in the image. Then, the morphological neighborhood is defined and used to derive additional features for classification. Classification is performed with support vector machines (SVMs) using the available spectral information and the extracted spatial information. Spatial postprocessing is next investigated to build more homogeneous and spatially consistent thematic maps. To that end, three presegmentation techniques are applied to define regions that are used to regularize the preliminary pixel-wise thematic map. Finally, a multiple-classifier (MC) system is defined to produce relevant markers that are exploited to segment the hyperspectral image with the minimum spanning forest algorithm. Experimental results conducted on three real hyperspectral images with different spatial and spectral resolutions and corresponding to various contexts are presented. They highlight the importance of spectral-spatial strategies for the accurate classification of hyperspectral images and validate the proposed methods.

Markov models represent a wide and general family of stochastic models for the temporal and spatial dependence properties associated to 1-D and multidimensional random sequences or random fields. Their applications range over a wide variety of subareas of the information and communication technology (ICT) field, including networking, automation, speech processing, genomic-sequence analysis, or image processing. Focusing on the applicative problem of land-cover mapping from very-high-resolution (VHR) remote sensing images, which is a relevant problem in many applications of environmental monitoring and natural resource exploitation, Markov models convey a great potential, thanks to their capability to effectively describe and incorporate the spatial information associated with image data into an image-classification process. In this framework, the main ideas and previous work about Markov modeling for VHR image classification will be recalled in this paper and processing results obtained through recent methods proposed by the authors will be discussed.

Remotely sensed hyperspectral imaging allows for the detailed analysis of the surface of the Earth using advanced imaging instruments which can produce high-dimensional images with hundreds of spectral bands. Supervised hyperspectral image classification is a difficult task due to the unbalance between the high dimensionality of the data and the limited availability of labeled training samples in real analysis scenarios. While the collection of labeled samples is generally difficult, expensive, and time-consuming, unlabeled samples can be generated in a much easier way. This observation has fostered the idea of adopting semisupervised learning techniques in hyperspectral image classification. The main assumption of such techniques is that the new (unlabeled) training samples can be obtained from a (limited) set of available labeled samples without significant effort/cost. In this paper, we develop a new approach for semisupervised learning which adapts available active learning methods (in which a trained expert actively selects unlabeled samples) to a self-learning framework in which the machine learning algorithm itself selects the most useful and informative unlabeled samples for classification purposes. In this way, the labels of the selected pixels are estimated by the classifier itself, with the advantage that no extra cost is required for labeling the selected pixels using this machine–machine framework when compared with traditional machine–human active learning. The proposed approach is illustrated with two different classifiers: multinomial logistic regression and a probabilistic pixelwise support vector machine. Our experimental results with real hyperspectral images collected by the National Aeronautics and Space Administration Jet Propulsion Laboratory's Airborne Visible–Infrared Imaging Spectrometer and the Reflective Optics Spectrographic Imaging System indicate that the use of self-learning represents an effective and promising strategy in the context of hyperspectral image classification.

This paper presents a new framework for the development of generalized composite kernel machines for hyperspectral image classification. We construct a new family of generalized composite kernels which exhibit great flexibility when combining the spectral and the spatial information contained in the hyperspectral data, without any weight parameters. The classifier adopted in this work is the multinomial logistic regression, and the spatial information is modeled from extended multiattribute profiles. In order to illustrate the good performance of the proposed framework, support vector machines are also used for evaluation purposes. Our experimental results with real hyperspectral images collected by the National Aeronautics and Space Administration Jet Propulsion Laboratory's Airborne Visible/Infrared Imaging Spectrometer and the Reflective Optics Spectrographic Imaging System indicate that the proposed framework leads to state-of-the-art classification performance in complex analysis scenarios.

Morphological and attribute profiles have been proven to be effective tools to fuse spectral and spatial information for classification of remote sensing data. A wide range of filters (i.e., number of levels in the profiles) is usually necessary in order to properly model the spatial information in a remote sensing scene. A dense sampling of the values of the parameters of the filters generates profiles that have both a very large dimensionality (leading to the Hughes phenomenon in classification) and a high redundancy. In this paper, a novel iterative technique based on genetic algorithms (GAs) is proposed to automatically optimize the selection of the optimal features from the profiles. The selection of the filtered images that compose the profile is performed by dividing them into three classes corresponding to high, medium, and low importance. We propose to measure the importance (modeled in terms of discriminative power in the classification task) using a random forest classifier, which provides a rank for each feature with its model. Only the set of images associated with the highest importance is selected, i.e., preserved for classification. The proposed technique is applied to the features labeled with medium importance, whereas the images with the lowest importance are removed from the profile. This method is employed to classify three hyperspectral data sets achieving significantly high classification accuracy values. A parallel computing implementation has been developed in order to significantly reduce the time required for the run of the GAs.

Linear spectral unmixing is a challenging problem in hyperspectral imaging that consists of decomposing an observed pixel into a linear combination of pure spectra (or endmembers) with their corresponding proportions (or abundances). Endmember extraction algorithms can be employed for recovering the spectral signatures while abundances are estimated using an inversion step. Recent works have shown that exploiting spatial dependencies between image pixels can improve spectral unmixing. Markov random fields (MRF) are classically used to model these spatial correlations and partition the image into multiple classes with homogeneous abundances. This paper proposes to define the MRF sites using similarity regions. These regions are built using a self-complementary area filter that stems from the morphological theory. This kind of filter divides the original image into flat zones where the underlying pixels have the same spectral values. Once the MRF has been clearly established, a hierarchical Bayesian algorithm is proposed to estimate the abundances, the class labels, the noise variance, and the corresponding hyperparameters. A hybrid Gibbs sampler is constructed to generate samples according to the corresponding posterior distribution of the unknown parameters and hyperparameters. Simulations conducted on synthetic and real AVIRIS data demonstrate the good performance of the algorithm.

This paper presents a classification method for hyperspectral images utilizing Differential Morphological Profiles (DMPs) which permit to include in the analysis spatial information since they can provide an estimate of the size and contrast characteristics of the structures in an image. Due to the wide variety of objects present in a scene, the pixels belonging to the same semantic structure may not have homogeneous spatial and spectral features. In addition, instead of a single peak (which can be related to a measure of the scale), multiple local maxima and multiple responses are usually observed in the DMP. In order to handle such intra-class variability, class-specific weighting functions are employed in order to differently modulate the DMP values according to the different characteristics of the land cover types. In such way, it is possible to differentiate the behaviors of the DMP for each pixel in the image according to its semantic, providing an increase of the separability of the classes. At first, a DMP computed with opening by reconstruction (DMPO) and one with closing by reconstruction (DMPC) are derived on each of the first principle components extracted from the hyperspectral image. Then, both profiles are weighted by each class-specific weighting function and concatenated in a single data structure. The constructed feature vectors are considered by a random forest classifier.

A domain adaptation problem is dealt with where the marginal probability in a target domain is different from but correlated to the one in the source domain but the classification tasks are the same. This problem occurs frequently in classification of remote sensing data, e.g., when data are collected in the same area but at different dates or when data are acquired by the same sensor with the same class label set but in different locations. Traditional learning machines cannot deal with this problem in a satisfactory manner. In this paper, we propose a rationale input-output-consistency where samples in the same cluster and defined by spectral signatures (input space) should have the same class label (output space) if they are accurately classified. With the rationale, samples of high confidence in the target domain are selected to define a new prediction function. Since two domains that are related can have different distributions, the data in the source domain which cannot adapt to the distribution in the target domain are deleted from the training data set. Therefore, the proposed algorithm is denoted as input-consistent-output domain adaptation (iCODA) and works in an iterative way. After the selection of highly-confident target samples and the deletion of source data, a new training data set is used to define a new prediction model. The proposed iCODA algorithm was evaluated on EO-1 hyperspectral data sets from Botswana. Experimental results demonstrate much better classification accuracies when compared to a traditionally used supervised classifier.

In change detection analysis, the computation of the no-change distribution is affected when changed pixels are in large number in the scene. Because of this, the performance of several techniques are compromised. In this paper we compare two well known automatic change detection techniques (ITPCA and IRMAD) by performing an initial elimination of the strong changes in order to minimize the contribution of the changed pixels to the radiometric normalization computation. These two techniques are ineffective in correctly estimating the distribution of the no-change pixels when this kind of scenario is encountered. The strong changes are identified by building an initial change mask (ICM), which is based on the statistical analysis of the given data set. In this paper we show two simple algorithms for building the ICM. From the experiments on a data set characterized by a high amount of changes due to the agriculture activity, the improvement in quality of the map of changes obtained by the proposed approach with respect to the ones obtained without using the ICM has been observed.

The detection of hedges in very high spatial resolution remote sensing image is discussed in the paper. A spatial-spectral detector is proposed. The spatial information is modeled per pixel as the local orientation of the structure to which the pixel belongs. The local orientation is computed from the morphological directional profile built with a series of linear openings in several directions. These features are used as inputs to a support vector data description, a detection algorithm. Experimental results on a real satellite image show that the local orientation helps in discriminating hedges from other woody elements, which is not possible using the spectral information only.

Images obtained using Synthetic Aperture Radar (SAR) are corrupted by speckle. Speckle noise results from the chaotic interference of backscattered electromagnetic waves and makes the analysis, interpretation and classification of SAR images difficult. In this paper, we present a denoising algorithm based on Total Variation (TV) regularization. While this kind of denoising algorithm is not new, we propose to select the regularization parameter by minimizing the estimate of the mean square error (MSE) between the denoised image and the clean image. We do not have to know the clean image because we use a statistically unbiased MSE estimate - Stein's Unbiased Risk Estimate (SURE), that depends on the observed image and the estimated image. However, since it is difficult to derive SURE analytically for this kind of problem, we estimate SURE using stochastic methods. We present results using both a simulated image and real SAR image.

In this paper we present a new method for the pansharpening of multi-spectral satellite imagery. This method is based on a simple explicit image formation model which leads to an ill posed problem that needs to be regularized for best results. We use both Tikhonov (ridge regression) and Total Variation (TV) regularization. We develop the solutions to these two problems and then we address the problem of selecting the optimal regularization parameter λ. We find the value of λ that minimizes Stein's unbiased risk estimate (SURE). For ridge regression this leads to an analytical expression for SURE while for the TV regularized solution we use Monte Carlo SURE where the estimate is obtained by stochastic means. Finally, we present experiment results where we use quality metrics to evaluate the spectral and spatial quality of the resulting pansharpened image.

Dealing with a high number of features belonging to different types of data such as Hyperspectral image and Morphological Attribute Profiles (MAPs) might lead to a poor predictive performance of the classifier and hence low final accuracies of classification. This is due to the Hughes effect that consistently decreases the power of prediction of the classifier, in case of a limited and fixed number of training samples. In order to reduce the number of features and only keeping those which are more informative, a novel supervised feature selection technique based on GAs and the measure of the relevance of the features is presented in this work. Moreover, the effectiveness of the proposed technique was demonstrated by experimenting on an optical remote sensed dataset.

In this paper, we propose a denoising method for hyperspectral images using 3D wavelets. We use the sparse analysis regularization using a 3D overcomplete wavelet dictionary. The minimization problem is solved using iterative Chambolle algorithm. The simulation results show that the 3D dictionary outperforms the 2D one, in terms of Peak Signal to Noise Ratio (PSNR). Denosing hysperspectral cubes is likely to increase the classification accuracy of the hyperspectral data since it can enhance the spectral profiles (or features) that can be useful to discriminate between information classes.

Hyperspectral unmixing is the process where the reflectance spectrum from a mixed pixel is decomposed into separate distinct spectral signatures (endmembers). A mixed pixel results when spectra from more than one material is recorded by a sensor in one pixel. The goal of linear unmixing is to identify the number of endmembers in an image, the endmembers themselves and their abundances in each pixel. This paper presents a new smooth method for unmixing hyperspectral images using nonnegative cyclic descent. The proposed method uses iterative cyclic descent algorithm to find the endmembers and their abundances. The algorithm uses an ℒ1norm to promote sparseness in the abundances. Because the spectrum of the endmembers varies smoothly, a first order roughness penalty is added to discourage roughness in the endmembers. The algorithm does not use any prior information about the data. The method is tested using a real hyperspectral image of an urban landscape.

Morphological profiles (MPs) have been proposed in recent literature as aiding tools to achieve better results for classification of remotely sensed data. MPs are in general built using features containing most of the information content of the data, such as the components derived from principal component analysis (PCA). Recently, nonlinear PCA (NLPCA), performed by autoassociative neural network, has emerged as a good unsupervised technique to fit the information content of hyperspectral data into few components. The aim of this letter is to investigate the classification accuracies obtained using extended MPs built from the features of NPCA. A comparison of the two approaches has been validated on two different data sets having different spatial and spectral resolutions/coverages, over the same ground truth, and also using two different classification algorithms. The results show that NLPCA permits one to obtain better classification accuracies than using linear PCA.

With the launch of several planetary missions in the last decade, a large amount of planetary images has been already acquired and much more will be available for analysis in the coming years. The image data need to be analyzed, preferably by automatic processing techniques because of the huge amount of data. Although many automatic feature extraction methods have been proposed and utilized for earth remote sensing images, these methods are not always applicable to planetary data that often present low contrast and uneven illumination characteristics. Here, we propose a new unsupervised method for the extraction of different features of elliptical and geometrically compact shapes, such as craters and rocks of compact shape (e.g., boulders), to be used for image registration purposes. This approach is based on the combination of several image processing techniques, including watershed segmentation and the generalized Hough transform. The method potentially has application for extraction of craters, rocks, and other geological features.

Advanced information processing and architectures will be needed to bridge the gap between the potential offered by the new generations of sensors and the needs of the end-users to actually face tomorrow's challenges in many applications with a very high societal impact. As remote sensing researchers and engineers, this is our passion, our charge, and our responsibility.

In this paper, a new method for supervised hyperspectral data classification is proposed. In particular, the notion of stochastic minimum spanning forest (MSF) is introduced. For a given hyperspectral image, a pixelwise classification is first performed. From this classification map, M marker maps are generated by randomly selecting pixels and labeling them as markers for the construction of MSFs. The next step consists in building an MSF from each of the M marker maps. Finally, all the M realizations are aggregated with a maximum vote decision rule in order to build the final classification map. The proposed method is tested on three different data sets of hyperspectral airborne images with different resolutions and contexts. The influences of the number of markers and of the number of realizations M on the results are investigated in experiments. The performance of the proposed method is compared to several classification techniques (both pixelwise and spectral-spatial) using standard quantitative criteria and visual qualitative evaluation.

The Hierarchical SEGmentation (HSEG) algorithm, which is a combination of hierarchical step-wise optimization and spectral clustering, has given good performances for hyperspectral image analysis. This technique produces at its output a hierarchical set of image segmentations. The automated selection of a single segmentation level is often necessary. We propose and investigate the use of automatically selected markers for this purpose. In this paper, a novel Marker-based HSEG (M-HSEG) method for spectral-spatial classification of hyperspectral images is proposed. First, a map of markers is constructed using classification results. Then, a novel constrained M-HSEG algorithm is applied. The experimental results show that the proposed approach yields accurate segmentation and classification maps, and thus is attractive for hyperspectral image analysis.

A new method for supervised hyperspectral data classification is proposed. In particular, the notion of Stochastic Minimum Spanning Forests (MSFs) is introduced. For a given hyper-spectral image, a pixelwise classification is first performed. From this classification map, M marker maps are generated by randomly selecting pixels and labeling them as markers for the construction of MSFs. The next step consists in building an MSF from each of the M marker maps. Finally, all the M realizations are aggregated with a maximum vote decision rule, resulting in a final classification map. The experimental results presented on an AVIRIS image of the vegetation area show that the proposed approach yields accurate classification maps, and thus is attractive for hyperspectral data analysis.

On behalf of the IEEE Geoscience and Remote Sensing Society (IEEE GRSS), I warmly welcome you to IGARSS 2011 in Vancouver. Our annual symposium, IGARSS, is recognized today as a premier event in remote sensing and provides an ideal forum for obtaining up-to-date information about the latest developments, exchanging ideas, identifying future trends in your research area, and making contacts with the international remote sensing community.

Semi-supervised learning using both labeled and unlabeled data is usually adopted to design a high-accuracy and robust classification system on small-size remote sensing training data set. As suggested in the machine learning literature, the larger amount of unlabeled patterns are used, the better classification accuracies can be obtained. Nevertheless, most recently proposed semi-supervised algorithms are unable to handle a large amount of unlabeled samples. In the paper, we present a scalable semi-supervised learning algorithm by using whole hyperspectral remote sensing image. In particular, both spectral features and spatial information of a remote sensing image are adopted for the scalable semi-supervised learning. The accuracy and the reliability of the proposed algorithm have been evaluated on the ROSIS university hyperspectral remote sensing image. The accuracies are better or comparable when compared to the supervised state-of-the-art algorithms on both small-size and the original training sets.

A kernel adapted to the spectral dimension of hyperspectral images is proposed in this paper. A distance based on a statistical cluster model is used to construct a radial kernel. This class specific kernel realizes a compromise between a conventional Gaussian kernel and a Gaussian kernel on the first principal components of the considered class. An automatic gradient optimization is used to select the optimal hyperparameters. Experimental results on a real hyperspectral image show the kernel is effective compared to the conventional Gaussian kernel. Furthermoren the proposed kernel is less sensitive to one hyperparameter compared to the Gaussian kernel applied on the first principal components of the data.

Low spatial resolution is a major limitation for remote sensing classification, especially in a urban environment. In this work, we will focus on the simulation of urban area environment at a low spatial resolution, comparable to the new hyperspectral sensors that will be launched in the next few years. The aim is to better understand the possibility offered by the new sensors, in a challenging scenario like the one represented by a highly mixed image. Particular attention is placed on the characteristics of the sensor EnMap, produced by DLR. The experiments conducted on a real data set confirm the challenges posed by low spatial resolution when analyzing a urban environment.

A method for change detection between two hyperspectral datasets is presented. The iteratively reweighted multivariate alteration detection (IR-MAD) method is used for change detection. The strong changes are first eliminated based on the principal component analysis (PCA) of the difference image and IR-MAD is applied on the datasets after feature reduction with the PCA of the original bands. The method is demonstrated on a bitemporal hyperspectral dataset. The results show good correlation with ground truth.

In this paper a general approach based on morphological connected filters for the spatial simplification of very high resolution remote sensing images is introduced. In greater detail, the proposed approach is made up of two steps: i) the selection of the parameters defining the connected filters driven by the information available on the scene and on the specific application; and ii) the application of the tuned filter to the input image. This work aims at: i) explicitly delineating the characteristic of an approach for the spatial simplification of images based on connected filters; ii) defining a general architecture suitable for the analysis in different scenarios modeling common different operative conditions; iii) giving guidelines for the automation of the simplification process according to different operational settings; iv) qualitatively evaluating the application of the proposed approach on a real data set in different scenarios.

Extended Morphological Attribute Profiles (EAPs) are extension of Extended Morphological Profiles (EMPs). They are based on the more general Morphological Attribute Profiles (APs) rather than the conventional Morphological Profiles (MPs). EAPs are computed on few of the first principle components (PCs) extracted from the multi-/hyper-spectral data. In this paper, we propose to compute EAPs on features derived from supervised feature extraction techniques such as discriminant analysis feature extraction (DAFE), decision boundary feature extraction (DBFE) and non-parametric weighted feature extraction (NWFE)) instead of using unsupervised principal component analysis (PCA).

The unsupervised classification of hyperspectral images containing mixed pixels is addressed in this paper. Hyperspectral images are characterized by a trade-off between the spectral and the spatial resolution, this leading to data sets containing mixed pixels, e.g. pixels jointly occupied by more than a single land cover class. In [1], a preliminary research based on spectral unmixing concepts was conducted, in order to handle mixed pixels and to obtain thematic maps at a finer spatial resolution. In this work, we extend the investigation by proposing a new methodology based on image clustering. Experiments conducted on real data show the comparative effectiveness of the proposed method, which provides good results in terms of accuracy and is less sensitive to pixels with extreme values of reflectance.

In this letter, a technique based on independent component analysis (ICA) and extended morphological attribute profiles (EAPs) is presented for the classification of hyperspectral images. The ICA maps the data into a subspace in which the components are as independent as possible. APs, which are extracted by using several attributes, are applied to each image associated with an extracted independent component, leading to a set of extended EAPs. Two approaches are presented for including the computed profiles in the analysis. The features extracted by the morphological processing are then classified with an SVM. The experiments carried out on two hyperspectral images proved the effectiveness of the proposed technique.

In this paper, the use of Independent Component (IC) Discriminant Analysis (ICDA) for remote sensing classification is proposed. ICDA is a nonparametric method for discriminant analysis based on the application of a Bayesian classification rule on a signal composed by ICs. The method uses IC Analysis (ICA) to choose a transform matrix so that the transformed components are as independent as possible. When the data are projected in an independent space, the estimates of their multivariate density function can be computed in a much easier way as the product of univariate densities. A nonparametric kernel density estimator is used to compute the density functions of each IC. Finally, the Bayes rule is applied for the classification assignment. In this paper, we investigate the possibility of using ICDA for the classification of hyperspectral images. We study the influence of the algorithm used to enforce independence and of the number of IC retained for the classification, proposing an effective method to estimate the most suitable number. The proposed method is applied to several hyperspectral images, in order to test different data set conditions (urban/agricultural area, size of the training set, and type of sensor). Obtained results are compared with one of the most commonly used classifier of hyperspectral images (support vector machines) and show the comparative effectiveness of the proposed method in terms of accuracy.

The definition of the Mahalanobis kernel for the classification of hyperspectral remote sensing images is addressed. Class specific covariance matrices are regularized by a probabilistic model which is based on the data living in a subspace spanned by the p first principal components. The inverse of the covariance matrix is computed in a closed form and is used in the kernel to compute the distance between two spectra. Each principal direction is normalized by a hyperparameter tuned, according to an upper error bound, during the training of an SVM classifier. Results on real data sets empirically demonstrate that the proposed kernel leads to an increase of the classification accuracy by comparison to standard kernels.

In this paper we investigate the combined use of morphological attribute filters and feature extraction techniques for the classification of a high resolution hyperspectral image. In greater detail, we propose to model the spatial information with Extended Attribute Profiles computed on the hyperspectral data and to reduce the high dimensionality of the morphological features computed (which show a high degree of redundancy) with feature extraction techniques. The features extracted are analyzed by two classifiers. The experimental analysis was carried out on a high resolution hyperspectral image acquired by the airborne sensor ROSIS-03 on the University of Pavia, Italy. The obtained results compared to those obtained without feature reduction proved the importance of the application of a stage of feature extraction in the process.

Classification of high resolution urban remote sensing imagery is addressed. The classification is done by both considering the panchromatic imagery and the multi-spectral image obtained using the spectrally consistent fusion method introduced in [1]. The data are classified using support vector machines (SVM). To further enhance the classification accuracy, mathematical morphology is used to derive local spatial information from the panchromatic data. In particular we use the Morphological Profile (MP) in classification of satellite imagery as was proposed in [2, 3]. We also use the derivative of the MP (DMP). In the majority of the image fusion (pansharpening) techniques proposed today, there is a compromise between the spatial enhancement and the spectral consistency. By comparing classification results obtained by using our model based scheme [1] to results obtained using the IHS and Brovey fusion methods, we find that spectrally consistent data give better results when it comes to classification.

A new multiple classifier method for spectral-spatial classification of hyperspectral images is proposed. Several classifiers are used independently to classify an image. For every pixel, if all the classifiers have assigned this pixel to the same class, the pixel is kept as a marker, i.e., a seed of the spatial region, with the corresponding class label. We propose to use spectral-spatial classifiers at the preliminary step of the marker selection procedure, each of them combining the results of a pixel-wise classification and a segmentation map. Different segmentation approaches lead to different classification results. Furthermore, a minimum spanning forest is built, where each tree is rooted on a classification-driven marker and forms a region in the spectral-spatial classification map. Experimental results are presented on a 103-band ROSIS image of the University of Pavia, Italy. The proposed method significantly improves classification accuracies, when compared to previously proposed classification techniques.

A novel automatic method is developed for the detection of features in planetary images. Although many automatic feature extraction methods have been proposed for for remote sensing images of the Earth, these methods are typically unfeasible for planetary data that generally present low contrast and uneven illumination characteristics. Here, a novel technique for crater detection, based on a marked point process, is proposed. The main idea behind marked point processes is to model objects within a stochastic framework: They provide a powerful and methodologically rigorous framework to efficiently map and detect objects and structures in an image with an excellent robustness to noise. These methods are new and promising: They represent the last frontier of the stochastic image modeling. They have been used in different areas of the terrestrial remote sensing, but have not been applied to planetary image analysis yet. The proposed method for crater detection has many other areas applications. One such application area is image registration by matching the extracted features.

International audience

The high number of spectral bands acquired by hyperspectral sensors increases the capability to distinguish physical materials and objects, presenting new challenges to image analysis and classification. This letter presents a novel method for accurate spectral-spatial classification of hyperspectral images. The proposed technique consists of two steps. In the first step, a probabilistic support vector machine pixelwise classification of the hyperspectral image is applied. In the second step, spatial contextual information is used for refining the classification results obtained in the first step. This is achieved by means of a Markov random field regularization. Experimental results are presented for three hyperspectral airborne images and compared with those obtained by recently proposed advanced spectral-spatial classification techniques. The proposed method improves classification accuracies when compared to other classification approaches.

Morphological attribute profiles (APs) are defined as a generalization of the recently proposed morphological profiles (MPs). APs provide a multilevel characterization of an image created by the sequential application of morphological attribute filters that can be used to model different kinds of the structural information. According to the type of the attributes considered in the morphological attribute transformation, different parametric features can be modeled. The generation of APs, thanks to an efficient implementation, strongly reduces the computational load required for the computation of conventional MPs. Moreover, the characterization of the image with different attributes leads to a more complete description of the scene and to a more accurate modeling of the spatial information than with the use of conventional morphological filters based on a predefined structuring element. Here, the features extracted by the proposed operators were used for the classification of two very high resolution panchromatic images acquired by Quickbird on the city of Trento, Italy. The experimental analysis proved the usefulness of APs in modeling the spatial information present in the images. The classification maps obtained by considering different APs result in a better description of the scene (both in terms of thematic and geometric accuracy) than those obtained with an MP.

A new multiple-classifier approach for spectral–spatial classification of hyperspectral images is proposed. Several classifiers are used independently to classify an image. For every pixel, if all the classifiers have assigned this pixel to the same class, the pixel is kept as a marker, i.e., a seed of the spatial region with a corresponding class label. We propose to use spectral–spatial classifiers at the preliminary step of the marker-selection procedure, each of them combining the results of a pixelwise classification and a segmentation map. Different segmentation methods based on dissimilar principles lead to different classification results. Furthermore, a minimum spanning forest is built, where each tree is rooted on a classification-driven marker and forms a region in the spectral–spatial classification map. Experimental results are presented for two hyperspectral airborne images. The proposed method significantly improves classification accuracies when compared with previously proposed classification techniques.

The accuracy of supervised land cover classifications depends on factors such as the chosen classification algorithm, adequate training data, the input data characteristics, and the selection of features. Hyperspectral imaging provides more detailed spectral and spatial information on the land cover than other remote sensing resources. Over the past ten years, traditional and formerly widely accepted statistical classification methods have been superseded by more recent machine learning algorithms, e.g., support vector machines (SVMs), or by multiple classifier systems (MCS). This can be explained by limitations of statistical approaches with regard to high-dimensional data, multimodal classes, and often limited availability of training data. In the presented study, MCSs based on SVM and random feature selection (RFS) are applied to explore the potential of a synergetic use of the two concepts. We investigated how the number of selected features and the size of the MCS influence classification accuracy using two hyperspectral data sets, from different environmental settings. In addition, experiments were conducted with a varying number of training samples. Accuracies are compared with regular SVM and random forests. Experimental results clearly demonstrate that the generation of an SVM-based classifier system with RFS significantly improves overall classification accuracy as well as producer's and user's accuracies. In addition, the ensemble strategy results in smoother, i.e., more realistic, classification maps than those from stand-alone SVM. Findings from the experiments were successfully transferred onto an additional hyperspectral data set.

A new method for segmentation and classification of hyperspectral images is proposed. The method is based on the construction of a minimum spanning forest (MSF) from region markers. Markers are defined automatically from classification results. For this purpose, pixelwise classification is performed, and the most reliable classified pixels are chosen as markers. Each classification-derived marker is associated with a class label. Each tree in the MSF grown from a marker forms a region in the segmentation map. By assigning a class of each marker to all the pixels within the region grown from this marker, a spectral-spatial classification map is obtained. Furthermore, the classification map is refined using the results of a pixelwise classification and a majority voting within the spatially connected regions. Experimental results are presented for three hyperspectral airborne images. The use of different dissimilarity measures for the construction of the MSF is investigated. The proposed scheme improves classification accuracies, when compared to previously proposed classification techniques, and provides accurate segmentation and classification maps.

This paper presents a new method for extracting roads in Very High Resolution remotely sensed images based on advanced directional morphological operators. The proposed approach introduces the use of Path Openings and Closings in order to extract structural pixel information. These morphological operators remain flexible enough to fit rectilinear and slightly curved structures since they do not depend on the choice of a structural element shape and hence outperform standard approaches using rotating rectangular structuring elements. The method consists in building a granulometry chain using Path Openings and Closing to perform Morphological Profiles. For each pixel, the Morphological Profile constitutes the feature vector on which our road extraction is based.

Greedy modular eigenspaces (GME) has been developed for the band selection of hyperspectral images (HSI). GME attempts to greedily select uncorrelated feature sets from HSI. Unfortunately, GME is hard to find the optimal set by greedy operations except by exhaustive iterations. The long execution time has been the major drawback in practice. Accordingly, finding an optimal (or near-optimal) solution is very expensive. In this study we present a novel parallel mechanism, referred to as parallel particle swarm optimization (PPSO) band selection, to overcome this disadvantage. It makes use of a new particle swarm optimization scheme, a well-known method to solve the optimization problems, to develop an effective parallel feature extraction for HSI. The proposed PPSO improves the computational speed by using parallel computing techniques which include the compute unified device architecture (CUDA) of graphics processor unit (GPU), the message passing interface (MPI) and the open multi-processing (OpenMP) applications. These parallel implementations can fully utilize the significant parallelism of proposed PPSO to create a set of near-optimal GME modules on each parallel node. The experimental results demonstrated that PPSO can significantly improve the computational loads and provide a more reliable quality of solution compared to GME. The effectiveness of the proposed PPSO is evaluated by MODIS/ASTER airborne simulator (MASTER) HSI for band selection during the Pacrim II campaign.

Kernel Principal Component Analysis (KPCA) is investigated for feature extraction from hyperspectral remote-sensing data. Features extracted using KPCA are used to construct the Extended Morphological Profile (EMP). Classification results, in terms of accuracy, are improved in comparison to original approach which used conventional principal component analysis for constructing the EMP. Experimental results presented in this paper confirm the usefulness of the KPCA for the analysis of hyperspectral data. The overall classification accuracy increases from 79% to 96% with the proposed approach.

This paper proposes the use of morphological attribute profiles as an effective alternative to the conventional morphological operators based on the geodesic reconstruction for modeling the spatial information in very high resolution images. Attribute profiles, used in multilevel approaches, result particularly effective in terms of computational complexity and capabilities in characterizing the objects in the image. In addition they are more flexible than operators by reconstruction, thanks to the definition of possible different attributes. Experimental results obtained on a Quickbird panchromatic very high resolution image proved the effectiveness of the presented attribute filters and pointed out their main properties.

Synthetic aperture radar (SAR) images are corrupted by speckle noise due to random interference of electromagnetic waves. The speckle degrades the quality of the images and makes interpretation, analysis and classification of SAR images harder. Therefore, some speckle reduction is necessary prior to the processing of SAR images. The speckle noise can be modeled as multiplicative i.i.d. Rayleigh noise. Sveinsson and Benediktsson [1996], proposed an adaptive sigmoid thresholding method for SAR images in the wavelet domain. The coefficients thresholding for this method is based on the choice of parameters in the sigmoid thresholding function. They were chosen according to a visual appreciation, i.e., by anad hocmethod. We propose to select these parameters by minimizing an estimate of square error between the clean image and the denoised one. The key point is that we have in our proposal computable, statistically unbiased, MSE estimate - Stein's Unbiased Risk Estimate (SURE) - that depends on the noisy image alone, not on the clean image. We apply the proposed method on an SAR images, both simulated and real data.

The nonsubsampled contourlet transform (NSCT) is a new image representation approach that has sparser representation at both spatial and directional resolution and thus captures smooth contours in images. On the other hand, wavelet transform has sparser representation of homogeneous areas. In this paper, we are going to use the three combinations of undecimated wavelet and nonsubsampled contourlet transforms that was used in for denoising of TerraSAR-X images. Two of the methods use the undecimated wavelet transform to de-noise homogeneous areas and the nonsubsampled contourlet transform to denoise areas with edges. The segmentation between homogeneous areas and areas with edges is done by using total variation segmentation. The third method is a linear averaging of the two denoising methods. A thresholding in the wavelet and contourlet domain is done by non-linear functions which are adapted for each selected subband. The non-linear functions are based on sigmoid functions. Simulation results suggested that these denoising schemes achieve good and clean images.

A new method for segmentation and classification of hyper-spectral images is proposed. The method is based on a pixel-wise classification followed by selection of the most reliable classified pixels as markers for watershed segmentation. Furthermore, each marker defined from classification results is associated with a class label. By assigning the class label of each marker to all the pixels within the region grown from this marker, a spectral-spatial classification map is obtained. Experimental results are presented on a 200-band AVIRIS image of the Northwestern Indiana's Indian Pine site. The developed segmentation and classification scheme significantly decreases oversegmentation, improves classification accuracies and provides classification maps with more homogeneous regions, when compared to pixel-wise classification or previously proposed spectral-spatial classification techniques.

Independent component analysis (ICA) is a very popular method that has shown success in blind source separation, feature extraction and unsupervised recognition. In recent years ICA has been largely studied by researchers from the signal processing community. This paper addresses a more in-depth study on the use of this method, applied to hyper-spectral images used for remote sensing purposes. In a first part, source separation is addressed. Since the independence of sources is usually not verified in hyperspectral real data images, ICA, if used alone, is not a suitable tool to unmix sources. We propose a hierarchical approximation for the use of ICA as a pre-processing step for a Bayesian Positive Source Separation method. In a second part, the use of ICA for dimensionality reduction is studied in the frame of hyperspectral data classification. Experimental results show the effectiveness of ICA when used for hyperspectral image pre-processing for the two considered applications.

Classification of hyperspectral data with high spatial resolution from urban areas is investigated. The approach is an extension of existing approaches, using both spectral and spatial information for classification. The spatial information is derived by mathematical morphology and principal components of the hyperspectral data set, generating a set of different morphological profiles. The whole data set is classified by the Random Forest algorithm. However, the computational complexity as well as the increased dimensionality and redundancy of data sets based on morphological profiles are potential drawbacks. Thus, in the presented study, feature selection is applied, using nonparametric weighted feature extraction and the variable importance of the random forests. The proposed approach is applied to ROSIS data from an urban area. The experimental results demonstrate that a feature reduction is useful in terms of accuracy. Moreover, the proposed approach also shows excellent results with a limited training set.

An approach for spectral-spatial classification of multisource remote sensing data from agricultural areas is addressed. Mathematical morphology is used to derive the spatial information from the data sets. The different data sources (i.e., SAR and multispectral) are classified by support vector machines (SVM). Afterwards, the SVM outputs are transferred to probability measurements. These probability values are combined by different fusion strategies, to derive the final classification result. Comparing the results based on mathematical morphology the total accuracy increased by 6% compared to the pure-pixel classification results. Moreover the transfer of the SVM outputs into probability values and the subsequent fusion further increases the classification accuracy, resulting in an accuracy of 78.5%.

In real applications, it is difficult to obtain a sufficient number of training samples in supervised classification of hyperspectral remote sensing images. Furthermore, the training samples may not represent the real distribution of the whole space. To attack these problems, an ensemble algorithm which combines generative (mixture of Gaussians) and discriminative (support cluster machine) models for classification is proposed. Experimental results carried out on hyperspectral data set collected by the reflective optics system imaging spectrometer sensor, validates the effectiveness of the proposed approach.

A new spectral-spatial classification scheme for hyperspectral images is proposed. The method combines the results of a pixel wise support vector machine classification and the segmentation map obtained by partitional clustering using majority voting. The ISODATA algorithm and Gaussian mixture resolving techniques are used for image clustering. Experimental results are presented for two hyperspectral airborne images. The developed classification scheme improves the classification accuracies and provides classification maps with more homogeneous regions, when compared to pixel wise classification. The proposed method performs particularly well for classification of images with large spatial structures and when different classes have dissimilar spectral responses and a comparable number of pixels.

The pixel-wise classification of hyperspectral images with a reduced training set is addressed. The joint use of the spectral and the spatial information is investigated. The spectral information simply consists of the spectral value of each pixel. For the spatial information, we use an area filter to simplify the image and extract consistent connected components. These components are used to define an adaptive neighborhood for each pixel of the image. The vector median value of each component is defined as a spatial feature for the classification. Support Vector Machines are used for the classification and a composite kernel is used to combine both the spatial and the spectral information. Experiments are conducted on AVIRIS hyperspectral data. The proposed approach provides significant improvements in terms of classification accuracy when compared with a standard statistical method (maximum likelihood) and with a SVM classifier using the spectral information alone. Robustness with respect to the size of the training set is also investigated.

The classification of hyperspectral data is addressed using a classifier ensemble based on Support Vector Machines (SVM). First of all, the hyperspectral data set is decomposed into few sources according to the spectral bands correlation. Then, each source is treated separately and classified by an SVM classifier. Finally, all outputs are used as inputs for the final decision fusion, performed by an additional SVM classifier. The results of experiments, clearly show that the proposed SVM-based decision fusion outperforms a single SVM classifier in terms of overall accuracies.

Hyperspectral remote sensing images play a very important role in the discrimination of spectrally similar land-cover classes. In order to obtain a reliable classifier, a larger amount of representative training samples are necessary compared to multi-spectral remote sensing data. In real applications, it is difficult to obtain a sufficient number of training samples for supervised learning. Besides, the training samples may not represent the real distribution of the whole space. To attack the quality problems of training samples, we proposed a Cluster-based ENsemble Algorithm (CENA) for the classification of hyperspectral remote sensing images. Data set collected from ROSIS university validates the effectiveness of the proposed approach.

The nonsubsampled contourlet transform (NSCT) is a new image representation approach that has sparser representation at both spatial and directional resolution and thus captures smooth contours in images On the other hand, wavelet transform has sparser representation of homogeneous areas. In this paper, three combinations of undecimated wavelet and nonsubsampled contourlet transforms will be used for denoising of SAR images. Two of the methods use the wavelet transform to denoise homogeneous areas and the nonsubsampled contourlet transform to denoise areas with edges. The segmentation between homogeneous areas and areas with edges is done by using total variation segmentation. The third method is a linear averaging of the two denoising methods. A thresholding in the wavelet and contourlet domain is done by non-linear functions which are adapted for each selected subband. The non-linear functions are based on sigmoid functions. Simulation results suggested that these denoising schemes achieve good and clean images.

Synthetic Aperture Radar (SAR) images are inherently affected by multiplicative speckle noise, which is due to the coherent nature of the scattering phenomenon. This paper deals with the speckle reduction using the bandlet transform combined with the adaptive sigmoid thresholding. The operation needs to provide multiscale transform. We use the Undecimated Discrete Wavelet Transform (UDWT) and apply the bandlet transform on each resulting scale. Numerical tests applied on Lena image contaminated with multiplicative noise show that our method provides improvement both in terms of image visual fidelity and in terms of Peak Signal-to-Noise Ratio (PSNR). Comparisons are made with the standard wavelet transform and the shift invariant discrete time wavelet transform.

The paper presents a new segmentation and classification scheme to analyze hyperspectral (HS) data. The Robust Color Morphological Gradient of the HS image is computed, and the watershed transformation is applied to the obtained gradient. After the pixel-wise Support Vector Machines classification, the majority voting within the watershed regions is performed. Experimental results are presented on a 103-airborne ROSIS image, of the University of Pavia, Italy. The integration of the spatial information from the watershed segmentation into the HS image classification improves the classification accuracies, when compared to the pixel-wise classification.

The subject of this work is the model selection of kernels with multiple parameters for support vector machines (SVM), with the purpose of classifying hyperspectral remote sensing data. During the training process, the kernel parameters need to be tuned properly. In this work a gradient descent based algorithm is used to estimate the parameters. The selection of multiple parameters is addressed, and an approach based on the analysis of the variance values of individual bands was proposed. Several state of the art kernels were tested. Experiments were conducted on real hyperspectral data. Results obtained with the different approaches/kernels were compared statistically, and showed good results in terms classification accuracies and processing time.

The analysis of data sets, which were acquired within different time periods over the same geographical region is interesting for updating land cover maps and operational monitoring systems. In this context an adequate and temporally stable classification approach is worthwhile. In the presented study a classifier ensemble (i.e., random forests) is trained on a multispectral image from an agricultural region from and is successively modified and adapted, to classify a data set from another year. A detailed accuracy assessment clearly demonstrates that the proposed modification of the classifier significantly improves the overall accuracy, whereas a simple transfer of a classifier to a data set from another year is limited and results in a decreased accuracy. Thus the proposed approach can be recommended for classifying multiannual data sets and updating land cover maps.

An unsupervised technique for change detection (CD) in very high geometrical resolution images is proposed, which is based on the use of morphological filters. This technique integrates the nonlinear and adaptive properties of the morphological filters with a change vector analysis (CVA) procedure. Different morphological operators are analyzed and compared with respect to the CD problem. Alternating sequential filters by reconstruction proved to be the most effective, permitting the preservation of the geometrical information of the structures in the scene while filtering the homogeneous areas. Experimental results confirm the effectiveness of the proposed technique. It increases the accuracy of the CD process as compared with the standard CVA approach.

A method is proposed for pixel-level satellite image fusion derived directly from a model of the imaging sensor. By design, the proposed method is spectrally consistent. It is argued that the proposed method needs regularization, as is the case for any method for this problem. A framework for pixel neighborhood regularization is presented. This framework enables the formulation of the regularization in a way that corresponds well with our prior assumptions of the image data. The proposed method is validated and compared with other approaches on several data sets. Lastly, the intensity-hue-saturation method is revisited in order to gain additional insight of what implications the spectral consistency has for an image fusion method.

Announcement of Christopher J. Ruf from the University of Michigan as the new editor of the IEEE Transactions on Geoscience and Remote Sensing (TGRS).

A method is proposed for the classification of urban hyperspectral data with high spatial resolution. The approach is an extension of previous approaches and uses both the spatial and spectral information for classification. One previous approach is based on using several principal components (PCs) from the hyperspectral data and building several morphological profiles (MPs). These profiles can be used all together in one extended MP. A shortcoming of that approach is that it was primarily designed for classification of urban structures and it does not fully utilize the spectral information in the data. Similarly, the commonly used pixelwise classification of hyperspectral data is solely based on the spectral content and lacks information on the structure of the features in the image. The proposed method overcomes these problems and is based on the fusion of the morphological information and the original hyperspectral data, i.e., the two vectors of attributes are concatenated into one feature vector. After a reduction of the dimensionality, the final classification is achieved by using a support vector machine classifier. The proposed approach is tested in experiments on ROSIS data from urban areas. Significant improvements are achieved in terms of accuracies when compared to results obtained for approaches based on the use of MPs based on PCs only and conventional spectral classification. For instance, with one data set, the overall accuracy is increased from 79% to 83% without any feature reduction and to 87% with feature reduction. The proposed approach also shows excellent results with a limited training set.

Reports the death of Michael J. Barnsley, Pro Vice Chancellor of Swansea University, Swansea, U.K., and an Associate Editor for Transactions on Geoscience and Remote Sensing.

Different spatial priors for satellite image fusion are evaluated through experiments on three different data sets. The results are judged visually as well as quantified via different image quality metrics on a down-sampled data-set. It is done within our previously proposed spectrally consistent pan- sharpening framework (SCP). This is a per pixel based fusion framework constructed by considering the imaging physics.

The classification of very high resolution panchromatic images from urban areas is addressed. The spectral information, i.e. the gray level of each pixel, does generally not ensure a reliable classification. In this paper, we investigate the use of an area filter to extract information about the inter-pixel dependency. The classification is then performed using a support vector machines (SVM) classifier. Using a linear composition of kernels, we define a kernel using both the spectral (original gray level) and the spatial information. A weighting parameter, controlling the relative importance of each feature, is introduced and tuned during the SVM's training process. Experiments have been conducted on simulated panchromatic Pleiades data over Toulouse, France. Results obtained with the proposed approach is positively compared to those obtained with the standard use of gray value information only and classical SVM formulation.

Classification of hyperspectral data with high spatial resolution from urban areas is discussed. An approach has been proposed which is based on using several principal components from the hyperspectral data and build morphological profiles. These profiles can be used all together in one extended morphological profile. A shortcoming of the approach is that it is primarily designed for classification of urban structures and it does not fully utilize the spectral information in the data. Similarly, a pixel-wise classification solely based on the spectral content can be performed, but it lacks information on the structure of the features in the image. An extension is proposed in this paper in order to overcome these dual problems. The proposed method is based on the data fusion of the morphological information and the original hyperspectral data: the two vectors of attributes are concatenated. After a reduction of the dimensionality using Decision Boundary Feature Extraction, the final classification is achieved using a Support Vector Machines classifier. The proposed approach is tested in experiments on ROSIS data from urban areas. Significant improvements are achieved in terms of accuracies when compared to results of approaches based on the use of morphological profiles based on PCs only and conventional spectral classification.

This paper explains some of the the goals and objectives of the newly started HYPER-I-NET Marie Curie Research and Training Network. In particular, the requirements related to the definition and implementation of an efficient, adequate and sufficiently general data processing chain for hyperspectral data analysis are considered. Some of the research lines that are expected to play a central role in the activities of this network are also presented and briefly discussed.

Abstract—This paper addresses the main goals and objec-tives of the Hyperspectral Imaging Network (HYPER-I-NET), a recently started Marie Curie Research Training Network. The project is designed to build an interdisciplinary research community focusing on hyperspectral imaging activities. The core strategy of the network is to create a powerful interdisciplinary synergy between different domains of expertise closely related to hyperspectral imaging activities in Europe, ranging from sensor design and flight operation to data collection, processing, interpretation, and dissemination. Our main goals in this paper are to present the project to the Geoscience and Remote Sensing community and to provide an overview of the planned activities in each sub-activity covered by the network.

Several widely used methods have been proposed for fusing high resolution panchromatic data and lower resolution multi-channel data. However, many of these methods fail to maintain the spectral consistency of the fused high resolution image, which is of high importance to many of the applications based on satellite data. Additionally, most conventional methods are loosely connected to the image forming physics of the satellite image, giving these methods an ad hoc feel. Vesteinsson et al. [1] proposed a method of fusion of satellite images that is based on the properties of imaging physics in a statistically meaningful way and was called spectral consistent panshapening (SCP). In this paper we improve this framework for satellite image fusion by introducing a better image prior, via data-dependent image smoothing. The dependency is obtained via total variation edge detection method.

Synthetic aperture radar (SAR) images are corrupted by speckle noise due to random interference of electromagnetic waves. The speckle degrades the quality of the images and makes interpretations, analysis and classifications of SAR images harder. Therefore, some speckle reduction is necessary prior to the processing of SAR images. The speckle noise can be modeled as multiplicative i.i.d. Rayleigh noise. The discrete curvelet transform is a new image representation approach that codes image edges more efficiently than the wavelet transform. On the other hand, wavelet transform codes homogeneous areas better than curvelet transform. In this paper, two combinations of time invariant wavelet and curvelet transforms will be used for denoising of SAR images. Both of the methods use the wavelet transform to denoise homogeneous areas and the curvelet transform to denoise areas with edges. The segmentation between homogeneous areas and areas with edges is done by using total variation segmentation. Simulation results suggested that these denoised schemas can achieve good and clean images.

A concept for classifying multisensor data sets, consisting of multispectral and SAR imagery is introduced. Each data source is separately classified by a support vector machine (SVM). In a decision fusion the outputs of the preliminary SVMs are used to determine the final class memberships. This fusion is performed by another SVM as well as two common voting schemes. The results are compared with well-known parametric and nonparametric classifier methods. The proposed SVM-based fusion approach outperforms all other concepts and significantly improves the results of a single SVM that is trained on the whole multisensor data set.

The classification of multisensor data sets, consisting of multitemporal synthetic aperture radar data and optical imagery, is addressed. The concept is based on the decision fusion of different outputs. Each data source is treated separately and classified by a support vector machine (SVM). Instead of fusing the final classification outputs (i.e., land cover classes), the original outputs of each SVM discriminant function are used in the subsequent fusion process. This fusion is performed by another SVM, which is trained on the a priori outputs. In addition, two voting schemes are applied to create the final classification results. The results are compared with well-known parametric and nonparametric classifier methods, i.e., decision trees, the maximum-likelihood classifier, and classifier ensembles. The proposed SVM-based fusion approach outperforms all other approaches and significantly improves the results of a single SVM, which is trained on the whole multisensor data set.

Classification of hyperspectral remote sensing data with support vector machines (SVMs) is investigated. SVMs have been introduced recently in the field of remote sensing image processing. Using the kernel method, SVMs map the data into higher dimensional space to increase the separability and then fit an optimal hyperplane to separate the data. In this paper, two kernels have been considered. The generalization capability of SVMs as well as the ability of SVMs to deal with high dimensional feature spaces have been tested in the situation of very limited training set. SVMs have been tested on real hyperspectral data. The experimental results show that SVMs used with the two kernels are appropriate for remote sensing classification problems

International audience

International audience

Hyperspectral imaging offers the possibility of characterizing materials and objects in the air, land and water on the basis of the unique reflectance patterns that result from the interaction of solar energy with the molecular structure of the material. In this paper, we provide a seminal view on recent advances in techniques for hyperspectral data processing. Our main focus is on the development of approaches able to naturally integrate the spatial and spectral information available from the data. Special attention is paid to techniques that circumvent the curse of dimensionality introduced by high-dimensional data spaces. Experimental results, focused in this work on a specific case-study of urban data analysis, demonstrate the success of the considered techniques. This paper represents a first step towards the development of a quantitative and comparative assessment of advances in hyperspectral data processing techniques.

Several widely used methods have been proposed for fusing high resolution panchromatic data and lower resolution multi-channel data. However, many of these methods fail to maintain spectral consistency of the fused high resolution image, which is of high importance to many of the applications based on satellite data. Additionally, most conventional methods are loosely connected to the image forming physics of the satellite image, giving these methods an ad hoc feel. Vesteinsson et al. (2005) proposed a method of fusion of satellite images that is based on the properties of imaging physics in a statistically meaningful way. The fusion method was called spectral consistent panshapen- ing (SC) and it was shown that spectral consistency was a direct consequence of imaging physics and hence guaranteed by the SCP. In this paper exploit this framework and investigate two smoothing methods of the fused image obtain by SCP. The first smoothing method is based on Markov random field (MRF) model, while the second method uses wavelet domain hidden Markov models (HMM) for smoothing of the SCP fused image.

The classification of very high-resolution remotely sensed images from urban areas is addressed. Previous studies have shown the interest of exploiting the local geometrical information of each pixel to improve the classification. This is performed using the derivative morphological profile (DMP) obtained with a granulometric approach, using opening and closing operators. For each pixel, this DMP constitutes the feature vector on which the classification is based. In this letter, we present an interpretation of the DMP in terms of a fuzzy measurement of the characteristic size and contrast of each structure. This fuzzy measure can be compared to predefined possibility distributions to derive a membership degree for a set of given classes. The decision is taken by selecting the class with the highest membership degree. This model is illustrated and validated in a classification problem using IKONOS images.

The classification of very high resolution remote sensing images from urban areas is addressed by considering the fusion of multiple classifiers which provide redundant or complementary results. The proposed fusion approach is in two steps. In a first step, data are processed by each classifier separately, and the algorithms provide for each pixel membership degrees for the considered classes. Then, in a second step, a fuzzy decision rule is used to aggregate the results provided by the algorithms according to the classifiers' capabilities. In this paper, a general framework for combining information from several individual classifiers in multiclass classification is proposed. It is based on the definition of two measures of accuracy. The first one is a pointwise measure which estimates for each pixel the reliability of the information provided by each classifier. By modeling the output of a classifier as a fuzzy set, this pointwise reliability is defined as the degree of uncertainty of the fuzzy set. The second measure estimates the global accuracy of each classifier. It is defined a priori by the user. Finally, the results are aggregated with an adaptive fuzzy operator ruled by these two accuracy measures. The method is tested and validated with two classifiers on IKONOS images from urban areas. The proposed method improves the classification results when compared with the separate use of the different classifiers. The approach is also compared with several other fuzzy fusion schemes

Abstract — The fusion of methods for the classification of panchromatic high resolution satellite remote sensing images from urban areas is addressed. In this paper we propose to aggregate the results provided by different classifiers with complementary properties. Classical combination rules fail to properly combine conflictual information or information provided by sources with different reliabilities. To overcome those problems, the proposed approach is based on the definition of two measures of accuracy. Based on fuzzy set theory, both a local and a global accuracy are defined for each classifier. The fusion is then performed with an adaptive fuzzy combination operator. In terms of classification ac-curacy, the proposed method performs better than each classifier used separately. Results are presented on IKONOS images. I.

Two random forest (RF) approaches are explored; the RF-BHC (binary hierarchical classifier) and the RF-CART (classification and regression tree). Both methods are based on a collection (forest) of tree-like classifier systems where the difference is in the way the trees are grown. The BHC approach depends on class separability measures and the Fisher projection, which maximizes the Fisher discriminant where each tree is a class hierarchy, and the number of leaves is the same as the number of classes. The CART approach is based on CART-like trees where trees are grown to minimize an impurity measure. Here, these different RF approaches are compared in experiments. The RF approaches were investigated in experiments by classification of an urban area from Pavia, Italy using hyperspectral ROSIS (reflective optics system imaging spectrometer) data provided by DLR.

Classification of high-resolution hyperspectral data is investigated. Previously, in classification of high-resolution panchromatic data, simple morphological profiles have been constructed with a repeated use of morphological opening and closing operators with a structuring element of increasing size, starting with the original panchromatic image. This approach has recently been extended for hyperspectral data. In the extension, principal components of the hyperspectral imagery have been computed in order to produce an extended morphological profile. In this paper, we investigate the use of independent components instead of principal components in extended morphological profiles, i.e., selected independent components are used as base images for an extended morphological profile. In the proposed approach, the extended morphological profiles based on the independent components are used as inputs to a neural network classifier. In experiments, a hyperspectral data sets from an urban area in Pavia, Italy is classified.

Abstract — A method for street tracking is proposed. The method consists of two steps. First, a “blob image ” of possible street candidates is created. Then, the street segments from that blob image are extracted. Two feature extraction approaches based on mathematical morphology are applied as preprocessing for the street tracking. One method is based on using differential morphological profiles but the other uses morphological opening and closing operators with a rotating structuring element (SE). The method is tested on an AIRSAR image from Los Angeles with and without noise filtering. The obtained results are measured using two indexes; correctness and completeness. Of the two methods used in the feature extraction, the SE rotation appears to give better results. Noise filtering does not have a major effect in street tracking for the AIRSAR image. I.

Classification of hyperspectral data with high spatial resolution from urban areas is investigated. A method based on mathematical morphology for preprocessing of the hyperspectral data is proposed. In this approach, opening and closing morphological transforms are used in order to isolate bright (opening) and dark (closing) structures in images, where bright/dark means brighter/darker than the surrounding features in the images. A morphological profile is constructed based on the repeated use of openings and closings with a structuring element of increasing size, starting with one original image. In order to apply the morphological approach to hyperspectral data, principal components of the hyperspectral imagery are computed. The most significant principal components are used as base images for an extended morphological profile, i.e., a profile based on more than one original image. In experiments, two hyperspectral urban datasets are classified. The proposed method is used as a preprocessing method for a neural network classifier and compared to more conventional classification methods with different types of statistical computations and feature extraction.

Classification of hyperspectral data with high spatial resolution using both spatial and spectral approaches is discussed. The spatial approach is based on mathematical morphology. In the method, several principal components (PCs) from the hyperspectral data are used. From each of the PCs, a morphological profile is built. These profiles are used together in one extended morphological profile, which is then classified with a neural network. The spectral classification approach is based on maximum likelihood classification and nonparametric weighted feature extraction (NWFE). The results from the spectral and spatial modeling are finally fused together using several different fusion rules. Experimental results are given on a hyperspectral data from an urban area.

The problem of classification of high-resolution remotely sensed images from urban areas is addressed. Previous studies have shown the interest of exploiting the local geometrical information of each pixel to improve the classification. This is performed using the derivative morphological profile (DMP) obtained with a granulometric approach, using respectively opening and closing operators. For each pixel, this DMP constitutes the feature vector on which the classification is based. In this paper, this vector is considered as a fuzzy measurement of the size of the structure. Compared with some possibility distributions, a membership degree is computed for each class. The decision is taken by selecting the class with the highest membership degree.

The use of random forests for classification of multisource data is investigated in this paper. Random Forest is a classifier that grows many classification trees. Each tree is trained on a bootstrapped sample of the training data, and at each node the algorithm only searches across a random subset of the variables to determine a split. To classify an input vector in random forest, the vector is submitted as an input to each of the trees in the forest, and the classification is then determined by a majority vote. The experiments presented in the paper were done on a multisource remote sensing and geographic data set. The experimental results obtained with random forests were compared to results obtained by bagging and boosting methods.

Classification of hyperspectral remote sensing data with support vector machines (SVMs) is investigated. SVMs have shown to perform well in terms of classification accuracies for hyperspectral data sets. On the other hand, the computational burden of SVMs in hyperdimensional space can be quite intense. Therefore, it is important to explore approaches, which lighten the computational burden without sacrificing the overall classification accuracies. Two different feature extraction methods, decision boundary feature extraction and nonparametric weighted feature extraction are tested. The hyperspectral data are split into several "independent data sources". The data from each data source are transformed using feature extraction, then two approaches are investigated. In the first approach the data from all sources are classified together with a multisource SVM kernel. In the second approach, the data are classified separately using classical SVM RBF kernel. The results from the SVMs are then fused for final classification. Results are compared and discussed.

Synthetic aperture radar (SAR) images are corrupted by speckle noise due to random interference of electromagnetic waves. The speckle degrades the quality of the images and makes interpretations, analysis and classifications of SAR images harder. Therefore, some speckle reduction is necessary prior to the processing of SAR images. The speckle noise can be modeled as multiplicative i.i.d. Rayleigh noise. Logarithmic transformation of SAR images convert the multiplicative noise models to additive noise. In this paper, two combinations of time invariant wavelet and curvelet transforms will be used for denoising of SAR images. The first one is called the combined filtering algorithm (CFA). This method is based on a constrained optimization problem, both in the wavelet and curvelet domains. The second method is called the adaptive combined method (ACM) which uses the wavelet transform to denoise homogeneous areas and the curvelet transform to denoise areas with edges

Very high resolution hyperspectral data should be very useful to provide detailed maps of urban land cover. In order to provide such maps, both accurate and precise classification tools need, however, to be developed. In this letter, new methods for classification of hyperspectral remote sensing data are investigated, with the primary focus on multiple classifications and spatial analysis to improve mapping accuracy in urban areas. In particular, we compare spatial reclassification and mathematical morphology approaches. We show results for classification of DAIS data over the town of Pavia, in northern Italy. Classification maps of two test areas are given, and the overall and individual class accuracies are analyzed with respect to the parameters of the proposed classification procedures.

The problem of classification of high-resolution remotely sensed images from urban areas is addressed. Previous studies have shown the interest in exploiting the local geometrical information of each pixel to improve the classification. This is performed using the derivative morphological profile obtained with a granulometric approach using respectively opening and closing operators. We propose to replace this by a morphological alternated sequential filter, where the openings and the closings are applied alternately. The results and the robustness provided by the ASF are presented on IKONOS panchromatic data.

The use of Support Vector Machines (SVMs) for classification of multisource data is investigated. SVMs have been shown to have difficulties in classifying multiclass data. To over- come that,the multiclass classification problem considered here was reduced to multiple margin-based binary problems. Several possibilities of binary problems were investigated,including one- against-all,all-pairs and nearly random decompositions of the multiclass problem. To combine the outputs from the binary problems three approaches were tested: a) voting schemes,b) two loss functions,and c) decoding function based on condi- tional probability estimation. An extension of the radial basis function kernel for multisource data is also proposed. The kernel concentrates on local distance between features from each data source. The experimental results show the proposed approach to be appropriate for multisource data classification. I.I NTRODUCTION

The classification of urban data with high spectral and spatial resolution is considered. For processing, a morphological profile is constructed. The morphological profile is based on the repeated use of opening and closings with a differently sized structuring element. Morphological profiles have been shown to contain redundancies. Therefore, feature extraction is applied on the profile. The morphological approach is applied in experiments on high resolution DAIS remote sensing data from an urban area. To apply the morphological approach on the DAIS data, the first principal component is used as a basis for the morphological transformations. In experiments, the use of the morphological method performs well in terms of classification accuracies. With feature extraction, it is observed that classification on reduced features gives higher accuracies than in the original feature space.

Synthetic aperture radar (SAR) images are corrupted by speckle noise due to random interference of electromagnetic waves. The speckle degrades the quality of the images and makes interpretation, analysis and classification of SAR images harder. In this paper we will consider the use of the curvelet transform (CT), for speckle reduction of SAR images. The CT is a new approach for image representation approach that codes image edges more efficiently then the wavelet transform. Edges are very important in image perception and with fewer coefficients to represent edges, a better denoising scheme can be achieved. We will use three denoising methods: Wavelet-domain hidden Markov tree models, hard thresholding of the curvelet coefficients, and an adaptive combined method (ACM) proposed here, which uses the desired aspects of both aforementioned methods.

Wavelet footprints, proposed by Dragotti (2002), are used for speckle reduction of synthetic aperture radar (SAR) images. Wavelet footprints contain all wavelet coefficients associated with a singular structure of a signal. Consequently, the dependency across scales that is inherent in wavelet transformation is eliminated. In the present paper, coefficients of wavelet footprints are thresholded with hard thresholding. The denoising method shows great promise for speckle removal and hence provides good detection performance for SAR based recognition.

Classification of panchromatic high-resolution data from urban areas using morphological and neural approaches is investigated. The proposed approach is based on three steps. First, the composition of geodesic opening and closing operations of different sizes is used in order to build a differential morphological profile that records image structural information. Although, the original panchromatic image only has one data channel, the use of the composition operations will give many additional channels, which may contain redundancies. Therefore, feature extraction or feature selection is applied in the second step. Both discriminant analysis feature extraction and decision boundary feature extraction are investigated in the second step along with a simple feature selection based on picking the largest indexes of the differential morphological profiles. Third, a neural network is used to classify the features from the second step. The proposed approach is applied in experiments on high-resolution Indian Remote Sensing 1C (IRS-1C) and IKONOS remote sensing data from urban areas. In experiments, the proposed method performs well in terms of classification accuracies. It is seen that relatively few features are needed to achieve the same classification accuracies as in the original feature space.

Multisource classification methods based on neural networks, statistical modeling, genetic algorithms, and fuzzy methods are considered. For most of these methods, the individual data sources are at first treated separately and classified by either statistical or neural methods. Then, several decision fusion schemes are applied to combine information from the individual data sources. These schemes include weighted consensus theory where the weights of the individual data sources control the influence of the sources in the combined classification. Using all the data sources individually in consensus-theoretic classification can lead to a redundancy in the classification process. Therefore, a special focus in this letter is on neural networks based on pruning and regularization for combination and classification. The considered methods are applied in classification of a multisource dataset.

Two wavelet transformations are used for speckle reduction and enhancement of synthetic aperture radar (SAR) images. First, a discrete wavelet transformation (DWT) based on oversampled filter banks is used. The oversampled DWT is called a double-density DWT (DD-DWT) and is based on a single-scaling function (low pass) and two distinct wavelet functions (high pass). Second, a discrete wavelet transformation based on two dual real wavelet trees is applied. Each tree produces a set of real DWTs, which together form the complex wavelet transformation (CWT), i.e., a transformation with both real and imaginary parts. Both of these DWTs are almost translation invariant and are useful for speckle reduction through their subband images, and the speckle reduction is obtained by thresholding the subband image coefficients of the digitized SAR images. A thresholding method based on the use of nonlinear functions, which are adapted for each selected subband, is used. The nonlinear functions are based on sigmoid functions. The denoising method presented shows great promise for speckle removal and, hence, can provide good detection performance for SAR-based recognition.

Summary form only. Classification of panchromatic IKONOS data from an urban area in Reykjavik, Iceland is investigated. It is well known that conventional classification algorithms like the Gaussian maximum likelihood method are not appropriate for classification of high-resolution data from urban areas. Therefore, in the paper, an approach based on morphological pre-processing and neural networks is applied. Several types of morphological transformations were used, each based on compositions of morphological opening and closing transforms. Three data sets were created: A) Data 1: Normal morphological opening and closing operations were used. The size of the structural elements was determined as the size of a square structural element and a quadratic increment step was used. The processed image consisted of the original image, 15 opening channels and 15 closing channels (31 channels in total). The derivative of the morphological profile was also computed, resulting in 30 channels. B) Data 2: Open and closing by reconstruction was used with a linear increment step of 1. The size was the number of connected pixels (the area) of structures, and that was in contrast to Data 1, independent of the shape. The original image, 20 opening channels, and 20 closing channels were classified (41 channels in total). The derivative of the morphological profile was also computed and used in classification, resulting in 40 channels. C) Data 3: Same as Data 2, except, a linear increment step of 2 was used instead of an increment of 1. The classification of these different data sets showed interesting characteristics in classification.

Multiple Self-Organizing Feature Maps (MSOMs) can be considered attractive for classification of remote sensing data with many input features. The MSOMs have several advantages, e.g., they are non-parametric, the computational cost for them only grows linearly as a function of the number of features, and they have been shown to approximate posterior probabilities. In the paper MSOMs are investigated for classification of a multisource remote sensing and geographic data set. In the experiments, the MSOM showed potential for classification of the multisource data set.

Discrete wavelet transformations (DWTs) based on oversampled filter banks, proposed by I. W. Selesnick (2001), are used for speckle reduction of SAR images. The oversampled DWT is called double density DWT (DD-DWT) and is based on a single scaling function (lowpass) and two distinct wavelet functions (highpass). The DD-DWT is useful for speckle reduction through its subband images and the speckle reduction is obtained by thresholding the subband-image coefficients of the digitized SAR images. A thresholding method using non-linear functions which are adapted for each selected subband is used in the paper. The non-linear functions are based on sigmoid functions. The denoising method shows great promise for speckle removal and hence can provide good detection performance for SAR based recognition.

A linear feature extraction method based on the discrete wavelet transform (DWT) is applied. A binary genetic algorithm is used to select the best features from the different DWT representations in terms of cost. The feature extraction/selection methods are applied in classification of multisource remote sensing and geographic data. In experiments, the proposed methods performed well in terms of overall accuracies as compared to results obtained with other well-known feature extraction/selection methods.

Curvelet transform (CT), proposed by E. Candes et al. (1999), is used for speckle reduction of SAR images. The CT is useful for speckle reduction through its subband images and the speckle reduction is obtained by thresholding the subband-image coefficients of the digitized SAR images. Two thresholding methods are used; hard thresholding and soft thresholding. The denoising method shows great promise for speckle removal and hence provides good detection performance for SAR based recognition.

The combination of multisource remote sensing and geographic data is believed to offer improved accuracies in land cover classification. For such classification, the conventional parametric statistical classifiers, which have been applied successfully in remote sensing for the last two decades, are not appropriate, since a convenient multivariate statistical model does not exist for the data. In this paper, several single and multiple classifiers, that are appropriate for the classification of multisource remote sensing and geographic data are considered. The focus is on multiple classifiers: bagging algorithms, boosting algorithms, and consensus-theoretic classifiers. These multiple classifiers have different characteristics. The performance of the algorithms in terms of accuracies is compared for two multisource remote sensing and geographic datasets. In the experiments, the multiple classifiers outperform the single classifiers in terms of overall accuracies.

A new segmentation method based on the morphological characteristic of connected components in images is proposed. Theoretical definitions of morphological leveling and morphological spectrum are used in the formal definition of a morphological characteristic. In multiscale segmentation, this characteristic is formalized through the derivative of the morphological profile. Multiscale segmentation is particularly well suited for complex image scenes such as aerial or fine resolution satellite images, where very thin, enveloped and/or nested regions must be retained. The proposed method performs well in the presence of both low radiometric contrast and relatively low spatial resolution. Those factors may produce a textural effect, a border effect, and ambiguity in the object/background distinction. Segmentation examples for satellite images are given.

Decision level fusion has shown great potential to increase classification accuracy beyond the level reached by individual classifiers. A considerable body of literature exists on identifying optimal ways to combine classifiers. However, the selection of the classifiers to be combined is equally, if not more, crucial if an improvement is to be made for certain classifier combination schemes. Agreement among classifiers can inhibit the gains obtained regardless of the method used to combine them. The level of agreement between different classifiers used in remote sensing is assessed based on statistical measures. A study is performed in which an image is classified by several methods with different degrees of agreement between them. The results are then combined using decision fusion schemes and the increase of accuracy is observed for each combination of the individual classifications.

Multisource classification methods based on neural networks and statistical modeling are considered. For these methods, the individual data sources are at first treated separately and modeled by statistical methods. Then several decision fusion schemes are applied to combine the information from the individual data sources. These schemes include weighted consensus theory where the weights of the individual data sources reflect the reliability of the sources. The weights are optimized in order to improve the combined classification accuracies. Other considered decision fusion schemes are based on two-stage approaches which use voting in the first stage and reject samples if either the majority or all of the classifiers for the data sources do not agree on a classification of a sample. For the second stage, a neural network is used to classify the rejected samples. The proposed methods are applied in the classification of multisource and hyperdimensional data sets, and the results compared to accuracies obtained with conventional classification schemes.

Hybrid classification methods based on consensus from several data sources are considered. Each data source is at first treated separately and modeled using statistical methods. Then weighting mechanisms are used to control the influence of each data source in the combined classification. The weights are optimized in order to improve the combined classification accuracies. Both linear and nonlinear optimization methods are considered and used in classification of two multisource remote sensing and geographic data sets. A nonlinear method which utilizes a neural network gives excellent experimental results. The hybrid statistical/neural method outperforms all other methods in terms of test accuracies in the experiments.

A new type of a neural-network architecture, the parallel consensual neural network (PCNN), is introduced and applied in classification/data fusion of multisource remote sensing and geographic data. The PCNN architecture is based on statistical consensus theory and involves using stage neural networks with transformed input data. The input data are transformed several times and the different transformed data are used as if they were independent inputs. The independent inputs are first classified using the stage neural networks. The output responses from the stage networks are then weighted and combined to make a consensual decision. In this paper, optimization methods are used in order to weight the outputs from the stage networks. Two approaches are proposed to compute the data transforms for the PCNN, one for binary data and another for analog data. The analog approach uses wavelet packets. The experimental results obtained with the proposed approach show that the PCNN outperforms both a conjugate-gradient backpropagation neural network and conventional statistical methods in terms of overall classification accuracy of test data.

Statistical classification methods based on consensus from several data sources are considered. The methods need weighting mechanisms to control the influence of each data source in the combined classification. The weights are optimized in order to improve the combined classification accuracies. Both linear and non-linear methods are considered for the optimization. A non-linear method which utilizes a neural network is proposed and gives excellent results in experiments. Consensus theory optimized with neural networks outperforms all other methods both in terms of training and test accuracies in the experiments.

The processing of Airborne Visible-Infrared Imaging Spectrometer (AVIRIS) data is discussed both in terms of feature extraction and classification. The recently proposed decision boundary feature extraction method is reviewed and then applied in experiments. Results of classifications for AVIRIS data from Iceland 1991 are given with emphasis on geological applications. The classifiers used include neural network methods and statistical approaches. The decision boundary feature extraction method shows excellent performance for these data.>

Consensus theory is adopted as a means of classifying geographic data from multiple sources. The foundations and usefulness of different consensus theoretic methods are discussed in conjunction with pattern recognition. Weight selections for different data sources are considered and modeling of non-Gaussian data is investigated. The application of consensus theory in pattern recognition is tested on two data sets: (1) multisource remote sensing and geographic data, and (2) very-high-dimensional remote sensing data. The results obtained using consensus theoretic methods are found to compare favorably with those obtained using well-known pattern recognition methods. The consensus theoretic methods can be applied in cases where the Gaussian maximum likelihood method cannot. Also, the consensus theoretic methods are computationally less demanding than the Gaussian maximum likelihood method and provide a means for weighting data sources differently.>