Context Handling imbalanced data poses a significant challenge in binary classification tasks across various domains. The low prevalence of rare events or minority classes in such datasets often results in biased models, negatively affecting their predictive performance and reliability. This issue becomes even more critical in extremely imbalanced databases, where the minority class represents <1 % of the total data. Objective A research gap was identified in the literature concerning approaches for binary classification in extremely imbalanced datasets. Therefore, this Systematic Literature Review (SLR) aims to synthesize existing knowledge by providing insights into the applicability and effectiveness of different approaches—categorized into preprocessing techniques, classifiers, and ensemble methods—for addressing extreme class imbalance. Method The development of this SLR followed a rigorous, phased protocol for gathering, reviewing, and synthesizing existing literature based on well-defined selection and quality criteria. Ultimately, 22 articles were selected, restricted to primary (experimental) studies focused exclusively on extremely imbalanced databases across various application domains. Results The findings of this SLR highlight key directions based on summarized, quantitative data and the top-performing approaches identified across experiments, conducted on 52 databases. The use of combined approaches demonstrated superior, performance across multiple evaluation metrics. Particularly notable were, preprocessing techniques paired with ensemble methods—specifically, oversampling techniques combined with the Random Forest (RF) algorithm—which consistently achieved the best performance in extreme imbalance scenarios. Conclusion In conclusion, the adoption of tailored and efficient approaches to address extreme imbalance in binary classification tasks, as presented in this SLR, can support the initial selection of methods based on database characteristics—thereby reducing time consumption, computational resources, and potential rework.

In this study, an end-to-end semantic segmentation method (ConvSegFormer) is proposed by utilizing the multispectral imaging capability of UAVs for images containing multispectral bands, with a special focus on thermal infrared bands. Experimental results show that the use of multispectral images, especially thermal infrared bands, achieves higher segmentation accuracy through spectral information. In addition, the end-to-end deep learning semantic segmentation method can directly learn the complex mapping relationship between image pixels and semantic categories without step-by-step feature extraction and classification, which is more direct and efficient. Finally, the maximum values of Mean Pixel Accuracy (MPA) and Mean Intersection Over Union (MIOU) are 90.35% and 73.87%. In the segmentation task of the wetland area, the maximum values of PA and IOU reached 95.42% and 90.46%. This indicates that the method is effective and feasible in automatically extracting the segmentation of wetlands and other land types.

Remote sensing image captioning is a research domain that aims to automatically generate natural language descriptions of the contents within remote sensed images. Providing accurate depictions of image contents holds great significance for downstream applications such as image retrieval and image understanding. While there is a pressing need for reliable results, current research predominantly focuses on single captioning algorithms, striving to enhance their performance on specific target-oriented datasets. Undoubtedly, this research trajectory is highly important. However, we believe that relying solely on the output of a single captioner may introduce a vulnerability from a robustness standpoint. This concern is particularly relevant in remote sensing, where the scarcity of large-scale datasets can limit the robustness and reliability of resulting algorithms. In this paper, we propose an approach that harnesses the advantages of ensembles to enhance accuracy and reliability in the context of image captioning. Our method introduces a novel technique for utilizing an ensemble of diverse captioning algorithms and automatically selecting the most suitable caption from the set of predictions. By decoupling the description generation and selection phases, this approach enables high flexibility of integration of architecturally different captioning algorithms in the pipeline.

Image captioning (IC) systems are generally based on encoder–decoder architecture where convolutional neural networks (CNNs) are employed to represent an image with discriminative features and recurrent neural networks (RNNs) sequentially generate a sentence description. Even though a lot of effort has been devoted lately to designing reliable IC systems, the task is far from being solved. The generated descriptions can be affected by different errors related to the attributes and the objects present in the scene. Moreover, once an error occurs, it can be propagated in the recurrent layers of the decoder generating non-accurate descriptions. To solve this problem, we propose two postprocessing strategies applied to the generated descriptions to rectify the errors and improve their quality. The proposed postprocessing strategies are based on hidden Markov models (HMMs) and Viterbi algorithm. The proposed postprocessing strategies can be applied to any encoder–decoder IC system. They are applied at test time once the IC system is trained. In particular, we propose to rectify a sentence once it is fully generated (post-generation strategy) or at each time instant of the generation process (in-generation strategy). Experiments conducted on four different IC datasets confirm the promising capabilities of the proposed postprocessing strategies to rectify the output of a simple encoder–decoder by generating more coherent descriptions. The achieved results are competitive and sometimes better than complex IC systems.

Regression analysis has a crucial role in many Earth Ob-servation (EO) applications. The increasing availability and recent development of new computing technologies moti-vate further research to expand the capabilities and enhance the performance of data analysis algorithms. In this paper, the biophysical variable estimation problem is addressed. A novel approach is proposed, which consists in a reformulated Support Vector Regression (SVR) and leverages Quantum Annealing (QA). In particular, the SVR optimization prob-lem is reframed to a Quadratic Unconstrained Binary Opti-mization (QUBO) problem. The algorithm is then tested on the D-Wave Advantage quantum annealer. The experiments presented in this paper show good results, despite current hardware limitations, suggesting that this approach is viable and has great potential.

The increasing availability of quantum computers motivates researching their potential capabilities in enhancing the performance of data analysis algorithms. Similarly, as in other research communities, also in Remote Sensing (RS) it is not yet defined how its applications can benefit from the usage of quantum computing. This paper proposes a formulation of the Support Vector Regression (SVR) algorithm that can be executed by D-Wave quantum computers. Specifically, the SVR is mapped to a Quadratic Unconstrained Binary Optimization (QUBO) problem that is solved with Quantum Annealing (QA). The algorithm is tested on two different types of computing environments offered by D-Wave: The Advantage system, which directly embeds the problem into the Quantum Processing Unit (QPU), and a Hybrid solver that employs both classical and quantum computing resources. For the evaluation, we considered a biophysical variable estimation problem with RS data. The experimental results show that the proposed quantum SVR implementation can achieve comparable or in some cases better results than the classical implementation. This work is one of the first attempts to provide insight into how QA could be exploited and integrated in future RS workflows based on Machine Learning (ML) algorithms.

Recently, vision-language models based on transformers are gaining popularity for joint modeling of visual and textual modalities. In particular, they show impressive results when transferred to several downstream tasks such as zero and few-shot classification. In this paper, we propose a visual question answering (VQA) approach for remote sensing images based on these models. The VQA task attempts to provide answers to image-related questions. While VQA has gained popularity in computer vision, in remote sensing it is not widespread. First, we use the contrastive language image pre-training (CLIP) network for embedding the image patches and question words into a sequence of visual and textual representations. Then, we learn attention mechanisms to capture the intra-and-inter dependencies within and between these representations. Afterward, we generate the final answer by averaging the predictions of two classifiers mounted on the top of the resulting contextual representations. In the experiments, we study the performance of the proposed approach on two datasets acquired with Sentinel-2 and aerial sensors. In particular, we demonstrate that our approach can achieve better results with reduced training size compared to the recent state-of-the-art.

Change detection (CD) is among the most important applications in remote sensing that allows identifying the changes that occurred in a given geographical area across different times. Even though CD systems have seen a lot of progress in RS, their output is either a binary map highlighting the changing area or a semantic change map that indicates the type of change for each pixel. The change maps are often difficult to interpret by end-users and they omit important information such are relationships and attributes of the changed areas. Motivated by the recent advancement of image captioning in the RS community, in this article we propose to describe the changes over bi-temporal images through change sentence descriptions. The aim of this article is to provide a user-friendly interpretation of the occurred changes. To this end, we propose two change captioning (CC) systems that take as input bi-temporal images and generate coherent sentence descriptions of the occurred changes. Convolutional neural networks (CNNs) are used to extract discriminative features from the bi-temporal images and recurrent neural networks (RNNs) or support vector machines (SVMs) are exploited to generate coherent change descriptions. Furthermore, in absence of a CC dataset to test our systems, we propose two new datasets. One is based on very high-resolution RGB images and the other one is based on multispectral RS images. The obtained experimental results show promising capabilities of the proposed systems to generate coherent change descriptions from the bi-temporal images. The datasets are available at the following link: https://disi.unitn.it/~melgani/datasets.html.

Most of the remote sensing image captioning (IC) models are based on encoder–decoder frameworks where a convolutional neural network (CNN) encodes the image information and a recurrent neural network (RNN) decodes the image information into a sentence description. In order to achieve good accuracies, encoder–decoder frameworks relying on RNNs typically require a huge amount of annotated samples. Furthermore, they demand high and expensive computational power in order to have reasonable training and testing time. In this article, we aim to address these issues by introducing a novel decoder that is based on support vector machines (SVMs). In particular, instead of RNNs, we propose a novel network of SVMs to decode the image information into a sentence description. The proposed IC system is particularly interesting when just a limited amount of training samples is available. Experiments conducted on four different IC datasets confirm the promising capability of the proposed IC system to generate descriptions that are highly correlated with the image content. The proposed IC system is characterized by short training and inference times compared to other state-of-the-art models.

Motivated by the good performance recorded for image captioning (IC) techniques in different remote sensing (RS) applications, we propose in this paper a change detection (CD) system based on IC. It aims to the creation of a user-friendly solution that describes, via human-like sentences, the changes detected comparing bi-temporal images acquired over the same geographical area. The model we propose, is based on a convolutional neural network (CNN), and a multimodal recurrent neural network (RNN). Our experiments have been performed combining a set of aerial images with semantic information that we generated to describe the changes observed for different types of objects. This work yielded to encouraging results, evaluated using the BLEU metric.

Recent developments in Quantum Computing (QC) have paved the way for an enhancement of computing capabilities. Quantum Machine Learning (QML) aims at developing Machine Learning (ML) models specifically designed for quantum computers. The availability of the first quantum processors enabled further research, in particular the exploration of possible practical applications of QML algorithms. In this work, quantum formulations of the Support Vector Machine (SVM) are presented. Then, their implementation using existing quantum technologies is discussed and Remote Sensing (RS) image classification is considered for evaluation.

Remote sensing (RS) image captioning (IC) is a novel technique introduced recently in the RS community to enrich the description of very high resolution (VHR) images. The goal of RSIC is to generate a sentence that summarizes the content of an image. In general, RSIC is developed in a supervised way where annotated samples are needed to train the system. However, obtaining annotated samples is costly and time consuming in particular when the labels are sentence descriptions that are very subjective. In order to cope with the problem of having large amounts of training samples in this work we propose an active learning solution for RSIC that selects the most important samples to label and include in the training. Experimental results performed on UCM caption dataset show the promising effectiveness of the proposed active learning strategy.

A recent work on retro-remote sensing (converting ancient text descriptions into images) was proposed using a multilabel encoding scheme in which an input text description is represented by a binary vector indicating the presence or absence of specific objects. However, this kind of encoding disregards information such as object attributes and spatial relationship between multiple objects in a description, resulting in images that do not semantically (fully) conform to the input description. In this letter, we propose an improved text-encoding mechanism that takes into account different levels of information available from an input text. The encoded text is then used as conditional information to guide the image synthesis process using generative adversarial networks (GANs). Besides, we present a modified GAN architecture intending to improve the semantic content of the generated images. Both the qualitative and quantitative results obtained indicate that the proposed method is particularly promising.

With the fast development of remote sensing (RS) technology we are now able to acquire high resolution images. To cope with the new challenges of analyzing such images, a recently introduced tool is RS image captioning (IC). With respect to conventional techniques such as scene classification, RSIC provides more information about an image. It aims to generate a description that summarizes the content of an image. Most of RSIC systems are based on deep learning frameworks (encoder-decoder). The performance of these frameworks strongly depend on the number of annotated samples used during training. In this paper, we propose an alternative RSIC system for a relatively small dataset based on support vector machines (SVMs). A pre-trained CNN is used to extract the image visual features and a network of SVMs is used to generate the descriptions. Experimental results on a RS image archive composed of images acquired by unmanned aerial vehicles (UAV), show that the proposed IC system could be an interesting alternative to deep learning frameworks when only small training samples are available.

In this letter, we discuss unsupervised feature extraction on hyperspectral imagery (HSI) and propose a novel approach based on autoencoder (AE) networks to extract spectral-spatial features from HSI. Our approach takes the data relations into consideration, i.e., the input dependency with adjacent inputs, which the normal AE-based feature extractors often disregard. Specifically, the loss function of the normal AE is modified so as to make pixels share the common features among the neighboring pixels. The process enables the generation of smooth compressed images represented by features provided by the AE. Numerical experiments were conducted on real-world HSI data sets for land cover classification. The results demonstrated that spectral-spatial features extracted by our approach are more discriminative for land cover classification than those done by conventional approaches.

Detecting objects becomes an increasingly important task in very high resolution (VHR) remote sensing imagery analysis. With the development of GPU-computing capability, a growing number of deep convolutional neural networks (CNNs) have been designed to address the object detection challenge. However, compared with CPU, GPU is much more costly. Therefore, GPU-based methods are less attractive in practical applications. In this article, we propose a CPU-based method that is based on convolutional support vector machines (CSVMs) to address the object detection challenge in VHR images. Experiments are conducted on three VHR and two unmanned aerial vehicle (UAV) data sets with very limited training data. Results show that the proposed CSVM achieves competitive performance compared to U-Net which is an efficient CNN-based model designed for small training data sets.

Text-to-image synthesis is a research topic that has not yet been addressed by the remote sensing community. It consists in learning a mapping from text description to image pixels. In this paper, we propose to address this topic for the very first time. More specifically, our objective is to convert ancient text descriptions of geographic areas written by past explorers into an equivalent remote sensing image. To this effect, we rely on generative adversarial networks (GANs) to learn the mapping. GANs aim to represent the distribution of a dataset using weights of a deep neural network, which are trained as an adversarial competition between two networks. We collected ancient texts dating back to 7 BC to train our network and obtained interesting results, which form the basis to highlight future research directions to advance this new topic.

The classification of land-cover classes in remote sensing images can suit a variety of interdisciplinary applications such as the interpretation of natural and man-made processes on the Earth surface. The Convolutional Support Vector Machine (CSVM) network was recently proposed as binary classifier for the detection of objects in Unmanned Aerial Vehicle (UAV) images. The training phase of the CSVM is based on convolutional layers that learn the kernel weights via a set of linear Support Vector Machines (SVMs). This paper proposes the Multi-scale Convolutional Support Vector Machine (MCSVM) network, that is an ensemble of CSVM classifiers which process patches of different spatial sizes and can deal with multi-class classification problems. The experiments are carried out on the EuroSAT Sentinel-2 dataset and the results are compared to the one obtained with recent transfer learning approaches based on pre-trained Convolutional Neural Networks (CNNs).

In this article, we propose a new semi-supervised method to detect changes occurring in a geographical area after a major event such as war, an earthquake or flood. The detection is made by processing a pair of bi-temporal remotely sensed images of the area under consideration. The proposed method adopts a patch-based approach, where successive pairs of patches from the input images are compared using a deep machine learning method trained with augmented data. Our main contribution consists of proposing an approach for generating a training dataset from unlabeled pair of input images. The genuine training patch-pairs are directly generated from the transformed maps of the image taken before the event, while the impostor patch-pairs are generated by pairing the image taken before the event with any images, from the Internet, with textures that resemble the change shown in the image taken after the event. Several experiments were conducted on pairs of images related to five major events. The obtained subjective results demonstrate the effectiveness of the proposed method.

This paper presents a novel remote sensing (RS) image retrieval system that is defined based on generation and exploitation of textual descriptions that model the content of RS images. The proposed RS image retrieval system is composed of three main steps. The first one generates textual descriptions of the content of the RS images combining a convolutional neural network (CNN) and a recurrent neural network (RNN) to extract the features of the images and to generate the descriptions of their content, respectively. The second step encodes the semantic content of the generated descriptions using word embedding techniques able to produce semantically rich word vectors. The third step retrieves the most similar images with respect to the query image by measuring the similarity between the encoded generated textual descriptions of the query image and those of the archive. Experimental results on RS image archive composed of RS images acquired by unmanned aerial vehicles (UAVs) are reported and discussed.

Semantic segmentation is understanding images at pixel level, which becomes increasingly vital in unmanned aerial vehicles (UAV) imagery classification tasks. With the powerful calculating ability of GPU, a growing number of deep convolutional neural networks (DCNNs) are designed to address semantic segmentation challenges. However, compared with CPU, GPU is much more costly, and GPU relies on powerful supplementary equipments to support it. Therefore, GPU-based methods are hard to carry out in practical applications. In this study, we propose a CPU-based method named fully convolutional support vector machine (FCSVM) to address the semantic segmentation challenge in UAV images. On the one hand, we adopt the SVM kernel from the original convolutional SVM networks (CSVM), which completes classification at image level. On the other hand, FCSVM consists of two processes which are a compressed process and a extensive process. In the compressed process, the FCSVM has convolutional layers and reduction layers. In the extensive process, the FCSVM has convolutional layers and upsampling layers. This structure allows FCSVM to classify images at pixel level using limited number of training data. The experiments are implemented on one UAV dataset with very little training data. The result shows our FCSVM achieves competitive performance compared to modern state-of-the-art semantic segmentation methods.

In this letter, a low-cost semisupervised domain adaptation technique has been proposed using a two-level fusion of artificial neural networks. The proposed technique has been aimed to solve the problem of sample selection bias using a one-time collection of a few patterns from the target domain under semisupervised framework. To assess the effectiveness, experiments are conducted on the two source-target data sets acquired over India. The results are found to be encouraging.

Recent trends in image classification focus on training deep neural networks that require having a large amount of training images related to the considered task. However, obtaining enough labeled image samples is often time-consuming and expensive. An alternative solution proposed is to transfer the knowledge learned while solving one problem to another but related problem, also called transfer learning. Domain adaptation is a type of transfer learning that deals with learning a model that performs well on two datasets that have different (but somehow correlated) data distributions. In this work, we present a new domain adaptation method based on generative adversarial networks (GANs) in the context of aerial image classification. Experimental results obtained on two datasets for a single object scenario show that the proposed method is particularly promising.

Domain invariant representation learning is one of the domain adaptation techniques, which aims at learning a representation that is not affected by a possible shift in the data distribution between training and test sets. Traditional domain adaptation methods proposed in the literature focus on first learning a domain invariant transformation from the input space to a new representation space and then training a classifier in the new space. Alternatively, in this paper, we propose an adversarial domain adaptation technique that combines representation learning, domain adaptation and classifier learning in a single training process. Moreover, the proposed method performs the domain adaptation by using the Wasserstein metrics to minimize the domain discrepancy. We applied the proposed method on a hyperspectral image classification problem and the results obtained show the effectiveness of the method.

In this letter, we formulate the multilabeling classification problem of unmanned aerial vehicle (UAV) imagery within a conditional random field (CRF) framework with the aim of exploiting simultaneously spatial contextual information and cross-correlation between labels. The pipeline of the framework consists of two main phases. First, the considered input UAV image is subdivided into a grid of tiles, which are processed thanks to an opportune representation and a multilayer perceptron classifier providing thus tile-wise multilabel prediction probabilities. In the second phase, a multilabel CRF model is applied to integrate spatial correlation between adjacent tiles and the correlation between labels within the same tile, with the objective to improve iteratively the multilabel classification map associated with the considered input UAV image. Experimental results achieved on two different UAV image data sets are reported and discussed.

Nowadays, unmanned aerial vehicles (UAVs) are viewed as effective acquisition platforms for several civilian applications. They can acquire images with an extremely high level of spatial detail compared to standard remote sensing platforms. However, these images are highly affected by illumination, rotation, and scale changes, which further increases the complexity of analysis compared to those obtained using standard remote sensing platforms. In this paper, we introduce a novel convolutional support vector machine (CSVM) network for the analysis of this type of imagery. Basically, the CSVM network is based on several alternating convolutional and reduction layers ended by a linear SVM classification layer. The convolutional layers in CSVM rely on a set of linear SVMs as filter banks for feature map generation. During the learning phase, the weights of the SVM filters are computed through a forward supervised learning strategy unlike the backpropagation algorithm widely used in standard convolutional neural networks (CNNs). This makes the proposed CSVM particularly suitable for detecting problems characterized by very limited training sample availability. The experiments carried out on two UAV data sets related to vehicles and solar-panel detection issues, with a 2-cm resolution, confirm the promising capability of the proposed CSVM network compared to recent state-of-the-art solutions based on pretrained CNNs.

We describe a novel multilabel classification approach based on a support vector machine (SVM) for the extremely high-resolution remote sensing images. Its underlying ideas consist to: 1) exploit inter-label relationships by means of a structured SVM and 2) incorporate spatial contextual information by adding to the cost function a term that encourages spatial smoothness into the structural SVM optimization process. The resulting formulation appears as an extension of the traditional SVM learning, in which our proposed model integrates the output structure and spatial information simultaneously during the training. Numerical experiments conducted on two different UAV- and airborne-acquired sets of images show the interesting properties of the proposed model, in particular, in terms of classification accuracy.

The accurate reconstruction of areas obscured by clouds is among the most challenging topics for the remote sensing community since a significant percentage of images archived throughout the world are affected by cloud covers which make them not fully exploitable. The purpose of this paper is to propose new methods to recover missing data in multispectral images due to the presence of clouds by relying on a formulation based on an autoencoder (AE) neural network. We suppose that clouds are opaque and their detection is performed by dedicated algorithms. The AE in our methods aims at modeling the relationship between a given cloud-free image (source image) and a cloud-contaminated image (target image). In particular, two strategies are developed: the first one performs the mapping at a pixel level while the second one at a patch level to take profit from spatial contextual information. Moreover, in order to fix the problem of the hidden layer size, a new solution combining the minimum descriptive length criterion and a Pareto-like selection procedure is introduced. The results of experiments conducted on three different data sets are reported and discussed together with a comparison with reference techniques.

In this article, we address the problem of recognizing an event from a single related picture. Given the large number of event classes and the limited information contained in a single shot, the problem is known to be particularly hard. To achieve a reliable detection, we propose a combination of multiple classifiers, and we compare three alternative strategies to fuse the results of each classifier, namely: (i) induced order weighted averaging operators, (ii) genetic algorithms, and (iii) particle swarm optimization. Each method is aimed at determining the optimal weights to be assigned to the decision scores yielded by different deep models, according to the relevant optimization strategy. Experimental tests have been performed on three event recognition datasets, evaluating the performance of various deep models, both alone and selectively combined. Experimental results demonstrate that the proposed approach outperforms traditional multiple classifier solutions based on uniform weighting, and outperforms recent state-of-the-art approaches.

This paper proposes a novel two-stage framework for event recognition in still images. First, for a generic event image, deep features, obtained via different pre-trained models, are fed into an ensemble of classifiers, whose posterior classification probabilities are thereafter fused by means of an order-induced scheme, which penalizes the yielded scores according to their confidence in classifying the image at hand, and then averages them. Second, we combine the fusion results with a reverse matching paradigm in order to draw the final output of our proposed pipeline. We evaluate our approach on three challenging datasets and we show that better results can be attained, advancing recent leading works.

We propose in this work new strategies to reconstruct areas obscured by opaque clouds in multispectral images. They are based on an autoencoder (AE) neural network which opportunely models the relationships between a given source (cloud-free) image and a target (cloud-contaminated) image. The first strategy estimates the relationship model at a pixel level while the second one operates at a patch level in order profit from spatial contextual information. Experimental results obtained on FORMOSAT-2 images are reported and discussed together with a comparison with reference techniques.

This paper describes the use of Unmanned Aerial Vehicle (UAV) technology to fight apple scab. Specifically, it shows how it is possible to improve the scab risk evaluation basing on the actual apple leaves development status, yielded from UAV images, as input to the infection model. For this purpose, we introduce a new index, called Leaf Development Index (LDI), which is evaluated during the main growth phases of the apple trees using an UAV equipped with both multispectral and thermal sensors. Preliminary results are reported and discussed.

In this letter, we face the problem of multilabeling unmanned aerial vehicle (UAV) imagery, typically characterized by a high level of information content, by proposing a novel method based on convolutional neural networks. These are exploited as a means to yield a powerful description of the query image, which is analyzed after subdividing it into a grid of tiles. The multilabel classification task of each tile is performed by the combination of a radial basis function neural network and a multilabeling layer (ML) composed of customized thresholding operations. Experiments conducted on two different UAV image data sets demonstrate the promising capability of the proposed method compared to the state of the art, at the expense of a higher but still contained computation time.

In this paper, we present a domain adaptation network to deal with classification scenarios subjected to the data shift problem (i.e., labeled and unlabeled images acquired with different sensors and over completely different geographical areas). We rely on the power of pretrained convolutional neural networks (CNNs) to generate an initial feature representation of the labeled and unlabeled images under analysis, referred as source and target domains, respectively. Then we feed the resulting features to an extra network placed on the top of the pretrained CNN for further learning. During the fine-tuning phase, we learn the weights of this network by jointly minimizing three regularization terms, which are: 1) the cross-entropy error on the labeled source data; 2) the maximum mean discrepancy between the source and target data distributions; and 3) the geometrical structure of the target data. Furthermore, to obtain robust hidden representations we propose a mini-batch gradient-based optimization method with a dynamic sample size for the local alignment of the source and target distributions. To validate the method, in the experiments we use the University of California Merced data set and a new multisensor data set acquired over several regions of the Kingdom of Saudi Arabia. The experiments show that: 1) pretrained CNNs offer an interesting solution for image classification compared to state-of-the-art methods; 2) their performances can be degraded when dealing with data sets subjected to the data shift problem; and 3) how the proposed approach represents a promising solution for effectively handling this issue.

In recent years, unmanned aerial vehicles (UAVs) have been widely used for civilian remote sensing applications. One of them is to assess damages due to man-made or natural disasters and search for bodies in the debris. In this work, we propose to support avalanche search and rescue (SAR) operation with UAVs. The image acquired by the UAV is processed through a pre-trained convolutional neural network (CNN) to extract discriminative features. A linear support vector machine (SVM) is integrated at the top of the CNN to detect objects of interest. Experimental results show encouraging detection performance at a reasonable processing time.

Wave-based estimation of coastal bathymetry from satellite images is challenging because it often implies relying on ancillary data. Moreover, spatial resolution remains coarse compared to the image resolution because state-of-the-art wave analysis techniques decompose images into large-size patches. The method proposed here is based on adaptive wave tracing where the patch size depends on the wavelength observed. Waves are tracked from the open sea to the coast. When the waves approach the coast, their wavelength decreases and so does the patch size. Combining this method with linear wave theory leads to improved water depth estimation. Furthermore, with wave tracing we are also able to compute the wave period, allowing the estimation of water depth without resorting to ancillary data.

In this paper, we present a multilabel classification method for images acquired by means of Unmanned Ariel Vehicles (UAV) over urban areas. Due to the fact that UAV-grabbed images are characterized by extremely high spatial resolution, usual recognition schemes (such as traditional satellite or airborne based images) are likely to fail. In this work, tile-based multilabel classification framework is adopted to overcome such issue. In particular, a given UAV-shot image is first subdivided into a grid of equal tiles. Next, deep neural network-induced features are extracted from each tile and then fed into a radial basis function neural network classifier in order to infer the corresponding object list. We apply a refinement step at the top of the complete deep network architecture to boost the classification results. The proposed method was evaluated on a dataset acquired over the city of Trento, Italy with an hexacopter UAV. Superior classification rates have been scored with respect to the state-of-the-art.

In this letter, we propose a novel three-layer convex network termed as 3CN for domain adaptation in multitemporal very high resolution (VHR) remote sensing images. 3CN is composed of three main layers: 1) mapping source training samples to the target domain via a special single-layer feedforward neural network called extreme learning machine (ELM); 2) target image classification via ELM too; and 3) spatial regularization via the random-walker algorithm, which models the target image as a lattice graph and then minimizes an energy functional. This network is convex because all three layers have closed-form solutions. In the preprocessing step, we use scale-invariant feature transform to extract a set of matching key points called inliers from source and target images. Then, these inliers are used by layer 1 of 3CN to spectrally map the source training samples to the target domain. Next, in layer 2, we use the mapped training set to classify the target image. In layer 3, we exploit the spatial contextual information in the target image to reduce noise and generate an improved classification map. In the final step, we iteratively fine-tune the network to increase its discrimination ability and reduce the shift between the target and source domains. In the experiments, we report and discuss the results of the proposed method on two data sets of VHR image pairs acquired by IKONOS-2 and GeoEye-1.

We propose a method based on combining wave tracing and linear wave theory for the estimation of wave period and bathymetry in coastal areas from satellite images. The method depends on several parameters for which we provide ranges of variations adapted to the instrument. Experimental results are conducted on several sites located around the Hawaiian island of Oahu, using 13 Landsat-8 images. Results show that wave period estimations are compatible with the wave buoy measurements in all cases. In addition, bathymetry estimation results show a standard deviation of less than 15% of the observed depth out of the surf zone until 20 m for sites with a direct exposure to the swell and with an absence of clouds. The proposed method, which does not rely on ancillary data, represents a promising tool for bathymetry estimation using satellite images in which waves are present.

In this paper, we present a fusion method contextualized within a land use classification framework. At first, feature vectors are extracted from all the color channels of the given test image. Then, the generated vectors are recovered over a bunch of training feature vectors extracted from training images. The resulting reconstruction residuals feed a fusion mechanism to further compose a final residual that serves for inferring the final decision of the class pertaining to the test image. Validated on a benchmark dataset, the presented method shows to promote drastic improvements over using only one single spectral channel. Furthermore, encouraging gains have been recorded with respect to reference works.

In order to describe images acquired with unmanned aerial vehicles (UAV), we introduce in this paper a multilabeling classification method. It starts by subdividing the original UAV image into a grid of tiles which are then analyzed separately. From each tile, a signature which encodes texture information is extracted and compared with the signatures of the tiles belonging to a pre-built training dictionary in order to acquire the binary multilabel vector of the most similar tile. In order to represent and match the tiles, we exploit a well-known texture operator and a common distance measure, respectively. Promising experimental results, in particular for some classes of objects, are obtained on real UAV images acquired over urban areas.

In this letter, we propose an efficient multiclass active learning (AL) method for remote sensing image classification. We fuse the capabilities of an extreme learning machine (ELM) classifier and graph-based optimization methods to boost the classification accuracy while minimizing the user interaction. First, we use the ELM to generate an initial label estimation of the unlabeled image pixels. Then, we optimize a graph-based functional energy that integrates the ELM outputs as an initial estimation of the image structure. As for the ELM, the solution to this multiclass optimization problem leads to a system of linear equations. Due to the sparse Laplacian matrix built from the lattice graph defined on the image pixels, the optimization problem is solved in a linear time. In the experiments, we report and discuss the results of the proposed AL method on two very high resolution images acquired by IKONOS-2 and GoeEye-1, as well as the well-known Pavia University hyperspectral image.

In this letter, we formulate a land-use (LU) classification problem within a compressive sensing (CS) fusion framework. CS aims at providing a compact representation form after a given query image has been processed with an opportune feature extraction type. In particular, residuals are generated from the image reconstruction with dictionaries associated with the available set of possible LUs and gathered to form a single-feature image pattern. The patterns obtained from different types of features are then fused to provide the final LU estimate. Two simple fusion strategies are adopted for such purpose. As demonstrated by experiments ran on the basis of a public benchmark database, the proposed method can achieve substantial classification accuracy gains over reference methods.

The aim of this letter is twofold. First, we assess the compressive sensing (CS) approach as a classification tool for multispectral remote sensing images, assuming severe scarcity of training samples (at most, ten for each class). Then, we propose a new strategy to perform domain adaptation using a CS approach for classifying images at large spatial scales (continental mapping). In particular, the “most confusing” training samples in the target domain are collected by exploiting plenty of training samples available in the source domain under the transfer learning framework. For assessing the proposed method, experiments are performed on three remotely sensed images captured by the Landsat 8 satellite in different regions of India. Results obtained using the proposed approach are found to be promising.

This paper introduces a new portable camera-based method for helping blind people to recognize indoor objects. Unlike state-of-the-art techniques, which typically perform the recognition task by limiting it to a single predefined class of objects, we propose here a completely different alternative scheme, defined as coarse description. It aims at expanding the recognition task to multiple objects and, at the same time, keeping the processing time under control by sacrificing some information details. The benefit is to increment the awareness and the perception of a blind person to his direct contextual environment. The coarse description issue is addressed via two image multilabeling strategies which differ in the way image similarity is computed. The first one makes use of the Euclidean distance measure, while the second one relies on a semantic similarity measure modeled by means of Gaussian process estimation. To achieve fast computation capability, both strategies rely on a compact image representation based on compressive sensing. The proposed methodology was assessed on two indoor datasets representing different indoor environments. Encouraging results were achieved in terms of both accuracy and processing time.

This paper presents a novel method to “coarsely” describe extremely high-resolution (EHR) images acquired by means of unmanned aerial vehicles (UAVs) over urban areas. Standard image analysis approaches cannot be directly exploited for the automatic description of UAV images due to their EHR. For this reason, we propose an alternative approach that consists first in the subdivision of the original UAV image in a grid of tiles. Then, each tile is compared with a library of training tiles to inherit the binary multilabel vector of the most similar training tile. This vector conveys a list of classes likely present in the considered tile. Our multiclass tile-based approach needs the definition of two main ingredients: 1) a suitable tile-representation strategy; and 2) a tile-to-tile matching operation. Various tile-representation and matching strategies are investigated. In particular, we present three global representation strategies, which process each tile as a whole and two point-based strategies that exploit points of interest within the considered tile. Regarding the matching strategies, two simple measures of distance, namely, the Euclidean and the chi-squared histogram distances, are explored. Interesting experimental results conducted on a rich set of real UAV images acquired over an urban area are reported and discussed.

Unmanned Aerial Vehicles (UAVs) exhibit an interesting operational flexibility considering that they can be used when and where it is necessary. These cutting-edge acquisition systems are able to fly very close to the ground allowing the acquisition of images in which objects are described by a very high level of detail. In this work, a method to detect moving objects and estimate their speed is presented. The method starts with the registration of two successive images belonging to a sequence acquired by means of a UAV. Then, the method proceeds with the identification of moving objects thanks to an opportune comparison of the couple of images. In the last step, to each moving object a speed estimate is assigned by exploiting the corresponding shift of spatial image coordinates. Experimental results have been conducted on a real UAV sequence of images. They show that the proposed method allows providing interesting estimation performances.

The monitoring of possible damages on industrial plants from aerial images represents a challenging task. In this work, we present a methodology to monitor the changes due to corrosion damages on industrial plants by using Unmanned Aerial Vehicle (UAV) images. First, a couple of images acquired at two different times is considered and aligned to each other through a geometric transformation. Then, the possible changes are highlighted in both images by exploiting a simple automatic thresholding technique based on the assumption that damages have usually different aspects with respect to the surrounding structures. At the end, the images are compared to obtain an estimation of the damage growth. The methodology has been tested on extremely high resolution images obtained with different acquisition conditions. The achieved results demonstrate the precision of the method and suggest the possible use of such a technique in practical applications.

In this letter, we show how active learning can be particularly promising for classifying remote sensing images at large scales. The classification model constructed on samples extracted from a limited region of the image, called source domain, exhibits generally poor accuracies when used to predict the samples of a different region, called target domain, due to possible changes in class distributions throughout the image. To alleviate this problem, we suggest selecting and labeling additional samples from the new domain in order to improve generalization capabilities of the model. We propose to implement an initialization strategy based on clustering before applying the traditional active learning method in order to cope with distribution changes and better explore the feature space of the target domain. Experiments on a MODIS dataset for the generation of a land-cover map at European scale show good capabilities of the proposed approach for this purpose.

Recently, a new machine learning approach that is termed as the extreme learning machine (ELM) has been introduced in the literature. This approach is characterized by a unified formulation for regression, binary, and multiclass classification problems, and the related solution is given in an analytical compact form. In this letter, we propose an efficient classification method for hyperspectral images based on this machine learning approach. To address the model selection issue that is associated with the ELM, we develop an automatic-solution-based differential evolution (DE). This simple yet powerful evolutionary optimization algorithm uses cross-validation accuracy as a performance indicator for determining the optimal ELM parameters. Experimental results obtained from four benchmark hyperspectral data sets confirm the attractive properties of the proposed DE-ELM method in terms of classification accuracy and computation time.

This paper presents a solution to solve the car detection and counting problem in images acquired by means of unmanned aerial vehicles (UAVs). UAV images are characterized by a very high spatial resolution (order of few centimeters), and consequently by an extremely high level of details which calls for appropriate automatic analysis methods. The proposed method starts with a screening step of asphalted zones in order to restrict the areas where to detect cars and thus to reduce false alarms. Then, it performs a feature extraction process based on scalar invariant feature transform thanks to which a set of keypoints is identified in the considered image and opportunely described. Successively, it discriminates between keypoints assigned to cars and all the others, by means of a support vector machine classifier. The last step of our method is focused on the grouping of the keypoints belonging to the same car in order to get a “one keypoint-one car” relationship. Finally, the number of cars present in the scene is given by the number of final keypoints identified. The experimental results obtained on a real UAV scene characterized by a spatial resolution of 2 cm show that the proposed method exhibits a promising car counting accuracy.

This paper presents a new method for the automatic detection of cars in unmanned aerial vehicle (UAV) images acquired over urban contexts. UAV images are characterized by an extremely high spatial resolution, which makes the detection of cars particularly challenging. The proposed method starts with a screening operation in which the asphalted areas are identified in order to make the car detection process faster and more robust. Subsequently, filtering operations in the horizontal and vertical directions are performed to extract histogram-of-gradient features and to yield a preliminary detection of cars after the computation of a similarity measure with a catalog of cars used as reference. Three different strategies for computing the similarity are investigated. Successively, for the image points identified as potential cars, an orientation value is computed by searching for the highest similarity value in 36 possible directions. The last step is devoted to the merging of the points which belong to the same car because it is likely that a car is identified by more than one point due to the extremely high resolution of UAV images. As outcomes, the proposed method provides the number of cars in the image, as well as the position and orientation for each of them. Interesting experimental results, conducted on a set of real UAV images acquired over an urban area, are presented and discussed.

In the last few years, active learning has been gaining growing interest in the remote sensing community in optimizing the process of training sample collection for supervised image classification. Current strategies formulate the active learning problem in the spectral domain only. However, remote sensing images are intrinsically defined both in the spectral and spatial domains. In this paper, we explore this fact by proposing a new active learning approach for support vector machine classification. In particular, we suggest combining spectral and spatial information directly in the iterative process of sample selection. For this purpose, three criteria are proposed to favor the selection of samples distant from the samples already composing the current training set. In the first strategy, the Euclidean distances in the spatial domain from the training samples are explicitly computed, whereas the second one is based on the Parzen window method in the spatial domain. Finally, the last criterion involves the concept of spatial entropy. Experiments on two very high resolution images show the effectiveness of regularization in spatial domain for active learning purposes.

In order to reconstruct missing data in very high resolution (VHR) multispectral images, several methodologies were proposed in the literature. However, missing data reconstruction still represents a complex image processing challenge to solve. A recent possibility comes from the compressive sensing (CS) theory, in particular the basis pursuit (BP) concept, which allows to find sparse signal representations in underdetermined linear equation systems. In this work, we propose an alternative selection method for the reconstruction of images adopting a histogram matching (HM) strategy. Experiments were conducted on FORMOSAT-2 images. The reported results include a simulation study and a comparison with a state-of-the-art technique for cloud removal.

Reconstructing missing data in very high resolution (VHR) multispectral images represents a complex image processing challenge. In this paper, we propose a new method for the reconstruction of areas obscured by clouds. It is based on compressive sensing (CS) theory, which allows to find sparse signal representations in underdetermined linear equation systems. Here we propose a novel implementation which exploits genetic algorithms (GAs) and a new strategy for the selection of atoms belonging to the dictionary. To illustrate the performances of the proposed method, a thorough experimental analysis on FORMOSAT-2 images is reported and discussed. It includes a simulation study and a comparison with a state-of-the-art technique for cloud removal.

Unmanned aerial vehicles (UAV) are among the fast growing remote sensing technologies in these last few years. This is mainly because UAVs allow acquiring images characterized by an extremely high spatial resolution and they exhibit an interesting operational flexibility. Taking advantage from these unique characteristics can help in addressing problems typical of the civilian contexts. In particular, identifying and monitoring cars inside an urban environment is viewed as an important and challenging problem because it could limit issues related to traffic jams and pollution. In this work, we investigate the use of several detectors and descriptors to find the best representation of cars for their classification in UAV images. Experimental results on real UAV images are reported and discussed.

Mathematical morphology has shown to be an effective tool to extract spatial information for remote-sensing image classification. Its application is performed by means of a structuring element (SE), whose shape and size play a fundamental role for appropriately extracting structures in complex regions such as urban areas. In this letter, we propose a novel method, which automatically tailors both the shape and the size of the SE according to the considered classification task. For this purpose, the SE design is formulated as an optimization problem within a particle swarm optimization framework. The experiments conducted on two real images suggest that better accuracies can be achieved with respect to the common procedure for finding the best regular SE, which, so far, is heuristically done.

Over the past few years, the reconstruction of missing data due to the presence of clouds received an important attention. Applying region-based inpainting strategies or conventional regression methods, such as support vector (SV) machine regression, may not be the optimal way. In this letter, we propose new combinations of kernel functions with which we obtain a better reconstruction. In particular, in the regression, we add to the radiometric information, i.e., the position information of the pixels in the image. For each kind of information adopted in the regression, a specific kernel is selected and adapted. Adopting this new kernel combination in a SV regression (SVR) comes out that only few SVs are needed to reconstruct a missing area. This means that we also perform a compression in the number of values needed for a good reconstruction. We illustrate the proposed approaches through some simulations on FORMOSAT-2 multitemporal images.

The intent of this paper is to propose new methods for the reconstruction of areas obscured by clouds. They are based on compressive sensing (CS) theory, which allows finding sparse signal representations in underdetermined linear equation systems. In particular, two common CS solutions are adopted for our reconstruction problem: the basis pursuit and the orthogonal matching pursuit methods. A novel alternative CS solution is also proposed through a formulation within a multiobjective genetic optimization scheme. To illustrate the performances of the proposed methods, a thorough experimental analysis on FORMOsa SATellite-2 and Satellite Pour l'Observation de la Terre-5 multispectral images is reported and discussed. It includes a detailed simulation study that aims at assessing the accuracy of the methods in different qualitative and quantitative cloud-contamination conditions. Compared with state-of-the-art techniques for cloud removal, the proposed methods show a clear superiority, which makes them a promising tool in cleaning images in the presence of clouds.

Very high resolution (VHR) images are appreciated for their high-level details, which significantly increase their application potential. However, typically, VHR images are affected by the presence of shadows. An attempt solution to the problem of shadows is to restore shadow-contaminated regions by compensating the value of shaded pixels. Unfortunately, it may happen that not all shadow areas are possible to restore. In this paper, we propose different criteria useful to help in understanding a priori if it is possible or not to reconstruct a specific shadow area. An ideal reconstructability criterion should not tolerate that an unreconstructable shadow area is assigned as reconstructable and, at the same time, should maximize the probability of detection of reconstructable areas. Several evaluation criteria working at the pixel and textural levels are presented. Furthermore, in order to select the best criteria, a fuzzy logic combination of the criteria is explored. A thorough experimental analysis is reported and discussed. It leads to the definition of a final global index based on the fusion of two single criteria, which are the Kullback–Leibler divergence and the angular second-moment difference.

In the remote sensing field, ground-truth design for collecting training samples represents a tricky and critical problem since it has a direct impact on most of the subsequent image processing and analysis steps. In this paper, we propose a novel framework for assisting a human user in designing ground-truth by photointerpretation for optical remote sensing image classification. The proposed approach is (almost) completely automatic and comprehensive since it aims at assisting the human user from the first to the last step of the process. It is based on unsupervised methods of segmentation and clustering, in order to investigate both the spatial and the spectral information in the process of ground-truth design. The resulting ground-truth is classifier-free and can be further improved by making it classifier-driven through an active learning process. To validate the proposed framework, an experimental study was conducted on very high spatial resolution and hyperspectral images acquired by the IKONOS and the Reflective Optics System Imaging Spectrometer sensors, respectively. The obtained results show the usefulness and effectiveness of the proposed approach.

Reconstructing missing data in very high resolution (VHR) multispectral images represents a complex image processing challenge. In this paper, we propose a new method for the reconstruction of areas obscured by clouds. It is based on compressive sensing (CS) theory, which allows to find sparse signal representations in underdetermined linear equation systems. In particular a common CS solution is adopted for our reconstruction problem: the orthogonal matching pursuit (OMP) method. To illustrate the performances of the proposed method, a through experimental analysis on FORMOSAT-2 multispectral images is reported and discussed. It includes a simulation study and a comparison with a state-of-the-art technique for cloud removal.

A typical issue encountered in very high resolution (VHR) optical imagery is the undesired presence of shadows. Their compensation by means of appropriate techniques is however not always successful. In this work, we present different criteria, which aim at understanding a priori if it is possible or not to reconstruct a specific shadow area. They are based on a quantitative comparison between the histograms characterizing the shadow and non-shadow areas of the same typology of ground cover. A thorough experimental analysis performed on three different images is reported and discussed.

In the last years, the advent of unmanned aerial vehicles (UAVs) for civilian remote sensing purposes has generated a lot of interest because of the various new applications they can offer. One of them is represented by the automatic detection and counting of cars. In this paper, we propose a novel car detection method. It starts with a feature extraction process based on scalar invariant feature transform (SIFT) thanks to which a set of keypoints is identified in the considered image and opportunely described. Successively, the process discriminates between keypoints assigned to cars and those associated with all remaining objects by means of a support vector machine (SVM) classifier. Experimental results have been conducted on a real UAV scene. They show how the proposed method allows providing interesting detection performances.

In the remote sensing field, classification of images at large scale represents a very important problem. Most of the proposed classification strategies are based on supervised methods, which can give excellent performances, but depend strongly on the training samples used to construct the classification model. In particular, they can fail if such samples are not representative of the distributions associated with the classes. This problem is critical in a large scale scenario, in which the training samples acquired from a limited region of the image, called source domain, are not representative for classifying samples extracted from a different region, called target domain. In this work, we propose to alleviate this problem by adopting an active learning approach, in which few additional samples are selected and labeled from the new domain in order to improve generalization capabilities of the model. In particular, we suggest implementing an initialization strategy before applying the traditional active learning process. The proposed approach is validated experimentally on a MODIS data set for the discrimination between vegetation and non-vegetation areas at European scale.

This paper proposes a novel semisupervised regression framework for estimating chlorophyll concentrations in subsurface waters from remotely sensed imagery. This framework integrates multiobjective optimization and Gaussian processes (GPs) for boosting the accuracy of the estimation process when conditioned by limited labeled-sample availability. To this end, the labeled samples are exploited in conjunction with unlabeled ones (available at zero cost from the image under analysis) for learning the regression model. The estimation of the target of these unlabeled samples is handled by the simultaneous optimization of two different criteria expressing the generalization capabilities of the GP estimator. The first is the empirical risk quantified in terms of the mean square error measure, and the second is the log marginal likelihood, which merges two terms expressing the model complexity and the data fit capability, respectively. In order to alleviate the computational burden and, possibly, to improve the estimation process accuracy, two different selection strategies of unlabeled samples are compared to the simple random-sampling procedure. They are based on the estimated variance provided by the GP estimator and the differential entropy measure, respectively. Experimental results obtained on simulated and real data sets are reported and discussed.

The presence of shadows in very high resolution (VHR) images can represent a serious obstacle for their full exploitation. This paper proposes to face this problem as a whole through the proposal of a complete processing chain, which relies on various advanced image processing and pattern recognition tools. The first key point of the chain is that shadow areas are not only detected but also classified to allow their customized compensation. The detection and classification tasks are implemented by means of the state-of-the-art support vector machine approach. A quality check mechanism is integrated in order to reduce subsequent misreconstruction problems. The reconstruction is based on a linear regression method to compensate shadow regions by adjusting the intensities of the shaded pixels according to the statistical characteristics of the corresponding nonshadow regions. Moreover, borders are explicitly handled by making use of adaptive morphological filters and linear interpolation for the prevention of possible border artifacts in the reconstructed image. Experimental results obtained on three VHR images representing different shadow conditions are reported, discussed, and compared with two other reconstruction techniques.

In this paper, we face the problem of collecting training samples for regression problems under an active learning perspective. In particular, we propose various active learning strategies specifically developed for regression approaches based on Gaussian processes (GPs) and support vector machines (SVMs). For GP regression, the first two strategies are based on the idea of adding samples that are dissimilar from the current training samples in terms of covariance measure, while the third one uses a pool of regressors in order to select the samples with the greater disagreements between the different regressors. Finally, the last strategy exploits an intrinsic GP regression outcome to pick up the most difficult and hence interesting samples to label. For SVM regression, the method based on the pool of regressors and two additional strategies based on the selection of the samples distant from the current support vectors in the kernel-induced feature space are proposed. The experimental results obtained on simulated and real data sets show that the proposed strategies exhibit a good capability to select samples that are significant for the regression process, thus opening the way to the active learning approach for remote-sensing regression problems.

Reconstructing missing data in very high resolution (VHR) multispectral images represents a complex image processing challenge. In this paper, we face this problem through a novel solution based on inpainting. Inpainting is a technique to reconstruct missing regions in a given image by propagating the spectro-geometrical information retrieved from the remaining parts of the image. Our proposed solution relies on the idea to formulate the inpainting process under a multiresolution processing scheme. Experiments conducted on a VHR image are reported and discussed.

In this work, we propose a framework to help in the design of the ground-truth for the classification of remote sensing images. It consists first to segment the considered image by means of a level set method and then to extract the segments characterized by the largest numbers of pixels. Afterward, the selected segments are labeled by a human user. Experimental results obtained on a very high resolution image show encouraging performances of the proposed framework.

In this work, we face the problem of training sample collection for the estimation of biophysical parameters by adopting the active learning approach. In particular, we propose two active learning strategies specifically developed for Gaussian Process (GP) regression. The first one is based on adding samples that are distant from the current training samples in the kernel space while the second one exploits an intrinsic GP regression outcome to pick up the most difficult samples. Experiments on simulated and real data sets show the effectiveness of active selection of training samples for regression problems.

Active learning process represents an interesting solution to the problem of training sample collection for the classification of remote sensing images. In this work, we propose a criterion based on the spatial information that can be used in combination with a spectral criterion in order to improve the selection of training samples. Experimental results obtained on a very high resolution image show the effectiveness of regularization in spatial domain and open challenging perspectives for terrain campaigns planning.

Missing data in very high spatial resolution (VHR) optical imagery take origin mainly from the acquisition conditions. Their accurate reconstruction represents a great methodological challenge because of the complexity and the ill-posed nature of the problem. In this letter, we present three different solutions, with all based on the inpainting approach, which consists in reconstructing the missing regions in a given image by propagating the spectrogeometrical information retrieved from the remaining parts of the image. They rely on the idea to enrich the patch search process by including local image properties or by isometric transformations or to reformulate it under a multiresolution processing scheme, respectively. Thorough experiments conducted on two different VHR images are reported and discussed.

Active learning is showing to be a useful approach to improve the efficiency of the classification process for remote sensing images. This letter introduces a new active learning strategy specifically developed for support vector machine (SVM) classification. It relies on the idea of the following: 1) reformulating the original classification problem into a new problem where it is needed to discriminate between significant and nonsignificant samples, according to a concept of significance which is proper to the SVM theory; and 2) constructing the corresponding significance space to suitably guide the selection of the samples potentially useful to better deal with the original classification problem. Experiments were conducted on both multi- and hyperspectral images. Results show interesting advantages of the proposed method in terms of convergence speed, stability, and sparseness.

In this paper, we propose a novel semisupervised Gaussian regression approach for the estimation of biophysical parameters from remote sensing data with limited training samples. During the learning phase, unlabeled samples are exploited to inflate the training set. The estimation of the targets associated with these samples is carried out by solving an optimization problem formulated within a genetic optimization framework. The search process of the target estimates is guided by the separate or joint optimization of two different criteria expressing the generalization capabilities of the GP estimator. The first is the empirical risk quantified in terms of the mean square error (MSE) measure; and the second is the log marginal likelihood. This last merges two terms expressing the model complexity and the data fit capability, respectively. Experimental results obtained on a real dataset representing chlorophyll concentrations in coastal waters confirm the interesting capabilities of the proposed approach.

In this work, we present a new support vector machine (SVM)-based active learning method for the classification of remote sensing images. Starting from an initial suboptimal training set, an iterative process defines the regions of significance in the feature space, then selects additional samples from a large set of unlabeled data and adds them to the training set after their manual labeling. Experimental results on a very high resolution (VHR) image show that the proposed method exhibits promising capabilities to select samples that are really significant for the classification problem, both in terms of accuracy and stability.

In this letter, we explore the effectiveness of a novel regression method in the context of the estimation of biophysical parameters from remotely sensed imagery as an alternative to state-of-the-art regression methods like those based on artificial neural networks and support vector machines. This method, called Gaussian process (GP) regression, formulates the learning of the regressor within a Bayesian framework, where the regression model is derived by assuming the model variables follow a Gaussian prior distribution encoding the prior knowledge about the output function. One of its interesting properties, which gives it a key advantage over state-of-the-art regression methods, is the possibility to tune the free parameters of the model in an automatic way. Experiments were focused on the problem of estimating chlorophyll concentration in subsurface waters. The achieved results suggest that the GP regression method is very promising from both viewpoints of estimation accuracy and free parameter tuning. Moreover, it handles particularly well the problem of limited availability of training samples, typically encountered in biophysical parameter estimation applications.

Recently, a promising pattern-recognition system has been presented to deal with the extraction of buried-object characteristics in ground-penetrating-radar images. In particular, it allows the detecting of buried objects by means of a search method based on genetic algorithms and the recognizing of the material type of the identified objects through a classification approach based on support vector machines. In this letter, we propose to extend the processing capabilities of this system by addressing the issue of the detected buried-object size estimation. This problem is viewed as a regression issue where it is aimed at reproducing the relationship between a set of opportunely extracted features and the object size. For such purpose, it is formulated within a Gaussian process (GP) regression approach. A detailed experimental study is reported, showing encouraging object-size-estimation accuracies even when buried objects are close to each other.

Gaussian processes (GPs) represent a powerful and interesting theoretical framework for Bayesian classification. Despite having gained prominence in recent years, they remain an approach whose potentialities are not yet sufficiently known. In this paper, we propose a thorough investigation of the GP approach for classifying multisource and hyperspectral remote sensing images. To this end, we explore two analytical approximation methods for GP classification, namely, the Laplace and expectation-propagation methods, which are implemented with two different covariance functions, i.e., the squared exponential and neural-network covariance functions. Moreover, we analyze how the computational burden of GP classifiers (GPCs) can be drastically reduced without significant losses in terms of discrimination power through a fast sparse-approximation method like the informative vector machine. Experiments were designed aiming also at testing the sensitivity of GPCs to the number of training samples and to the curse of dimensionality. In general, the obtained classification results show clearly that the GPC can compete seriously with the state-of-the-art support vector machine classifier.

In this paper, the unsupervised change-detection problem in remote sensing images is formulated as a segmentation issue where the discrimination between changed and unchanged classes in the difference image is achieved by defining a proper energy functional. The minimization of this functional is carried out by means of a level set method which iteratively seeks to find a global optimal contour splitting the image into two mutually exclusive regions associated with changed and unchanged classes, respectively. In order to increase the robustness of the method to noise and to the choice of the initial contour, a multiresolution implementation, which performs an analysis of the difference image at different resolution levels, is proposed. The experimental results obtained on three different multitemporal remote sensing images acquired by low- as well as high-spatial-resolution optical remote sensing sensors suggest a clear superiority of the proposed approach compared with state-of-the-art change-detection methods.

In this paper, we present three active learning strategies for the classification of electrocardiographic (ECG) signals. Starting from a small and suboptimal training set, these learning strategies select additional beat samples from a large set of unlabeled data. These samples are labeled manually, and then added to the training set. The entire procedure is iterated until the construction of a final training set representative of the considered classification problem. The proposed methods are based on support vector machine classification and on the: 1) margin sampling; 2) posterior probability; and 3) query by committee principles, respectively. To illustrate their performance, we conducted an experimental study based on both simulated data and real ECG signals from the MIT-BIH arrhythmia database. In general, the obtained results show that the proposed strategies exhibit a promising capability to select samples that are significant for the classification process, i.e., to boost the accuracy of the classification process while minimizing the number of involved labeled samples.

In this paper, we propose a variational level-set method for unsupervised change-detection in remote sensing images. The discrimination between changed and unchanged classes in the difference image is achieved by defining an energy functional known as the piecewise constant approximation Mumford-Shah segmentation model. The minimization of this energy functional is realized according to an attractive level-set method seeking to find an optimal contour which splits the image into two mutually exclusive regions associated with changed and unchanged classes, respectively. In order to increase the robustness against the initialization issue, we adopt a multiresolution level-set approach by analyzing the difference image at different resolution levels. The experimental results obtained on two multitemporal remote sensing images acquired by low as well as very high spatial remote sensing sensors confirm the promising capabilities of the proposed approach.

In this paper, we propose an automatic method for counting olive trees in very high spatial remote sensing images. In a first step, olive trees are separated from other land-cover classes present in the image by means of a Gaussian process classifier (GPC). Due to the important role of the spatial information in very high resolution imagery, we feed the GPC with different morphological features computed from the original image. The output of this step is a binary classification map containing olive trees seen as foreground and other classes as background. In the second step, the number of blobs in the image representing possible olive trees is counted using an automatic procedure. Each blob is considered valid if its size is within a range specified a priori referring to the real size of trees. Experimental results obtained on a very high spatial remote sensing image acquired by the IKONOS-2 sensor are reported.

This work presents a procedure to optimize a wavelet filter in terms of discrimination capability between the classes characterizing a given hyperspectral remote sensing image. To this end, this procedure estimates the coefficients of the wavelet filter bank by means of a particle swarm optimization (PSO) so that to maximize the average Bhattacharyya distance. The obtained experimental results show that PSO-based optimized wavelets can significantly outperform conventional wavelets.

A new methodology for the unsupervised classification of hyperspectral images is proposed. Based on swarm intelligence, it addresses simultaneously two different issues which are: 1) the estimation of the cluster parameters; and 2) the detection of the best discriminative bands. For such purpose, it optimizes jointly two different criteria, which are the log likelihood function and the Bhattacharyya statistical distance between classes. Experimental results show that, despite the completely unsupervised nature of the proposed methodology, very encouraging performances in terms of classification accuracy can be achieved.

In this work, we present a pattern recognition system for the automatic analysis of ground penetrating radar (GPR) images. This system comprises pre-processing, segmentation, object detection, object material recognition, and object dimension estimation stages. Object detection is done using an unsupervised strategy based on genetic algorithms (GA) which allows to localize linear/hyperbolic patterns in GPR images. Object material recognition is approached as a classification issue, which is solved by means of a support vector machine (SVM) classifier. Dimension estimation is formulated within a Gaussian process (GP) regression approach. Results on synthetic images, representing random exploration scenarios, are reported and discussed.

Wavelets are known to be a valuable tool for analyzing hyperspectral images. In this letter, we propose to further improve their performance by means of a novel classification-driven design scheme that aims at deriving a wavelet that best represents in terms of between-class discrimination capability the spectral signatures conveyed by a given hyperspectral image. This is achieved by adopting a polyphase representation of the wavelet filter bank and formulating the wavelet optimization problem within a particle-swarm-optimization (PSO) framework. Experimental results show that the proposed wavelet design method outperforms the popular Daubechies wavelets whatever the classifier type adopted in the classification process.

In this paper, we propose a novel method that aims at assisting the ground-truth expert through an automatic detection of potentially mislabeled learning samples. This method is based on viewing the mislabeled sample detection issue as an optimization problem where it is looked for the best subset of learning samples in terms of statistical separability between classes. This problem is formulated within a genetic optimization framework, where each chromosome represents a candidate solution for validating/invalidating the learning samples collected by the ground-truth expert. The genetic optimization process is guided by the joint optimization of two different criteria which are the maximization of a between-class statistical distance and the minimization of the number of invalidated samples. Experiments conducted on both simulated and real data sets show that the proposed ground-truth validation method succeeds in the following: 1) in detecting the mislabeled samples with a high accuracy, even when up to 30% of the learning samples are mislabeled, and 2) in strongly limiting the negative impact of the mislabeling issue on the accuracy of the classification process.

In this paper, a novel method for semisupervised classification with limited training samples is presented. Its aim is to exploit unlabeled data available at zero cost in the image under analysis for improving the accuracy of a classification process based on support vector machines (SVMs). It is based on the idea to augment the original set of training samples with a set of unlabeled samples after estimating their label. The label estimation process is performed within a multiobjective genetic optimization framework where each chromosome of the evolving population encodes the label estimates as well as the SVM classifier parameters for tackling the model selection issue. Such a process is guided by the joint minimization of two different criteria which express the generalization capability of the SVM classifier. The two explored criteria are an empirical risk measure and an indicator of the classification model sparseness, respectively. The experimental results obtained on two multisource remote sensing data sets confirm the promising capabilities of the proposed approach, which allows the following: (1) taking a clear advantage in terms of classification accuracy from unlabeled samples used for inflating the original training set and (2) solving automatically the tricky model selection issue.

In this paper, we present a new methodology for clustering hyperspectral images. It aims at simultaneously solving the following three different issues: 1) estimation of the class statistical parameters; 2) detection of the best discriminative bands without requiring thea priorisetting of their number by the user; and 3) estimation of the number of data classes characterizing the considered image. It is formulated within a multiobjective particle swarm optimization (MOPSO) framework and is guided by three different optimization criteria, which are the log-likelihood function, the Bhattacharyya statistical distance between classes, and the minimum description length (MDL). A detailed experimental analysis was conducted on both simulated and real hyperspectral images. In general, the obtained results show that interesting classification performances can be achieved by the proposed methodology despite its completely unsupervised nature.

In this paper, we propose a novel pattern-recognition system to identify and classify buried objects from ground-penetrating radar (GPR) imagery. The entire process is subdivided into four steps. After a preprocessing step, the GPR image is thresholded to put under light the regions containing potential objects. The third step of the system consists of automatically detecting the objects in the obtained binary image by means of a search of linear/hyperbolic patterns formulated within a genetic optimization framework. In the genetic optimizer, each chromosome models the apex position and the curvature associated with the candidate pattern, while the fitness function expresses the Hamming distance between that pattern and the binary image content. Finally, in the fourth step, the problem of the recognition of the material type of the identified objects is approached as a classification issue, which is solved by means of an opportune feature-extraction strategy and a support vector machine classifier. To illustrate the performances of the proposed system, we conducted a thorough experimental study based on GPR images generated by a GPR simulator based on the finite-difference time-domain method so as to construct different acquisition scenarios by varying the number of buried objects, their position, their size, their shape, and their material type. In general, the obtained experimental results show that the proposed system exhibits promising performances both in terms of object detection and material recognition.

In this paper, we explore the effectiveness of the Bayesian Gaussian process approach for classifying hyperspectral remote sensing images. In particular, we consider two analytical approximation methods for Gaussian process classification, which are the Laplace and the expectation propagation methods. Experimental results obtained on a benchmark hyperspectral dataset show that, in terms of classification accuracy, Gaussian process classification can compete seriously with the state-of-the-art classification approach based on support vector machines.

In this paper, we propose a novel genetic method that aims at providing the ground-truth expert with a binary information of the kind "validated"/"invalidated" for each ground-truth (learning) sample collected. For each invalidated sample, the expert may confirm or not the invalidation, and thus correct or maintain the adopted labeling before creating the final learning set that will be exploited in the classification process. Experimental results confirm the effectiveness of the proposed method in correctly detecting mislabeled learning samples and thus in limiting their negative impact on the classification process.

Recently, a new machine learning approach that is based on the Gaussian process (GP) theory has been introduced in the literature. According to this approach, the learning of a machine (regressor or classifier) is formulated in terms of a Bayesian estimation problem, where the parameters of the machine are assumed to be random variables which follow jointly a Gaussian distribution. The purpose of this work is to investigate this approach in the context of the estimation of biophysical parameters. Experimental results obtained on synthetic and real data, which simulate the spectral behavior of the chlorophyll concentration in subsurface waters, are reported and compared with those yielded by the general regression neural network (GRNN) and the epsiv-insensitive support vector regression (SVR) methods.

This work presents a novel pattern recognition approach for the automatic analysis of ground penetrating radar (GPR) images. The developed system comprises pre-processing, segmentation, object detection and material recognition stages. Object detection is done using an innovative unsupervised strategy based on genetic algorithms (GA) that allows to localize linear/hyperbolic patterns in GPR images. Object material recognition is approached as a classification issue, which is solved by means of a support vector machine (SVM) classifier. Results on synthetic images show that the proposed system exhibits promising performances both in terms of object detection and material recognition.

A general method has been proposed recently for the contextual reconstruction of cloud-contaminated areas in multitemporal multispectral images. It is based on the idea of making the prediction process learn from information available in the cloud-free neighborhood of contaminated areas. Though promising, this method does not fully exploit all available information, thus leaving room for further methodological enhancements. This letter presents a postreconstruction methodology for improving the contextual reconstruction process by opportunely capturing spatial and spectral correlations characterizing the considered image. In addition, we propose a solution to a problem that has not yet been addressed in the remote sensing literature, i.e., the generation of an error map beside the reconstructed images to provide end-users with helpful indications about reconstruction reliability. Thorough experiments conducted on a multitemporal sequence of Landsat-7 ETM+ images are reported and discussed.

The updating of classification maps, as new image acquisitions are obtained, raises the problem of ground-truth information (training samples) updating. In this context, semisupervised multitemporal classification represents an interesting though still not well consolidated approach to tackle this issue. In this letter, we propose a novel methodological solution based on this approach. Its underlying idea is to update the ground-truth information through an automatic estimation process, which exploits archived ground-truth information as well as basic indications from the user about allowed/forbidden class transitions from an acquisition date to another. This updating problem is formulated by means of the support vector machine classification approach and a constrained multiobjective optimization genetic algorithm. Experimental results on a multitemporal data set consisting of two multisensor (Landsat-5 Thematic Mapper and European Remote Sensing satellite synthetic aperture radar) images are reported and discussed.

In this paper, we present a new variant of the k-nearest neighbor (kNN) classifier based on the maximal margin principle. The proposed method relies on classifying a given unlabeled sample by first finding its k-nearest training samples. A local partition of the input feature space is then carried out by means of local support vector machine (SVM) decision boundaries determined after training a multiclass SVM classifier on the considered k training samples. The labeling of the unknown sample is done by looking at the local decision region to which it belongs. The method is characterized by resulting global decision boundaries of the piecewise linear type. However, the entire process can be kernelized through the determination of the k -nearest training samples in the transformed feature space by using a distance function simply reformulated on the basis of the adopted kernel. To illustrate the performance of the proposed method, an experimental analysis on three different remote sensing datasets is reported and discussed.

The aim of this paper is twofold. First, we present a thorough experimental study to show the superiority of the generalization capability of the support vector machine (SVM) approach in the automatic classification of electrocardiogram (ECG) beats. Second, we propose a novel classification system based on particle swarm optimization (PSO) to improve the generalization performance of the SVM classifier. For this purpose, we have optimized the SVM classifier design by searching for the best value of the parameters that tune its discriminant function, and upstream by looking for the best subset of features that feed the classifier. The experiments were conducted on the basis of ECG data from the Massachusetts Institute of Technology-Beth Israel Hospital (MIT-BIH) arrhythmia database to classify five kinds of abnormal waveforms and normal beats. In particular, they were organized so as to test the sensitivity of the SVM classifier and that of two reference classifiers used for comparison, i.e., the k-nearest neighbor (kNN) classifier and the radial basis function (RBF) neural network classifier, with respect to the curse of dimensionality and the number of available training beats. The obtained results clearly confirm the superiority of the SVM approach as compared to traditional classifiers, and suggest that further substantial improvements in terms of classification accuracy can be achieved by the proposed PSO-SVM classification system. On an average, over three experiments making use of a different total number of training beats (250, 500, and 750, respectively), the PSO-SVM yielded an overall accuracy of 89.72% on 40438 test beats selected from 20 patient records against 85.98%, 83.70%, and 82.34% for the SVM, the kNN, and the RBF classifiers, respectively.

In this paper, we present a novel multiobjective particle swarm optimization (MOPSO) approach for SVM regression with limited training samples. This approach, which is applied to the estimation of biophysical parameters from remote sensing images, is an extension of a work recently presented in the literature. It aims at exploiting unlabeled samples available from the image under analysis at zero cost to increase further the accuracy of the estimation process. The integration of such samples is made by optimizing simultaneously two criteria expressing the generalization capability of the SVM estimator, namely, the support vector count and the empirical risk. Experimental results obtained on synthetic and real multispectral data, which simulate the spectral behavior of the chlorophyll concentration in subsurface waters, are reported and discussed.

Recently, a novel general method for the contextual reconstruction of cloud-contaminated areas in multitemporal multispectral images has been proposed in the literature. Though promising, this method does not fully exploit all available information, thus leaving room for further methodological enhancements. The aim of this work is to improve the contextual reconstruction process by opportunely capturing spatial and spectral correlations characterizing the considered image. In addition, we propose a novel procedure to generate an error map for providing the end-user with helpful indications about the reconstruction reliability. Thorough experiments conducted on a multitemporal sequence of Landsat-7 ETM+ images are reported and discussed.

This work aims at proposing a methodological solution to the challenging problem of semi-supervised classification map updating. The underlying idea of the proposed method is to update automatically the ground-truth information that will be exploited to train a support vector machine (SVM) classifier for the image under analysis. Such updating problem is formulated within a constrained multiobjective genetic algorithm (MOGA) which makes use of temporal information provided by the user under the form of allowed/forbidden class transitions. Experimental results on a multitemporal data set consisting of two multisensor (Landsat-5 TM and ERS-1 SAR) images are reported and discussed.

In this paper, a novel semisupervised regression approach is proposed to tackle the problem of biophysical parameter estimation that is constrained by a limited availability of training (labeled) samples. The main objective of this approach is to increase the accuracy of the estimation process based on the support vector machine (SVM) technique by exploiting unlabeled samples that are available from the image under analysis at zero cost. The integration of such samples in the regression process is controlled through a particle swarm optimization (PSO) framework that is defined by considering separately or jointly two different optimization criteria, thus leading to the implementation of three different inflation strategies. These two criteria are empirical and structural expressions of the generalization capability of the resulting semisupervised PSO-SVM regression system. The conducted experiments were focused on the problem of estimating chlorophyll concentrations in coastal waters from multispectral remote sensing images. In particular, we report and discuss results of experiments that are designed in such a way as to test the proposed approach in terms of: 1) capability to capture useful information from a set of unlabeled samples for improving the estimation accuracy; 2) sensitivity to the number of exploited unlabeled samples; and 3) sensitivity to the number of labeled samples used for supervising the inflation process

Because of the strong statistical variability of remote sensing images, the selection of the best thresholding algorithm to detect changes between two successive temporal images of the same study area without any prior knowledge is often not easy. In this paper, we face this problem through a new robust change detection approach. In order to achieve robustness, the proposed unsupervised approach is based on a Markov random field (MRF) fusion of change maps provided by an ensemble of different thresholding algorithms. Experimental results obtained on three images acquired by different sensors and referring to different kinds of changes confirm the robustness of the proposed approach.

In this letter, we propose an extension of an automatic and unsupervised change-detection method for synthetic aperture radar images we presented earlier. By analyzing a properly defined cost function, the proposed method allows the automatic detection of the number (zero, one, or two) and the values of the decision thresholds associated with changes (if any) in the log-ratio image. This cost function is the minimum value of the criterion function adopted to select the decision threshold in the log-ratio image according to a modified double-thresholding Kittler-Illingworth algorithm (implemented under the generalized Gaussian assumption for changed and unchanged classes). Experimental results carried out both on real and simulated multitemporal synthetic aperture radar images proved the effectiveness of the proposed automatic method in detecting both the number of changes to be identified (the situation of no changes is also explicitly identified) and the related threshold values

This letter introduces the epsiv-Huber loss function in the support vector regression (SVR) formulation for the estimation of biophysical parameters extracted from remotely sensed data. This cost function can handle the different types of noise contained in the dataset. The method is successfully compared to other cost functions in the SVR framework, neural networks and classical bio-optical models for the particular case of the estimation of ocean chlorophyll concentration from satellite remote sensing data. The proposed model provides more accurate, less biased, and improved robust estimation results on the considered case study, especially significant when few in situ measurements are available

The most common methodology to carry out an automatic unsupervised change detection in remotely sensed imagery is to find the best global threshold in the histogram of the so-called difference image. The unsupervised nature of the change detection process, however, makes it nontrivial to find the most appropriate thresholding algorithm for a given difference image, because the best global threshold depends on its statistical peculiarities, which are often unknown. In this letter, a solution to this issue based on the fusion of an ensemble of different thresholding algorithms through a Markov random field framework is proposed. Experiments conducted on a set of five real remote sensing images acquired by different sensors and referring to different kinds of changes show the high robustness of the proposed unsupervised change detection approach

Recent remote sensing literature has shown that support vector machine (SVM) methods generally outperform traditional statistical and neural methods in classification problems involving hyperspectral images. However, there are still open issues that, if suitably addressed, could allow further improvement of their performances in terms of classification accuracy. Two especially critical issues are: 1) the determination of the most appropriate feature subspace where to carry out the classification task and 2) model selection. In this paper, these two issues are addressed through a classification system that optimizes the SVM classifier accuracy for this kind of imagery. This system is based on a genetic optimization framework formulated in such a way as to detect the best discriminative features without requiring the a priori setting of their number by the user and to estimate the best SVM parameters (i.e., regularization and kernel parameters) in a completely automatic way. For these purposes, it exploits fitness criteria intrinsically related to the generalization capabilities of SVM classifiers. In particular, two criteria are explored, namely: 1) the simple support vector count and 2) the radius margin bound. The effectiveness of the proposed classification system in general and of these two criteria in particular is assessed both by simulated and real experiments. In addition, a comparison with classification approaches based on three different feature selection methods is reported, i.e., the steepest ascent (SA) algorithm and two other methods explicitly developed for SVM classifiers, namely: 1) the recursive feature elimination technique and 2) the radius margin bound minimization method

The frequent presence of clouds in passive remotely sensed imagery severely limits its regular exploitation in various application fields. Thus, the removal of cloud cover from this imagery represents an important preprocessing task consisting in the reconstruction of cloud-contaminated data. The intent of this study is to propose two novel general methods for the reconstruction of areas obscured by clouds in a sequence of multitemporal multispectral images. Given a cloud-contaminated image of the sequence, each area of missing measurements is reconstructed through an unsupervised contextual prediction process that reproduces the local spectro-temporal relationships between the considered image and an opportunely selected subset of the remaining temporal images. In the first method, the contextual prediction process is implemented by means of an ensemble of linear predictors, each trained over a local multitemporal region that is spectrally homogeneous in each temporal image of the selected subset. In order to obtain such regions, each temporal image is locally classified by an unsupervised classifier based on the expectation-maximization (EM) algorithm. In the second method, the local spectro-temporal relationships are reproduced by a single nonlinear predictor based on the support vector machines (SVM) approach. To illustrate the performance of the two proposed methods, an experimental analysis on a sequence of three temporal images acquired by the Landsat-7 Enhanced Thematic Mapper Plus sensor over a total period of four months is reported and discussed. It includes a detailed simulation study that aims at assessing with different reconstruction quality criteria the accuracy of the methods in different qualitative and quantitative cloud contamination conditions. Compared with two techniques based on compositing algorithms for cloud removal, the proposed methods show a clear superiority, which makes them a promising and useful tool in solving the considered problem, whose great complexity is commensurate with its practical importance.

This paper presents a novel method for steganographic image watermarking with two main innovative features: i) it involves a hierarchical control, committing the watermark reconstruction to a trusted layered authority; and ii) it is deterministic, embedding a short meaningful signature into the cover image. Experimental results show that the embedded signature can be accurately recovered even in presence of a reasonable amount of image degradation due to image processing operators, such as filtering, geometric distorsions and compression.

We present a novel automatic and unsupervised change-detection approach specifically oriented to the analysis of multitemporal single-channel single-polarization synthetic aperture radar (SAR) images. This approach is based on a closed-loop process made up of three main steps: (1) a novel preprocessing based on a controlled adaptive iterative filtering; (2) a comparison between multitemporal images carried out according to a standard log-ratio operator; and (3) a novel approach to the automatic analysis of the log-ratio image for generating the change-detection map. The first step aims at reducing the speckle noise in a controlled way in order to maximize the discrimination capability between changed and unchanged classes. In the second step, the two filtered multitemporal images are compared to generate a log-ratio image that contains explicit information on changed areas. The third step produces the change-detection map according to a thresholding procedure based on a reformulation of the Kittler-Illingworth (KI) threshold selection criterion. In particular, the modified KI criterion is derived under the generalized Gaussian assumption for modeling the distributions of changed and unchanged classes. This parametric model was chosen because it is capable of better fitting the conditional densities of classes in the log-ratio image. In order to control the filtering step and, accordingly, the effects of the filtering process on change-detection accuracy, we propose to identify automatically the optimal number of despeckling filter iterations [Step 1] by analyzing the behavior of the modified KI criterion. This results in a completely automatic and self-consistent change-detection approach that avoids the use of empirical methods for the selection of the best number of filtering iterations. Experiments carried out on two sets of multitemporal images (characterized by different levels of speckle noise) acquired by the European Remote Sensing 2 satellite SAR sensor confirm the effectiveness of the proposed unsupervised approach, which results in change-detection accuracies very similar to those that can be achieved by a manual supervised thresholding.

An approach based on multiple estimator systems (MESs) for the estimation of biophysical parameters from remotely sensed data is proposed. The rationale behind the proposed approach is to exploit the peculiarities of an ensemble of different estimators in order to improve the robustness (and in some cases the accuracy) of the estimation process. The proposed MESs can be implemented in two conceptually different ways. One extends the use of an approach previously proposed in the regression literature to the estimation of biophysical parameters from remote sensing data. This approach integrates the estimates obtained from the different regression algorithms making up the ensemble by a direct linear combination (combination-based approach). The other consists of a novel approach that provides as output the estimate obtained by the regression algorithm (included in the ensemble) characterized by the highest expected accuracy in the region of the feature space associated with the considered pattern (selection-based approach). This estimator is identified based on a proper partition of the feature space. The effectiveness of the proposed approach has been assessed on the problem of estimating water quality parameters from multispectral remote sensing data. In particular, the presented MES-based approach has been evaluated by considering different operational conditions where the single estimators included in the ensemble are: 1) based on the same or on different regression methods; 2) characterized by different tradeoffs between correlated errors and accuracy of the estimates; 3) trained on samples affected or not by measurement errors. In the definition of the ensemble particular attention is devoted to support vector machines (SVMs), which are a promising approach to the solution of regression problems. In particular, a detailed experimental analysis on the effectiveness of SVMs for solving the considered estimation problem is presented. The experimental results point out that the SVM method is effective and that the proposed MES approach is capable of increasing both the robustness and accuracy of the estimation process.

This paper presents a novel approach to unsupervised change detection in multitemporal SAR images. This approach is based on three main steps: (1) controlled preprocessing based on adaptive filtering (despeckling); (2) comparison between multitemporal images according to a proper operator; (3) automatic thresholding of the log-ratio image. The first step aims at reducing the speckle noise in a controlled way in order to maximize the separability between changed and unchanged classes. The second step is devoted to compare the two filtered images in order to generate a log-ratio image. Finally, the third step deals with the identification of changes by thresholding the log-ratio image according to a novel technique. Such a technique is based on the double thresholding Kittler & Illingworth (K&I) algorithm, which is reformulated under the Generalized Gaussian (GG) assumption for the changed and unchanged classes. Experimental results obtained on a multitemporal SAR data set confirm the effectiveness of the proposed approach.

In this paper, we propose a novel supervised approach to classification of high spatial resolution images. This approach is aimed at obtaining accurate and reliable classification maps by properly preserving the geometrical details present in the images. It is based on: i) a feature-extraction module, which exploits an adaptive, multilevel and hierarchical modeling of the investigated scene; ii) a support vector machine (SVM) classifier. The choice to adopt an SVM classification technique is motivated by the high number of parameters derived from the feature-extraction phase, which requires a classifier suitable to the analysis of hyperdimensional features spaces. Experimental results and comparisons with a standard technique developed for the analysis of high-spatial resolution images confirm the effectiveness of the proposed approach.

Robust estimation of biophysical parameters in large geographical areas from remote sensing images represents an important methodological issue. A possible approach to this problem consists in modeling and correcting the systematic errors (residues) generated by an estimator trained to approximate the relationship between the remote sensing measurements and the biophysical parameter of interest. In this paper, we propose to extend this approach by capturing information from residues of higher order to refine further the approximated model. The proposed technique was applied to the problem of estimating water quality parameters with a particular focus on the estimation of the chlorophyll concentration. Two data sets and two regression methods (based on Support vector Machines (SVM) and Multilayer Perceptron (MLP) neural networks) were considered for the experimental phase. The obtained results point out that the exploitation of residues of order smaller or equal than two can improve the estimation accuracy while, above this order, overfitting problems may appear.

A fuzzy-logic approach to the classification of multitemporal, multisensor remote-sensing images is proposed. The approach is based on a fuzzy fusion of three basic sources of information: spectral, spatial and temporal contextual information sources. It aims at improving the accuracy over that of single-time noncontextual classification. Single-time class posterior probabilities, which are used to represent spectral information, are estimated by Multilayer Perceptron neural networks trained for each single-time image, thus making the approach applicable to multisensor data. Both the spatial and temporal kinds of contextual information are derived from the single-time classification maps obtained by the neural networks. The expert's knowledge of possible transitions between classes at two different times is exploited to extract temporal contextual information. The three kinds of information are then fuzzified in order to apply a fuzzy reasoning rule for their fusion. Fuzzy reasoning is based on the "MAX" fuzzy operator and on information about class prior probabilities. Finally, the class with the largest fuzzy output value is selected for each pixel in order to provide the final classification map. Experimental results on a multitemporal data set consisting of two multisensor (Landsat TM and ERS-1 SAR) images are reported. The accuracy of the proposed fuzzy spatio-temporal contextual classifier is compared with those obtained by the Multilayer Perceptron neural networks and a reference classification approach based on Markov Random Fields (MRFs). Results show the benefit of adding spatio-temporal contextual information to the classification scheme, and suggest that the proposed approach represents an interesting alternative to the MRF-based approach, in particular, in terms of simplicity.

This paper addresses the problem of the classification of hyperspectral remote sensing images by support vector machines (SVMs). First, we propose a theoretical discussion and experimental analysis aimed at understanding and assessing the potentialities of SVM classifiers in hyperdimensional feature spaces. Then, we assess the effectiveness of SVMs with respect to conventional feature-reduction-based approaches and their performances in hypersubspaces of various dimensionalities. To sustain such an analysis, the performances of SVMs are compared with those of two other nonparametric classifiers (i.e., radial basis function neural networks and the K-nearest neighbor classifier). Finally, we study the potentially critical issue of applying binary SVMs to multiclass problems in hyperspectral data. In particular, four different multiclass strategies are analyzed and compared: the one-against-all, the one-against-one, and two hierarchical tree-based strategies. Different performance indicators have been used to support our experimental studies in a detailed and accurate way, i.e., the classification accuracy, the computational time, the stability to parameter setting, and the complexity of the multiclass architecture. The results obtained on a real Airborne Visible/Infrared Imaging Spectroradiometer hyperspectral dataset allow to conclude that, whatever the multiclass strategy adopted, SVMs are a valid and effective alternative to conventional pattern recognition approaches (feature-reduction procedures combined with a classification method) for the classification of hyperspectral remote sensing data.

This paper addresses the problem of change detection in multitemporal remote-sensing images. In particular, an approach to automatic unsupervised change detection, suitable to be used with multisource and multisensor data, is presented. This approach can be applied by exploiting two different data-fusion strategies: a pixel-based and a context-based strategy. The resulting robust change-detection method can also be applied to multispectral images by modeling each spectral channel as a separate information source, thus obtaining an alternative method to the standard change vector analysis technique. Experimental results confirm the effectiveness of the proposed approach.

In this paper, we present an advanced classification system aimed at obtaining an accurate and reliable supervised classification of remote-sensing images. The proposed system is based on a closed-loop architecture composed of an ensemble of classifiers, which are combined and integrated within a framework based on Markov Random Fields (MRFs). The basic ideas considered in the definition of the system are: i) to jointly exploit the effectiveness of the aforementioned methodologies to increase the classification accuracy with respect to standard classification approaches; ii) to implement an iterative procedure based on the back- propagation of the consensus (obtained from the ensemble of classifiers integrated with the MRF approach) from the output to the input of the system, for increasing the accuracy in the estimation of classifier parameters and hence the effectiveness of the system. I. INTRODUCTION Automatic classification is one of the most important and studied topics in the field of the analysis of remote sensing images. In the literature several approaches have been proposed for the supervised classification of remote-sensing data. These approaches range from standard statistical methods to advanced neural-network techniques, from knowledge- based methodologies to fuzzy algorithms, and from multiple classifier systems to contextual techniques. Although many of these approaches revealed effective in different application domains, often they do not exhibit accuracies sufficient for meeting end-user requirements in complex classification problems. Consequently, it is necessary to develop novel and advanced classification approaches capable to further increase accuracies in automatic classification of remote-sensing images. In this paper, we present an advanced classification system aimed at obtaining an accurate and reliable supervised classification of remote-sensing data. The proposed system is based on a closed-loop architecture, composed of an ensemble of classifiers, which are combined and integrated with a Markov Random Field (MRF) approach and with a recursive procedure of back-propagation of the consensus obtained in output of the system. II. THE PROPOSED CLASSIFICATION SYSTEM

In this paper, we address the problem of biophysical parameter estimation from measurements acquired by remote sensors. The objective of this work is to define the robust estimation method, which is characterized by a high accuracy over the whole feature space. In particular, we present a novel estimation technique that is based on the exploitation of the systematic errors (residuals) generated by an estimator trained to approximate the relationship between the remote-sensing measurements and the biophysical parameter of interest. The proposed Residual-Based Estimation (RBE) technique was applied to the problem of estimating water quality parameters, with a particular focus on the measure of concentration of chlorophyll. Experimental results pointed out the effectiveness of the RBE technique, which significantly increased the estimation accuracy with respect to the different kinds of standard neural-network estimators.

Markov random fields (MRFs) provide a useful and theoretically well-established tool for integrating temporal contextual information into the classification process. In particular, when dealing with a sequence of temporal images, the usual MRF-based approach consists in adopting a "cascade" scheme, i.e., in propagating the temporal information from the current image to the next one of the sequence. The simplicity of the cascade scheme makes it attractive; on the other hand, it does not fully exploit the temporal information available in a sequence of temporal images. In this paper, a "mutual" MRF approach is proposed that aims at improving both the accuracy and the reliability of the classification process by means of a better exploitation of the temporal information. It involves carrying out a bidirectional exchange of the temporal information between the defined single-time MRF models of consecutive images. A difficult issue related to MRFs is the determination of the MRF model parameters that weight the energy terms related to the available information sources. To solve this problem, we propose a simple and fast method based on the concept of "minimum perturbation" and implemented with the pseudoinverse technique for the minimization of the sum of squared errors. Experimental results on a multitemporal dataset made up of two multisensor (Landsat Thematic Mapper and European Remote Sensing 1 synthetic aperture radar) images are reported. The results obtained by the proposed "mutual" approach show a clear improvement in terms of classification accuracy over those yielded by a reference MRF-based classifier. The presented method to automatically estimate the MRF parameters yielded significant results that make it an attractive alternative to the usual trial-and-error search procedure.

In this paper, we address the problem of classification of hyperspectral remote-sensing images (in the original hyperdimensional feature space) by Support Vector Machines (SVMs). In particular, we investigate the effectiveness of SVMs in terms of classification accuracy, computational time and stability to parameter setting. Experiments, carried out on a standard AVIRIS hyperspectral data set, include a comparison with two other widely used nonparametric approaches, i.e., the K-nn and the Radial Basis Function (RBF) neural networks classifiers. The obtained results point out interesting properties of SVMs in hyperdimensional feature spaces and suggest them as a promising tool to classify hyperspectral remote-sensing images.

Given a temporal sequence of remote sensing images, the difficulty to collect regularly in time ground truth information makes it important to develop automatic unsupervised change detection techniques. Due to its simplicity, image differencing represents a popular approach for change detection. To separate the "change" and "no-change" classes in the difference image, a thresholding-based procedure can be applied. However, the main weakness of an unsupervised change detection approach is the absence of prior information about the scene as it resorts to the spectral information, only, which does not allow the analysis of the typologies of changes occurring between the acquisition dates. In the monitoring of a given study area, the main problem is that the ground truth collection does not usually follow the image acquisitions at the different dates. However, it is easier to have at least one image for which the ground truth is available. In this paper, we propose a partially supervised change detection scheme that is based on the exploitation of the ground truth availability for at least one temporal image. A clustering algorithm is applied to both acquired images and a thresholding-based unsupervised change detection algorithm is applied inside each cluster of the second date image in order not only to identify the presence of changes but also to distinguish between different typologies of changes.

In order to apply a statistical approach to the classification of multisource remote-sensing data, one of the main problems to face lies in the estimation of probability distribution functions. This problem arises out of the difficulty of defining a common statistical model for such heterogeneous data. A possible solution is to adopt nonparametric approaches, which rely on the availability of training samples without any assumption about the related statistical distributions. The purpose of this paper is to investigate the suitability of the concept of dependence trees for the integration of multisource information through estimation of probability distributions. First, this concept, introduced by Chow and Liu (1968), is used to provide an approximation of a probability distribution defined in an N-dimensional space by a product of N-1 probability distributions defined in two-dimensional (2-D) spaces; this approximation corresponds, in terms of graph theoretical interpretation, to a tree of dependence. For each land cover class, a dependence tree is generated by minimizing an appropriate closeness measure. Then, a nonparametric estimation of the second-order probability distributions is carried out through the Parzen window approach, based on the implementation of 2-D Gaussian kernels. In this way, it is possible to reduce the complexity of the estimation, while capturing a significant part of the interdependence among variables. A comparison with other multisource data fusion methods, namely, the multilayer perceptron (MLP) method, the k-nearest neighbor (k-NN) method, and a Bayesian hierarchical classifier (BHC), is made. Experimental results obtained on multisensor [airborne thematic mapper (ATM) and synthetic aperture radar (SAR)] and multisource (experimental synthetic aperture radar (E-SAR) and a textural feature) data sets show that the proposed fusion method based on dependence trees is able to provide a classification accuracy similar to those of the other methods considered, but with the advantage of a reduced computational load.

Fuzzy classification has become of great interest because of its capacity to provide more useful information for geographic information systems. This paper describes an explicit fuzzy supervised classification method which consists of three steps. The explicit fuzzyfication is the first step where the pixel is transformed into a matrix of membership degrees representing the fuzzy inputs of the process. Then, in the second step, a MIN fuzzy reasoning rule followed by a rescaling operation are applied to deduce the fuzzy outputs, or in other words, the fuzzy classification of the pixel. Finally, a defuzzyfication step is carried out to produce a hard classification. The classification results on Landsat TM data demonstrate the promising performances of the method and comparatively short classification time.