Heart sounds, or phonocardiograms (PCG), are important for diagnosing cardiovascular conditions, providing a non-invasive means to assess heart function through auscultation. Accurate classification of PCG signals can facilitate early detection of cardiac abnormalities, significantly improving patient outcomes. However, the complexity and variability of heart sound recordings present significant challenges for traditional classification methods, necessitating advanced approaches that can effectively handle the nuances of cardiac acoustics. This paper introduces a novel transfer learning approach that adapts OpenAI's Whisper model, originally designed for robust speech recognition, to the task of heart sound classification. In particular, we employ Whisper's encoder architecture to effectively capture acoustic features that generalize to cardiac auscultation, making it a promising candidate for PCG analysis. To tailor the model for this specialized task, we implement a modified encoder architecture optimized for heart sound characteristics. We process the input to the model using a Log-Mel spectrogram pipeline specifically designed to highlight the unique acoustic properties of PCG signals. Experimental results demonstrate that the adapted Whisper model achieves state-of-the-art performance, surpassing existing methods in both accuracy and robustness.

Scene classification in optical remote sensing (RS) imagery has been extensively investigated using both learning-from-scratch approaches and fine-tuning of ImageNet pretrained models. Meanwhile, contrastive language-image pretraining (CLIP) has emerged as a powerful foundation model for vision-language tasks, demonstrating remarkable zero-shot capabilities across various domains. Its image encoder is a key component in many vision instruction-tuning models, enabling effective alignment of text and visual modalities for diverse tasks. However, its potential for supervised RS scene classification remains unexplored. This work investigates the efficient adaptation of large CLIP models (containing over 300 M parameters) through low-rank adaptation (LoRA), specifically targeting the attention layers. By applying LoRA to CLIP’s attention mechanisms, we can effectively adapt the vision model for specialized scene classification tasks with minimal computational overhead, requiring fewer training epochs than traditional fine-tuning methods. Our extensive experiments demonstrate the promising capabilities of LoRA-CLIP. By training only on a small set of additional parameters, LoRA-CLIP outperforms models pretrained on ImageNet, demonstrating the clear advantages of using image–text pretrained backbones for scene classification.

In recent years, the evolution of machine learning techniques has significantly impacted the field of intrusion de-tection, particularly within the context of the Internet of Things (IoT). As IoT networks expand, the need for robust security measures to counteract potential threats has become increasingly critical. This paper introduces a hybrid Intrusion Detection System (IDS) that synergistically combines Kolmogorov-Arnold Networks (KANs) with the XGBoost algorithm. Our proposed IDS leverages the unique capabilities of KANs, which utilize learnable activation functions to model complex relationships within data, alongside the powerful ensemble learning techniques of XGBoost, known for its high performance in classification tasks. This hybrid approach not only enhances the detection accuracy but also improves the interpretability of the model, making it suitable for dynamic and intricate IoT environments. Experimental evaluations demonstrate that our hybrid IDS achieves an impressive detection accuracy exceeding 99 % in dis-tinguishing between benign and malicious activities. Additionally, we were able to achieve F1-scores, precision, and recall that are exceeding 98%. Furthermore, we conduct a comparative analysis against traditional Multi-Layer Perceptron (MLP) networks, assessing performance metrics such as Precision, Recall, and F1-score. The results underscore the efficacy of integrating KANs with XGBoost, highlighting the potential of this innovative approach to significantly strengthen the security framework of IoT networks.

Recent years have seen a rise in the use of video sensors in remote sensing (RS) as they represent a rich source for understanding earth dynamics and human activities. Accordingly, the volume of RS video data has rapidly increased. Accessing a video of interest from large repositories through text-to-video retrieval is preferable due to its flexibility and efficiency. In this letter, we propose a text-to-event retrieval model for aerial videos. The architecture of our model consists of two branches. The first is the video branch that extracts frame-level features from the video by using the vision transformer (ViT). Then, these features are concatenated into a unified representation and fed into a temporal attention module to incorporate the temporal aspects. The second branch is the text branch that extracts textual representations from the query by the bidirectional encoder representations from transformers (BERTs). The two branches are trained jointly on video and text pairs by minimizing a bidirectional contrastive loss. Experimental results on the CapERA dataset, which is an extension of the event recognition in aerial video (ERA) dataset, show the effectiveness of the proposed method. The dataset will be available athttps://www.github.com/yakoubbazi/CapEra.

In this paper, we present an open-ended visual question answering (VQA) model for remote sensing images, where the answers can be given in the form of short sentences, unlike closed-ended VQA. This model uses a vision and natural language transformers for embedding the image and its related question. The feature representations obtained at the output are concatenated and fed to a light transformer decoder for generating the answer in an autoregressive way. The complete architecture is trained in an end-to-end manner via the backpropagation algorithm. In the experiments, we evaluate the model on a manually labeled open-ended VQA dataset termed TextRS composed of 6245 image-question pairs.

Recently, multimodal retrieval has attracted increasing attention in remote sensing community. in particular, text-image retrieval showed a promising research topic due to its ability to enable a flexible retrieval experience. To achieve this end, we propose a transformer-based multilingual text-image retrieval approach. Specifically, we employ the transformer encoder for both textual and visual modalities. At the text encoder, we jointly train four languages: English, Arabic, French, and Italian. We conduct experiments on fine-grained multimodal datasets named RSITMD. The experimental studies of the proposed method demonstrate its superior performance on both single and multiple languages modalities compared with the state-of-the-art methods.

This paper presents an efficient approach for hyperspectral image classification based on ConvMixer networks. To boost the capabilities of this network, we add an attention layer based on the idea of selective kernels (SK). This layer combines the information obtained by applying kernels of different sizes to the feature maps. The aim is to capture better the spatial and channel-wise relationships for an enhanced representation of the data. The experimental results obtained on two hyperspectral datasets: WHU-Hi-HanChuan and WHU-Hi-HongHu datasets, confirm the promising capabilities of the proposed method compared to the state-of-the-art.

Most of the classification models are built for closed set environments, where the model is trained to assign samples to a set of predefined categories. This assumption cannot be hold for models built for UAV aerial videos, where novel videos are likely to be encountered in the test phase. Dealing with unknown videos is fundamental for a reliable classification model. Therefore, in this work, we propose a model for recognizing events acquired using UAV platforms with the non-event detection property. Our model utilized the power of the attention-based model to extract discriminative spatiotemporal features from the video clips. Then, the model is trained with IsoMax loss to detect out-of-distribution videos. The proposed model is evaluated on UAV Events Recognition Dataset (ERA), and the results show that our model is able to detect non-event videos with 70.87% precision. Moreover, non-event detection has increased the accuracy of recognizing known events to 68.44%, which outperforms the accuracy of other state-of-the-art models.

Most scene classification applications in remote sensing images are addressed from a closed set-setting perspective where both the training and testing sets have the same classes. In some applications, the testing set may encounter images belonging to classes not seen during training. In this case, the classifier will face the negative transfer problem, and assign these images to one of the known classes This raises the attention to develop specific open-set methods with unknown image rejection ability. In this paper, we propose an open-set classification method based on vision transformers. An energy-based model is used to learn the density of the training data by reinterpreting the logits of the token classification head of the transformer. At test time, we reject the images with low log-likelihood scores from classification and classify all other images to their labels. The method is evaluated on Optimal-31 a remote sensing dataset, showing comparable results to the state-of-art methods.

In this letter, we propose a continual learning approach for a set of sequential scene classification tasks, where each task contains a group of land-cover classes. Our aim is to learn new tasks in a continual way without significantly degrading the performances of the old ones, due to the tricky catastrophic forgetting problem inherent to neural networks. To this end, we propose a neural architecture composed of two trainable modules. The first module learns its weights by discriminating between the land-cover classes within the new task while keeping trace of the old ones. On the other side, the second module tries to maximize the separation between the tasks by learning on task-prototypes stored in a linear memory (one prototype per task). The experimental results on two scene data sets (Merced and Optimal31) confirm the promising capability of the proposed method.

Ongoing discoveries of water reserves have fostered an increasing adoption of crop circles in the desert in several countries. Automatically quantifying and surveying the layout of crop circles in remote areas can be of great use for stakeholders in managing the expansion of the farming land. This letter compares latest deep learning models for crop circle detection and counting, namely Detection Transformers, EfficientDet and YOLOv5 are evaluated. To this end, we build two datasets, via Google Earth Pro, corresponding to two large crop circle hot spots in Egypt and Saudi Arabia. The images were drawn at an altitude of 20 km above the targets. The models are assessed in within-domain and cross-domain scenarios, and yielded plausible detection potential and inference response.

Recently, vision-language models based on transformers are gaining popularity for joint modeling of visual and textual modalities. In particular, they show impressive results when transferred to several downstream tasks such as zero and few-shot classification. In this paper, we propose a visual question answering (VQA) approach for remote sensing images based on these models. The VQA task attempts to provide answers to image-related questions. While VQA has gained popularity in computer vision, in remote sensing it is not widespread. First, we use the contrastive language image pre-training (CLIP) network for embedding the image patches and question words into a sequence of visual and textual representations. Then, we learn attention mechanisms to capture the intra-and-inter dependencies within and between these representations. Afterward, we generate the final answer by averaging the predictions of two classifiers mounted on the top of the resulting contextual representations. In the experiments, we study the performance of the proposed approach on two datasets acquired with Sentinel-2 and aerial sensors. In particular, we demonstrate that our approach can achieve better results with reduced training size compared to the recent state-of-the-art.

In this paper, we present a scene classification method based on vision transformers. These types of networks, which are now the standard models in natural language processing (NLP) do not rely on convolution block as in convolutional neural networks (CNNs). Alternatively, they are based on a mechanism known as multi-head self-attention (MSA), which captures the contextual relations between image pixels regardless of their spatial distance. At the first step, the images under analysis are split into patches, then converted to sequence by flattening and embedding. The embedding position is encoded and added to the sequence to preserve the order of the patches. Then, the resulting sequence is fed to several MSA layers for generating the final representation. To increase the classification performance, we employed several data augmentation strategies to expand the size and the diversity of the training data. Additionally, we show experimentally that we can compress the network by pruning half of its layers while keeping the competing performance. We further investigate the performance of the data-efficient image transformers (DeiT), a version of the model that is trained by knowledge distillation with less amount of data. Experimental results on two remote sensing datasets show that vision transformers can outperform state-of-the-art methods based on CNNs.

In this work, we introduce a neural architecture to unsupervised domain from multiple source domains. This architecture uses an EfficientNet as a feature extractor coupled with a set of Softmax classifiers equal to the number of source domains followed by an opportune fusion layer. To reduce the domain discrepancy between each source and target domain, we adopt a Minmax entropy approach that is based on the idea of optimizing in an adversarial manner the conditional entropy of the target samples with respect to each source classifier and minimizes it with respect to the feature extractor. As for the fusion module, we propose a weighted average fusion layer with learnable weights for aggregating the outputs of the different Softmax classifiers. Experiments on a multisource data set composed of images acquired by manned and unmanned aerial vehicles (MAVs/UAVs) over different locations are reported and discussed.

In this study, we propose an electrocardiogram (ECG) system for the simultaneous and remote monitoring of multiple heart patients. It consists of three main components: patient, sever, and monitoring units. The patient unit uses a wearable miniature sensor that continuously measures ECG signals and sends them to a smart mobile phone via a Bluetooth connection. In the mobile device, the ECG signals can be stored, displayed on screen, and automatically transmitted to a distant server unit over the internet; the server stores ECG data from several patients. Health care stakeholders use a monitoring unit to retrieve the ECG signals of multiple patients at any time from the server for display and real-time automatic analysis. The analysis includes segmentation of the ECG signal into separate heartbeats followed by arrhythmia detection and classification. When compared to existing real-time ECG systems, where the detection of abnormalities is usually performed using simple rules, the proposed system implements a real-time classification module that is based on a support vector machine (SVM) classifier. Extensive experimental results on ECG data obtained from a TechPatientTMsimulator, a real person, and 20 records from the MIT arrhythmia database are reported and discussed.

In this work, we propose a two-branch neural network architecture for multi-label classification in UAV imagery. Compared to single-label classification, the multi-label classification problem aims to assign multiple class labels to the image, which is more challenging. To deal with this issue, the proposed network optimizes in an end-to-end manner three loss functions related to image-label similarity, label discrimination, in addition to the number of labels present in the image. The experiments carried out on two UAV datasets with a spatial resolution of 2-cm confirm the promising capability of the proposed method.

The classification of land-cover classes in remote sensing images can suit a variety of interdisciplinary applications such as the interpretation of natural and man-made processes on the Earth surface. The Convolutional Support Vector Machine (CSVM) network was recently proposed as binary classifier for the detection of objects in Unmanned Aerial Vehicle (UAV) images. The training phase of the CSVM is based on convolutional layers that learn the kernel weights via a set of linear Support Vector Machines (SVMs). This paper proposes the Multi-scale Convolutional Support Vector Machine (MCSVM) network, that is an ensemble of CSVM classifiers which process patches of different spatial sizes and can deal with multi-class classification problems. The experiments are carried out on the EuroSAT Sentinel-2 dataset and the results are compared to the one obtained with recent transfer learning approaches based on pre-trained Convolutional Neural Networks (CNNs).

In this paper, we present a novel method for cross-scene classification in remote sensing images based on generative adversarial networks (GANs). To this end, we train in an adversarial manner an encoder-decoder network coupled with a discriminator network on labeled and unlabeled data coming from two different domains. The encoder-decoder network aims to reduce the discrepancy between the distributions of the two domains, while the discriminator tries to discriminate between them. At the end of the optimization process, we train an extra network on the obtained encoded labeled data and then classify the encoded unlabeled data. Experimental results on two datasets acquired over the cities of Potsdam and Vaihingen with spatial resolutions of 5cm and 9cm, respectively, confirm the promising capability of the proposed method.

We introduce the idea of orthopair membership grades and the related idea of general orthopair fuzzy sets. It is noted that these generalize the intuitionistic and Pythagorean fuzzy sets by allowing the support for and against membership to be almost anywhere in [0, 1] × [0, 1], giving systems modelers great freedom in capturing human knowledge. The aggregation of generalized orthopair fuzzy sets is considered with particular concern for the OWA and Choquet aggregation. The concepts of possibility and certainty as well as plausibility and belief are investigated in this general orthopair environment. We study arithmetic operations on general orthopair fuzzy sets. We show how to obtain associated interval valued fuzzy sets from general orthopair fuzzy sets.

In this letter, we introduce an asymmetric adaptation neural network (AANN) method for cross-domain classification in remote sensing images. Before the adaptation process, we feed the features obtained from a pretrained convolutional neural network to a denoising autoencoder (DAE) to perform dimensionality reduction. Then the first hidden layer of AANN (placed on the top of DAE) maps the labeled source data to the target space, while the subsequent layers control the separation between the available land-cover classes. To learn its weights, the network minimizes an objective function composed of two losses related to the distance between the source and target data distributions and class separation. The results of experiments conducted on six scenarios built from three benchmark scene remote sensing data sets (i.e., Merced, KSA, and AID data sets) are reported and discussed.

Nowadays, unmanned aerial vehicles (UAVs) are viewed as effective acquisition platforms for several civilian applications. They can acquire images with an extremely high level of spatial detail compared to standard remote sensing platforms. However, these images are highly affected by illumination, rotation, and scale changes, which further increases the complexity of analysis compared to those obtained using standard remote sensing platforms. In this paper, we introduce a novel convolutional support vector machine (CSVM) network for the analysis of this type of imagery. Basically, the CSVM network is based on several alternating convolutional and reduction layers ended by a linear SVM classification layer. The convolutional layers in CSVM rely on a set of linear SVMs as filter banks for feature map generation. During the learning phase, the weights of the SVM filters are computed through a forward supervised learning strategy unlike the backpropagation algorithm widely used in standard convolutional neural networks (CNNs). This makes the proposed CSVM particularly suitable for detecting problems characterized by very limited training sample availability. The experiments carried out on two UAV data sets related to vehicles and solar-panel detection issues, with a 2-cm resolution, confirm the promising capability of the proposed CSVM network compared to recent state-of-the-art solutions based on pretrained CNNs.

The accurate reconstruction of areas obscured by clouds is among the most challenging topics for the remote sensing community since a significant percentage of images archived throughout the world are affected by cloud covers which make them not fully exploitable. The purpose of this paper is to propose new methods to recover missing data in multispectral images due to the presence of clouds by relying on a formulation based on an autoencoder (AE) neural network. We suppose that clouds are opaque and their detection is performed by dedicated algorithms. The AE in our methods aims at modeling the relationship between a given cloud-free image (source image) and a cloud-contaminated image (target image). In particular, two strategies are developed: the first one performs the mapping at a pixel level while the second one at a patch level to take profit from spatial contextual information. Moreover, in order to fix the problem of the hidden layer size, a new solution combining the minimum descriptive length criterion and a Pareto-like selection procedure is introduced. The results of experiments conducted on three different data sets are reported and discussed together with a comparison with reference techniques.

In this letter, we face the problem of multilabeling unmanned aerial vehicle (UAV) imagery, typically characterized by a high level of information content, by proposing a novel method based on convolutional neural networks. These are exploited as a means to yield a powerful description of the query image, which is analyzed after subdividing it into a grid of tiles. The multilabel classification task of each tile is performed by the combination of a radial basis function neural network and a multilabeling layer (ML) composed of customized thresholding operations. Experiments conducted on two different UAV image data sets demonstrate the promising capability of the proposed method compared to the state of the art, at the expense of a higher but still contained computation time.

In this paper, we present a domain adaptation network to deal with classification scenarios subjected to the data shift problem (i.e., labeled and unlabeled images acquired with different sensors and over completely different geographical areas). We rely on the power of pretrained convolutional neural networks (CNNs) to generate an initial feature representation of the labeled and unlabeled images under analysis, referred as source and target domains, respectively. Then we feed the resulting features to an extra network placed on the top of the pretrained CNN for further learning. During the fine-tuning phase, we learn the weights of this network by jointly minimizing three regularization terms, which are: 1) the cross-entropy error on the labeled source data; 2) the maximum mean discrepancy between the source and target data distributions; and 3) the geometrical structure of the target data. Furthermore, to obtain robust hidden representations we propose a mini-batch gradient-based optimization method with a dynamic sample size for the local alignment of the source and target distributions. To validate the method, in the experiments we use the University of California Merced data set and a new multisensor data set acquired over several regions of the Kingdom of Saudi Arabia. The experiments show that: 1) pretrained CNNs offer an interesting solution for image classification compared to state-of-the-art methods; 2) their performances can be degraded when dealing with data sets subjected to the data shift problem; and 3) how the proposed approach represents a promising solution for effectively handling this issue.

In this letter, we propose a novel three-layer convex network termed as 3CN for domain adaptation in multitemporal very high resolution (VHR) remote sensing images. 3CN is composed of three main layers: 1) mapping source training samples to the target domain via a special single-layer feedforward neural network called extreme learning machine (ELM); 2) target image classification via ELM too; and 3) spatial regularization via the random-walker algorithm, which models the target image as a lattice graph and then minimizes an energy functional. This network is convex because all three layers have closed-form solutions. In the preprocessing step, we use scale-invariant feature transform to extract a set of matching key points called inliers from source and target images. Then, these inliers are used by layer 1 of 3CN to spectrally map the source training samples to the target domain. Next, in layer 2, we use the mapped training set to classify the target image. In layer 3, we exploit the spatial contextual information in the target image to reduce noise and generate an improved classification map. In the final step, we iteratively fine-tune the network to increase its discrimination ability and reduce the shift between the target and source domains. In the experiments, we report and discuss the results of the proposed method on two data sets of VHR image pairs acquired by IKONOS-2 and GeoEye-1.

In this paper, we present a new semi-supervised method for the classification of hyperspectral and VHR remote sensing images. The method is based on a hierarchical learning paradigm which is composed of multiple layers feeding into each other: 1) feature extraction layer, 2) classification layer, and 3) spatial regularization layer. In the feature extraction layer, the method employs morphological operators. In case of hyperspectral images, a dimensionality reduction step is first applied using an algorithm such PCA. In layer 2, the Extreme Learning Machine is trained and used to build an initial classification map of the image. Finally, in layer 3, a regularization step is applied to exploit spatial information between all pixels in the image. The Random Walker (RW) algorithm is used for this purpose, which uses the output results of layer 2, such as the class map and the posterior probabilities, as inputs. Initial results are obtained using the PAVIA dataset, which outperform the state-of-the-art methods in terms of accuracy and execution times.

In this paper, we propose a novel deep convex network method for domain adaptation in multitemporal remote sensing imagery. We fuse the capabilities of the extreme learning machine (ELM) classifier and local feature descriptor techniques to boost the classification accuracy. We use the Affine Scale Invariant Feature Transform (ASIFT) to extract the key points from the image pair, i.e. source and target domain images. The neural network consist of two layers, one layer uses the keypoints extracted by ASIFT to map the training points of the source image to the target image, while layer 2 is used for the purpose of classification. Experimental results obtained on multitemporal VHR images acquired by the IKONOS2 confirm the promising capability of the proposed method.

In this letter, we propose an efficient multiclass active learning (AL) method for remote sensing image classification. We fuse the capabilities of an extreme learning machine (ELM) classifier and graph-based optimization methods to boost the classification accuracy while minimizing the user interaction. First, we use the ELM to generate an initial label estimation of the unlabeled image pixels. Then, we optimize a graph-based functional energy that integrates the ELM outputs as an initial estimation of the image structure. As for the ELM, the solution to this multiclass optimization problem leads to a system of linear equations. Due to the sparse Laplacian matrix built from the lattice graph defined on the image pixels, the optimization problem is solved in a linear time. In the experiments, we report and discuss the results of the proposed AL method on two very high resolution images acquired by IKONOS-2 and GoeEye-1, as well as the well-known Pavia University hyperspectral image.

In this letter, we formulate a land-use (LU) classification problem within a compressive sensing (CS) fusion framework. CS aims at providing a compact representation form after a given query image has been processed with an opportune feature extraction type. In particular, residuals are generated from the image reconstruction with dictionaries associated with the available set of possible LUs and gathered to form a single-feature image pattern. The patterns obtained from different types of features are then fused to provide the final LU estimate. Two simple fusion strategies are adopted for such purpose. As demonstrated by experiments ran on the basis of a public benchmark database, the proposed method can achieve substantial classification accuracy gains over reference methods.

This paper introduces a new portable camera-based method for helping blind people to recognize indoor objects. Unlike state-of-the-art techniques, which typically perform the recognition task by limiting it to a single predefined class of objects, we propose here a completely different alternative scheme, defined as coarse description. It aims at expanding the recognition task to multiple objects and, at the same time, keeping the processing time under control by sacrificing some information details. The benefit is to increment the awareness and the perception of a blind person to his direct contextual environment. The coarse description issue is addressed via two image multilabeling strategies which differ in the way image similarity is computed. The first one makes use of the Euclidean distance measure, while the second one relies on a semantic similarity measure modeled by means of Gaussian process estimation. To achieve fast computation capability, both strategies rely on a compact image representation based on compressive sensing. The proposed methodology was assessed on two indoor datasets representing different indoor environments. Encouraging results were achieved in terms of both accuracy and processing time.

This paper presents a novel method to “coarsely” describe extremely high-resolution (EHR) images acquired by means of unmanned aerial vehicles (UAVs) over urban areas. Standard image analysis approaches cannot be directly exploited for the automatic description of UAV images due to their EHR. For this reason, we propose an alternative approach that consists first in the subdivision of the original UAV image in a grid of tiles. Then, each tile is compared with a library of training tiles to inherit the binary multilabel vector of the most similar training tile. This vector conveys a list of classes likely present in the considered tile. Our multiclass tile-based approach needs the definition of two main ingredients: 1) a suitable tile-representation strategy; and 2) a tile-to-tile matching operation. Various tile-representation and matching strategies are investigated. In particular, we present three global representation strategies, which process each tile as a whole and two point-based strategies that exploit points of interest within the considered tile. Regarding the matching strategies, two simple measures of distance, namely, the Euclidean and the chi-squared histogram distances, are explored. Interesting experimental results conducted on a rich set of real UAV images acquired over an urban area are reported and discussed.

In this paper, we develop an automatic method for counting palm trees in UAV images. First we extract a set of keypoints using the Scale Invariant Feature Transform (SIFT). Then, we analyze these keypoints with an Extreme Learning Machine (ELM) classifier a priori trained on a set of palm and no-palm keypoints. As output, the ELM classifier will mark each detected palm tree by several keypoints. Then, in order to capture the shape of each tree, we propose to merge these keypoints with an active contour method based on level-sets (LS). Finally, we further analyze the texture of the regions obtained by LS with local binary patterns (LBPs) to distinguish palm trees from other vegetations. Experimental results obtained on a UAV image acquired over a palm farm are reported and discussed.

Recently, a new machine learning approach that is termed as the extreme learning machine (ELM) has been introduced in the literature. This approach is characterized by a unified formulation for regression, binary, and multiclass classification problems, and the related solution is given in an analytical compact form. In this letter, we propose an efficient classification method for hyperspectral images based on this machine learning approach. To address the model selection issue that is associated with the ELM, we develop an automatic-solution-based differential evolution (DE). This simple yet powerful evolutionary optimization algorithm uses cross-validation accuracy as a performance indicator for determining the optimal ELM parameters. Experimental results obtained from four benchmark hyperspectral data sets confirm the attractive properties of the proposed DE-ELM method in terms of classification accuracy and computation time.

In this letter, we propose to solve the change detection (CD) problem in multitemporal remote-sensing images using interactive segmentation methods. The user needs to input markers related to change and no-change classes in the difference image. Then, the pixels under these markers are used by the support vector machine classifier to generate a spectral-change map. To enhance further the result, we include the spatial contextual information in the decision process using two different solutions based on Markov random field and level-set methods. While the former is a region-driven method, the latter exploits both region and contour for performing the segmentation task. Experiments conducted on a set of four real remote-sensing images acquired by low as well as very high spatial resolution sensors and referring to different kinds of changes confirm the attractive capabilities of the proposed methods in generating accurate CD maps with simple and minimal interaction.

Very high resolution (VHR) images are appreciated for their high-level details, which significantly increase their application potential. However, typically, VHR images are affected by the presence of shadows. An attempt solution to the problem of shadows is to restore shadow-contaminated regions by compensating the value of shaded pixels. Unfortunately, it may happen that not all shadow areas are possible to restore. In this paper, we propose different criteria useful to help in understanding a priori if it is possible or not to reconstruct a specific shadow area. An ideal reconstructability criterion should not tolerate that an unreconstructable shadow area is assigned as reconstructable and, at the same time, should maximize the probability of detection of reconstructable areas. Several evaluation criteria working at the pixel and textural levels are presented. Furthermore, in order to select the best criteria, a fuzzy logic combination of the criteria is explored. A thorough experimental analysis is reported and discussed. It leads to the definition of a final global index based on the fusion of two single criteria, which are the Kullback–Leibler divergence and the angular second-moment difference.

In this paper, we propose to improve the classification accuracy of hyperspectral images by fusing the capabilities of the support vector machine (SVM) classifier and the fuzzy C-means (FCM) clustering algorithm. While the former is used to generate a spectral-based classification map, the latter is adopted to provide an ensemble of clustering maps. To reduce the computation complexity, the most representative spectral channels identified by the Markov Fisher Selector (MFS) algorithm are used during the clustering process. Then, these maps are successively labeled via a pairwise relabeling procedure with respect to the SVM-based classification map using voting rules. To generate the final classification result, we propose to aggregate the obtained set of spectro-spatial maps through two different fusion methods based on voting rules and Markov Random Field (MRF) theory.

This paper proposes an interactive change detection method in multitemporal remote sensing images. The user needs to input markers related to change and no-change classes in the Difference image. Then this information is used by a support vector machine classifier to generate a spectral-change map. Then two different solutions based on Markov Random Field or Level-Set methods are used to incorporate the spatial contextual information in the decision process. While the Markov Random Field method is region driven, the level-set method exploits both region and contour for performing the segmentation task. Experiments conducted on two real remote-sensing images confirm the promising capabilities of the proposed method.

This paper proposes a novel semisupervised regression framework for estimating chlorophyll concentrations in subsurface waters from remotely sensed imagery. This framework integrates multiobjective optimization and Gaussian processes (GPs) for boosting the accuracy of the estimation process when conditioned by limited labeled-sample availability. To this end, the labeled samples are exploited in conjunction with unlabeled ones (available at zero cost from the image under analysis) for learning the regression model. The estimation of the target of these unlabeled samples is handled by the simultaneous optimization of two different criteria expressing the generalization capabilities of the GP estimator. The first is the empirical risk quantified in terms of the mean square error measure, and the second is the log marginal likelihood, which merges two terms expressing the model complexity and the data fit capability, respectively. In order to alleviate the computational burden and, possibly, to improve the estimation process accuracy, two different selection strategies of unlabeled samples are compared to the simple random-sampling procedure. They are based on the estimated variance provided by the GP estimator and the differential entropy measure, respectively. Experimental results obtained on simulated and real data sets are reported and discussed.

In this paper, we face the problem of collecting training samples for regression problems under an active learning perspective. In particular, we propose various active learning strategies specifically developed for regression approaches based on Gaussian processes (GPs) and support vector machines (SVMs). For GP regression, the first two strategies are based on the idea of adding samples that are dissimilar from the current training samples in terms of covariance measure, while the third one uses a pool of regressors in order to select the samples with the greater disagreements between the different regressors. Finally, the last strategy exploits an intrinsic GP regression outcome to pick up the most difficult and hence interesting samples to label. For SVM regression, the method based on the pool of regressors and two additional strategies based on the selection of the samples distant from the current support vectors in the kernel-induced feature space are proposed. The experimental results obtained on simulated and real data sets show that the proposed strategies exhibit a good capability to select samples that are significant for the regression process, thus opening the way to the active learning approach for remote-sensing regression problems.

In this paper, a new feature named heartbeat shape (HBS) is proposed for ECG-based biometrics. HBS is computed from the morphology of segmented heartbeats. Computation of the feature involves three basic steps: 1) resampling and normalization of a heartbeat; 2) reduction of matching error; and 3) shift invariant transformation. In order to construct both gallery and probe templates, a few consecutive heartbeats which could be captured in a reasonably short period of time are required. Thus, the identification and verification methods become efficient. We have tested the proposed feature independently on two publicly available databases with 76 and 26 subjects, respectively, for identification and verification. The second database contains several subjects having clinically proven cardiac irregularities (atrial premature contraction arrhythmia). Experiments on these two databases yielded high identification accuracy (98% and 99.85%, respectively) and low verification equal error rate (1.88% and 0.38%, respectively). These results were obtained by using templates constructed from five consecutive heartbeats only. This feature compresses the original ECG signal significantly to be useful for efficient communication and access of information in telecardiology scenarios.

This paper present a novel ensemble method for clustering very high spatial resolution (VHR) images that is composed of four main steps. Firstly, because of the important role of the spatial component in VHR imagery, a set of morphological features are extracted from the original image using many openings and closings with increasing structural element sizes. Secondly, we construct the ensemble by running the k-means algorithm several times with different initializations. In order to increase the diversity, different subsets of features are randomly selected at each time. Third, an optimal relabeling of the ensemble with respect to a representative partition is made via a pairwise relabeling procedure. Finally, the relabeled maps are fused with a Markov Random Field (MRF) method. The Experimental results obtained on two real VHR images acquired by the sensors IKONOS-2 and GeoEye-1 over urban areas confirmed the promising capabilities of the proposed approach.

Active learning is showing to be a useful approach to improve the efficiency of the classification process for remote sensing images. This letter introduces a new active learning strategy specifically developed for support vector machine (SVM) classification. It relies on the idea of the following: 1) reformulating the original classification problem into a new problem where it is needed to discriminate between significant and nonsignificant samples, according to a concept of significance which is proper to the SVM theory; and 2) constructing the corresponding significance space to suitably guide the selection of the samples potentially useful to better deal with the original classification problem. Experiments were conducted on both multi- and hyperspectral images. Results show interesting advantages of the proposed method in terms of convergence speed, stability, and sparseness.

In this paper, we propose a novel semisupervised Gaussian regression approach for the estimation of biophysical parameters from remote sensing data with limited training samples. During the learning phase, unlabeled samples are exploited to inflate the training set. The estimation of the targets associated with these samples is carried out by solving an optimization problem formulated within a genetic optimization framework. The search process of the target estimates is guided by the separate or joint optimization of two different criteria expressing the generalization capabilities of the GP estimator. The first is the empirical risk quantified in terms of the mean square error (MSE) measure; and the second is the log marginal likelihood. This last merges two terms expressing the model complexity and the data fit capability, respectively. Experimental results obtained on a real dataset representing chlorophyll concentrations in coastal waters confirm the interesting capabilities of the proposed approach.

Gaussian processes (GPs) represent a powerful and interesting theoretical framework for Bayesian classification. Despite having gained prominence in recent years, they remain an approach whose potentialities are not yet sufficiently known. In this paper, we propose a thorough investigation of the GP approach for classifying multisource and hyperspectral remote sensing images. To this end, we explore two analytical approximation methods for GP classification, namely, the Laplace and expectation-propagation methods, which are implemented with two different covariance functions, i.e., the squared exponential and neural-network covariance functions. Moreover, we analyze how the computational burden of GP classifiers (GPCs) can be drastically reduced without significant losses in terms of discrimination power through a fast sparse-approximation method like the informative vector machine. Experiments were designed aiming also at testing the sensitivity of GPCs to the number of training samples and to the curse of dimensionality. In general, the obtained classification results show clearly that the GPC can compete seriously with the state-of-the-art support vector machine classifier.

In this paper, the unsupervised change-detection problem in remote sensing images is formulated as a segmentation issue where the discrimination between changed and unchanged classes in the difference image is achieved by defining a proper energy functional. The minimization of this functional is carried out by means of a level set method which iteratively seeks to find a global optimal contour splitting the image into two mutually exclusive regions associated with changed and unchanged classes, respectively. In order to increase the robustness of the method to noise and to the choice of the initial contour, a multiresolution implementation, which performs an analysis of the difference image at different resolution levels, is proposed. The experimental results obtained on three different multitemporal remote sensing images acquired by low- as well as high-spatial-resolution optical remote sensing sensors suggest a clear superiority of the proposed approach compared with state-of-the-art change-detection methods.

In this paper, we propose a variational level-set method for unsupervised change-detection in remote sensing images. The discrimination between changed and unchanged classes in the difference image is achieved by defining an energy functional known as the piecewise constant approximation Mumford-Shah segmentation model. The minimization of this energy functional is realized according to an attractive level-set method seeking to find an optimal contour which splits the image into two mutually exclusive regions associated with changed and unchanged classes, respectively. In order to increase the robustness against the initialization issue, we adopt a multiresolution level-set approach by analyzing the difference image at different resolution levels. The experimental results obtained on two multitemporal remote sensing images acquired by low as well as very high spatial remote sensing sensors confirm the promising capabilities of the proposed approach.

In this paper, we propose an automatic method for counting olive trees in very high spatial remote sensing images. In a first step, olive trees are separated from other land-cover classes present in the image by means of a Gaussian process classifier (GPC). Due to the important role of the spatial information in very high resolution imagery, we feed the GPC with different morphological features computed from the original image. The output of this step is a binary classification map containing olive trees seen as foreground and other classes as background. In the second step, the number of blobs in the image representing possible olive trees is counted using an automatic procedure. Each blob is considered valid if its size is within a range specified a priori referring to the real size of trees. Experimental results obtained on a very high spatial remote sensing image acquired by the IKONOS-2 sensor are reported.

In this paper, a novel method for semisupervised classification with limited training samples is presented. Its aim is to exploit unlabeled data available at zero cost in the image under analysis for improving the accuracy of a classification process based on support vector machines (SVMs). It is based on the idea to augment the original set of training samples with a set of unlabeled samples after estimating their label. The label estimation process is performed within a multiobjective genetic optimization framework where each chromosome of the evolving population encodes the label estimates as well as the SVM classifier parameters for tackling the model selection issue. Such a process is guided by the joint minimization of two different criteria which express the generalization capability of the SVM classifier. The two explored criteria are an empirical risk measure and an indicator of the classification model sparseness, respectively. The experimental results obtained on two multisource remote sensing data sets confirm the promising capabilities of the proposed approach, which allows the following: (1) taking a clear advantage in terms of classification accuracy from unlabeled samples used for inflating the original training set and (2) solving automatically the tricky model selection issue.

In this paper, we explore the effectiveness of the Bayesian Gaussian process approach for classifying hyperspectral remote sensing images. In particular, we consider two analytical approximation methods for Gaussian process classification, which are the Laplace and the expectation propagation methods. Experimental results obtained on a benchmark hyperspectral dataset show that, in terms of classification accuracy, Gaussian process classification can compete seriously with the state-of-the-art classification approach based on support vector machines.

The aim of this paper is twofold. First, we present a thorough experimental study to show the superiority of the generalization capability of the support vector machine (SVM) approach in the automatic classification of electrocardiogram (ECG) beats. Second, we propose a novel classification system based on particle swarm optimization (PSO) to improve the generalization performance of the SVM classifier. For this purpose, we have optimized the SVM classifier design by searching for the best value of the parameters that tune its discriminant function, and upstream by looking for the best subset of features that feed the classifier. The experiments were conducted on the basis of ECG data from the Massachusetts Institute of Technology-Beth Israel Hospital (MIT-BIH) arrhythmia database to classify five kinds of abnormal waveforms and normal beats. In particular, they were organized so as to test the sensitivity of the SVM classifier and that of two reference classifiers used for comparison, i.e., the k-nearest neighbor (kNN) classifier and the radial basis function (RBF) neural network classifier, with respect to the curse of dimensionality and the number of available training beats. The obtained results clearly confirm the superiority of the SVM approach as compared to traditional classifiers, and suggest that further substantial improvements in terms of classification accuracy can be achieved by the proposed PSO-SVM classification system. On an average, over three experiments making use of a different total number of training beats (250, 500, and 750, respectively), the PSO-SVM yielded an overall accuracy of 89.72% on 40438 test beats selected from 20 patient records against 85.98%, 83.70%, and 82.34% for the SVM, the kNN, and the RBF classifiers, respectively.

In this paper, we present a novel multiobjective particle swarm optimization (MOPSO) approach for SVM regression with limited training samples. This approach, which is applied to the estimation of biophysical parameters from remote sensing images, is an extension of a work recently presented in the literature. It aims at exploiting unlabeled samples available from the image under analysis at zero cost to increase further the accuracy of the estimation process. The integration of such samples is made by optimizing simultaneously two criteria expressing the generalization capability of the SVM estimator, namely, the support vector count and the empirical risk. Experimental results obtained on synthetic and real multispectral data, which simulate the spectral behavior of the chlorophyll concentration in subsurface waters, are reported and discussed.

In this paper, a novel semisupervised regression approach is proposed to tackle the problem of biophysical parameter estimation that is constrained by a limited availability of training (labeled) samples. The main objective of this approach is to increase the accuracy of the estimation process based on the support vector machine (SVM) technique by exploiting unlabeled samples that are available from the image under analysis at zero cost. The integration of such samples in the regression process is controlled through a particle swarm optimization (PSO) framework that is defined by considering separately or jointly two different optimization criteria, thus leading to the implementation of three different inflation strategies. These two criteria are empirical and structural expressions of the generalization capability of the resulting semisupervised PSO-SVM regression system. The conducted experiments were focused on the problem of estimating chlorophyll concentrations in coastal waters from multispectral remote sensing images. In particular, we report and discuss results of experiments that are designed in such a way as to test the proposed approach in terms of: 1) capability to capture useful information from a set of unlabeled samples for improving the estimation accuracy; 2) sensitivity to the number of exploited unlabeled samples; and 3) sensitivity to the number of labeled samples used for supervising the inflation process

Because of the strong statistical variability of remote sensing images, the selection of the best thresholding algorithm to detect changes between two successive temporal images of the same study area without any prior knowledge is often not easy. In this paper, we face this problem through a new robust change detection approach. In order to achieve robustness, the proposed unsupervised approach is based on a Markov random field (MRF) fusion of change maps provided by an ensemble of different thresholding algorithms. Experimental results obtained on three images acquired by different sensors and referring to different kinds of changes confirm the robustness of the proposed approach.

In this letter, we propose an extension of an automatic and unsupervised change-detection method for synthetic aperture radar images we presented earlier. By analyzing a properly defined cost function, the proposed method allows the automatic detection of the number (zero, one, or two) and the values of the decision thresholds associated with changes (if any) in the log-ratio image. This cost function is the minimum value of the criterion function adopted to select the decision threshold in the log-ratio image according to a modified double-thresholding Kittler-Illingworth algorithm (implemented under the generalized Gaussian assumption for changed and unchanged classes). Experimental results carried out both on real and simulated multitemporal synthetic aperture radar images proved the effectiveness of the proposed automatic method in detecting both the number of changes to be identified (the situation of no changes is also explicitly identified) and the related threshold values

The most common methodology to carry out an automatic unsupervised change detection in remotely sensed imagery is to find the best global threshold in the histogram of the so-called difference image. The unsupervised nature of the change detection process, however, makes it nontrivial to find the most appropriate thresholding algorithm for a given difference image, because the best global threshold depends on its statistical peculiarities, which are often unknown. In this letter, a solution to this issue based on the fusion of an ensemble of different thresholding algorithms through a Markov random field framework is proposed. Experiments conducted on a set of five real remote sensing images acquired by different sensors and referring to different kinds of changes show the high robustness of the proposed unsupervised change detection approach

Recent remote sensing literature has shown that support vector machine (SVM) methods generally outperform traditional statistical and neural methods in classification problems involving hyperspectral images. However, there are still open issues that, if suitably addressed, could allow further improvement of their performances in terms of classification accuracy. Two especially critical issues are: 1) the determination of the most appropriate feature subspace where to carry out the classification task and 2) model selection. In this paper, these two issues are addressed through a classification system that optimizes the SVM classifier accuracy for this kind of imagery. This system is based on a genetic optimization framework formulated in such a way as to detect the best discriminative features without requiring the a priori setting of their number by the user and to estimate the best SVM parameters (i.e., regularization and kernel parameters) in a completely automatic way. For these purposes, it exploits fitness criteria intrinsically related to the generalization capabilities of SVM classifiers. In particular, two criteria are explored, namely: 1) the simple support vector count and 2) the radius margin bound. The effectiveness of the proposed classification system in general and of these two criteria in particular is assessed both by simulated and real experiments. In addition, a comparison with classification approaches based on three different feature selection methods is reported, i.e., the steepest ascent (SA) algorithm and two other methods explicitly developed for SVM classifiers, namely: 1) the recursive feature elimination technique and 2) the radius margin bound minimization method

We present a novel automatic and unsupervised change-detection approach specifically oriented to the analysis of multitemporal single-channel single-polarization synthetic aperture radar (SAR) images. This approach is based on a closed-loop process made up of three main steps: (1) a novel preprocessing based on a controlled adaptive iterative filtering; (2) a comparison between multitemporal images carried out according to a standard log-ratio operator; and (3) a novel approach to the automatic analysis of the log-ratio image for generating the change-detection map. The first step aims at reducing the speckle noise in a controlled way in order to maximize the discrimination capability between changed and unchanged classes. In the second step, the two filtered multitemporal images are compared to generate a log-ratio image that contains explicit information on changed areas. The third step produces the change-detection map according to a thresholding procedure based on a reformulation of the Kittler-Illingworth (KI) threshold selection criterion. In particular, the modified KI criterion is derived under the generalized Gaussian assumption for modeling the distributions of changed and unchanged classes. This parametric model was chosen because it is capable of better fitting the conditional densities of classes in the log-ratio image. In order to control the filtering step and, accordingly, the effects of the filtering process on change-detection accuracy, we propose to identify automatically the optimal number of despeckling filter iterations [Step 1] by analyzing the behavior of the modified KI criterion. This results in a completely automatic and self-consistent change-detection approach that avoids the use of empirical methods for the selection of the best number of filtering iterations. Experiments carried out on two sets of multitemporal images (characterized by different levels of speckle noise) acquired by the European Remote Sensing 2 satellite SAR sensor confirm the effectiveness of the proposed unsupervised approach, which results in change-detection accuracies very similar to those that can be achieved by a manual supervised thresholding.

This paper presents a novel approach to unsupervised change detection in multitemporal SAR images. This approach is based on three main steps: (1) controlled preprocessing based on adaptive filtering (despeckling); (2) comparison between multitemporal images according to a proper operator; (3) automatic thresholding of the log-ratio image. The first step aims at reducing the speckle noise in a controlled way in order to maximize the separability between changed and unchanged classes. The second step is devoted to compare the two filtered images in order to generate a log-ratio image. Finally, the third step deals with the identification of changes by thresholding the log-ratio image according to a novel technique. Such a technique is based on the double thresholding Kittler & Illingworth (K&I) algorithm, which is reformulated under the Generalized Gaussian (GG) assumption for the changed and unchanged classes. Experimental results obtained on a multitemporal SAR data set confirm the effectiveness of the proposed approach.