In recent years, there has been a growing interest in using image super-resolution (SR) techniques in remote sensing. These techniques aim to reconstruct high-resolution (HR) imagery from low-resolution (LR) sources. Despite the development of sophisticated SR methodologies, determining what constitutes ‘good’ SR is still a matter of debate. Present-day literature often presents SR models through a strong computer vision perspective, heavily relying on synthetic datasets. Moreover, commonly used metrics often prioritize attributes that do not necessarily correspond to improvements in spatial resolution. To address this challenge, we presentOpenSR-test, a comprehensive benchmark designed exclusively for evaluating SR of remote sensing images. Our framework incorporates specific quality metrics and curated cross-sensor datasets, each spanning various scale factors with consistent metadata. UtilizingOpenSR-test, we evaluate state-of-the-art SR algorithms from a remote sensing perspective. TheOpenSR-testframework and datasets are publicly available at https://esaopensr.github.io/opensr-test/.

In Earth observation, deep learning models rely heavily on comprehensive datasets for training and evaluation. However, the relevance of data quality is often underestimated, leading to subpar generalization in real-world remote sensing scenarios. This study aims to bridge this gap by proposing a straightforward method to identify critical human annotation errors in semantic segmentation datasets. The approach is based on two indices: trustworthiness and hardness. By implementing these indices, we estimate the extent of human annotation errors in CloudSEN12, a global dataset specifically designed for cloud detection in Sentinel-2 imagery. Considering only the trustworthiness index, our approach identified 1794 potential labelling errors among 10,000 image patches. Out of these, 106 were confirmed as human errors, resulting in a true positive rate of 9.86%. When this method was applied to other extensive cloud masking datasets, such as KappaSet and Sentinel-2 Cloud Mask Catalogue, it was found that over 44% of the human labels were inaccurate. These results do not imply the inferior quality of these datasets, instead, they highlight the considerable shift between the annotation protocols, making inter-dataset benchmarking exercises inequitable.

The future Copernicus Imaging Microwave Radiometer (CIMR) mission is planned to be launched in the 2027+ time frame. At its present phase, the first version of each Algorithm Theoretical Basis Document (ATBD) must be defined. CIMR will provide observations at L (1.4 GHz), C (6.9 GHz), X (10.65 GHz), Ku (18.7 GHz) and Ka (36.5 GHz) microwave frequencies. These observations will be relevant to develop high resolution land surface products. Here we present a preliminary study with the aim of exploring the future capabilities that the synergy of CIMR frequencies can provide. Focused on the 0th-order Tau-Omega (τ-ω) model, we analysed the influence of soil roughness (H) and scattering albedo (ω) to retrieve soil moisture (SM) and vegetation optical depth (VOD) at L-band and how these parameters can be potentially estimated from higher frequency bands. We evaluated our results over CONUS, concluding that the soil roughness (H) parameter is affecting VOD and ω mainly in non-forested areas: in those areas, the increase of H produces a decrease in VOD. Our maps of ω revealed dependence with land cover type: generally, the lowest ω values were found in forested areas. Instead, our H map yielded patterns that could be mostly associated with topographic effects. Furthermore, by utilizing a depolarization index, TBdep, we discovered that its values were constrained to nearly zero (indicating minimal soil impact) in areas with vegetation, whereas in bare soils, topography had a significant influence on TBdep. We hypothesize that the use of this index could help in finding relationships among the multi-frequency information from CIMR, allowing us to understand the degree of sensitivity of each band to vegetation and topography.

This paper introduces DTACSNet, a Convolutional Neural Network (CNN) model specifically developed for efficient onboard atmospheric correction and cloud detection in optical Earth observation satellites. The model is developed with Sentinel-2 data. Through a comparative analysis with the operational Sen2Cor processor, DTACSNet demonstrates a significantly better performance in cloud scene classification (F2 score of 0.89 for DTACSNet compared to 0.51 for Sen2Cor v2.8) and a surface reflectance estimation with average absolute error below 2% in reflectance units. Moreover, we tested DTACSNet on hardware-constrained systems similar to recent deployed missions and show that DTACSNet is 11 times faster than Sen2Cor with a significantly lower memory consumption footprint. These preliminary results highlight the potential of DTACSNet to provide enhanced efficiency, autonomy, and responsiveness in onboard data processing for Earth observation satellite missions.

Clouds play a key role in regulating climate change but are difficult to simulate within Earth system models (ESMs). Improving the representation of clouds is one of the key tasks toward more robust climate change projections. This study introduces a new machine-learning-based framework relying on satellite observations to improve understanding of the representation of clouds and their relevant processes in climate models. The proposed method is capable of assigning distributions of established cloud types to coarse data. It facilitates a more objective evaluation of clouds in ESMs and improves the consistency of cloud process analysis. The method is built on satellite data from the Moderate Resolution Imaging Spectroradiometer (MODIS) instrument labeled by deep neural networks with cloud types defined by the World Meteorological Organization (WMO), using cloud-type labels from CloudSat as ground truth. The method is applicable to datasets with information about physical cloud variables comparable to MODIS satellite data and at sufficiently high temporal resolution. We apply the method to alternative satellite data from the Cloud_cci project (ESA Climate Change Initiative), coarse-grained to typical resolutions of climate models. The resulting cloud-type distributions are physically consistent and the horizontal resolutions typical of ESMs are sufficient to apply our method. We recommend outputting crucial variables required by our method for future ESM data evaluation. This will enable the use of labeled satellite data for a more systematic evaluation of clouds in climate models.

The present study focuses on retrieving soil and canopy temperatures, which are key parameters to estimate soil moisture and vegetation optical depth from multi-frequency microwaves information. Several retrieval algorithms assume that canopy and vegetation temperatures are similar in thermal equilibrium conditions, while others separate their contributions, as SMOS-IC, one of the consolidated retrieval algorithms for the Soil Moisture and Ocean Salinity (SMOS) satellite mission. Soil and canopy temperatures in SMOS-IC are modelled from the ECMWF (European Centre for Medium-Range Weather Forecasts) centre. Both SMOS and the Soil Moisture Active Passive (SMAP) missions are currently the only passive L-band (1.4 GHz) missions in operation, but their lifetime is limited. In this context, the upcoming Copernicus Imaging Microwave Radiometer (CIMR) mission will provide continuity on L-band measurements with complementary information in a range of microwave frequencies, from 1.4 to 36.5 GHz. This study uses in situ soil moisture information from the International Soil Moisture Network (ISMN) as input in the SMOS-IC algorithm to retrieve vegetation optical depth (VOD) and soil/canopy effective temperature (TGC). The retrieved effective temperature is then compared with modelled temperatures from ECMWF and with data from the Advanced Microwave Scanning Radiometer 2 (AMSR2), which acquires the higher frequency bands (C, X, Ka, and Ku) present in the future CIMR mission. Results confirm the potential of all high-frequency bands to estimate TGC, with C and X-bands being the most correlated. This study is a first approach to evaluate how microwave multi-frequency information can help modelling soil and canopy temperatures in the SMOS-IC retrieval algorithm, from which the upcoming CIMR mission may benefit.

With the objective to improve data quality in terms of cloud detection, absolute radiometric calibration and atmospheric correction, the PRoject for On-Board Autonomy-Vegetation (PROBA-V) data archive (October 2013 - June 2020) will be reprocessed to Collection 2 (C2). The product validation is organized in three phases and focuses on the intercomparison with PROBA-V Collection 1 (C1), but also consistency analysis with SPOT-VGT, Sentinel-3 SYN-VGT, Terra-MODIS and METOP-AVHRR is foreseen. First preliminary results show the better performance of cloud and snow/ice masking, and indicate that statistical consistency between PROBA-V C2 and C1 are in line with expectations. PROBA-V C2 data are to be released to the public in September 2021.

In this work, we propose to exploit both the temporal and spatial correlations in Earth observation satellite images through deep learning methods. In particular, the combination of a U-Net convolutional neural network together with a convolutional long short-term memory (LSTM) layer is proposed. This model is applied for cloud detection on MSG/SEVIRI image time series over selected landmarks. Implementation details are provided and our proposal is compared against a standard SVM and a U-Net without the convolutional LSTM layer but including temporal information too. Experimental results show that this combination of networks exploits both the spatial and temporal dependence and provides state-of-the-art classification results on this dataset.

Training machine learning algorithms for new satellites requires collecting new data. This is a critical drawback for most remote sensing applications and specially for cloud detection. A sensible strategy to mitigate this problem is to exploit available data from a similar sensor, which involves transforming this data to resemble the new sensor data. However, even taking into account the technical characteristics of both sensors to transform the images, statistical differences between data distributions still remain. This results in a poor performance of the methods trained on one sensor and applied to the new one. In this this work, we propose to use the generative adversarial networks (GANs) framework to adapt the data from the new satellite. In particular, we use Landsat-8 images, with the corresponding ground truth, to perform cloud detection in Proba-V. Results show that the GANs adaptation significantly improves the detection accuracy.

Cloud detection is a key issue for exploiting the information from Earth observation satellites multispectral sensors. For Proba-V, cloud detection is challenging due to the limited number of spectral bands. Advanced machine learning methods, such as convolutional neural networks (CNN), have shown to work well on this problem provided enough labeled data. However, simultaneous collocated information about the presence of clouds is usually not available or requires a great amount of manual labor. In this work, we propose to learn from the available Landsat -8 cloud masks datasets and transfer this learning to solve the Proba-V cloud detection problem. CNN are trained with Landsat images adapted to resemble Proba-V characteristics and tested on a large set of real Proba-V scenes. Developed models outperform current operational Proba-V cloud detection without being trained with any real Proba-V data. Moreover, cloud detection accuracy can be further increased if the CNN are fine-tuned using a limited amount of Proba-V data.

Kernel methods have been very successful in remote sensing problems because of their ability to deal with high dimensional non-linear data. However, they are computationally expensive to train when a large amount of samples are used. In this context, while the amount of available remote sensing data has constantly increased, the size of training sets in kernel methods is usually restricted to few thousand samples. In this work, we modified the kernel ridge regression (KRR) training procedure to deal with large scale datasets. In addition, the basis functions in the reproducing kernel Hilbert space are defined as parameters to be also optimized during the training process. This extends the number of free parameters from two (in the standard KRR with an RBF kernel) to more than fifty thousand in our experiments. The effectiveness of the proposal is illustrated in the problem of surface temperature estimation from MetOp-IASI hyperspectral infrared sounding data. The data set used contains more than one million samples, but the proposed method could potentially be trained with much more data.

This paper presents the development and implementation of a cloud detection algorithm for Proba-V. Accurate and automatic detection of clouds in satellite scenes is a key issue for a wide range of remote sensing applications. With no accurate cloud masking, undetected clouds are one of the most significant sources of error in both sea and land cover biophysical parameter retrieval. The objective of the algorithms presented in this paper is to detect clouds accurately providing a cloud flag per pixel. For this purpose, the method exploits the information of Proba-V using statistical machine learning techniques to identify the clouds present in Proba-V products. The effectiveness of the proposed method is successfully illustrated using a large number of real Proba-V images.

Emissivity is one of the most important parameters to improve the determination of the troposphere properties (thermodynamic properties, aerosols and trace gases concentration) and it is essential to estimate the radiative budget. With the second generation of infrared sounders, we can estimate emissivity spectra at high spectral resolution, which gives us a global view and long-term monitoring of continental surfaces. Statistically, this is an ill-posed retrieval problem, with as many output variables as inputs. We here propose nonlinear multi-output statistical regression based on kernel methods to estimate spectral emissivity given the radiances. Kernel methods can cope with high-dimensional input-output spaces efficiently. We give empirical evidence of models performance on Infrared Atmospheric Sounding Interferometer (IASI) simulated data. Kernel regression model largely improves previous least squares linear regression model quantitatively, with an average reduction of 25% in mean-square error.

Convolutional neural networks (CNN) have proven to be state of the art methods for many image classification tasks and their use is rapidly increasing in remote sensing problems. One of their major strengths is that, when enough data is available, CNN perform an end-to-end learning without the need of custom feature extraction methods. In this work, we study the use of different CNN architectures for cloud masking of Proba-V multispectral images. We compare such methods with the more classical machine learning approach based on feature extraction plus supervised classification. Experimental results suggest that CNN are a promising alternative for solving cloud masking problems.

The vast amount of data acquired by current high resolution Earth observation satellites implies some technical challenges to be faced. Google Earth Engine (GEE) platform provides a framework for the development of algorithms and products built over this data in an easy and scalable manner. In this paper, we take advantage of the GEE platform capabilities to exploit the wealth of information in the temporal dimension by processing a long time series of satellite images. A cloud detection algorithm for Landsat-8, which uses previous images of the same location to detect clouds, is implemented and tested on the GEE platform.

This brief addresses two main issues of the standard kernel entropy component analysis (KECA) algorithm: the optimization of the kernel decomposition and the optimization of the Gaussian kernel parameter. KECA roughly reduces to a sorting of the importance of kernel eigenvectors by entropy instead of variance, as in the kernel principal components analysis. In this brief, we propose an extension of the KECA method, named optimized KECA (OKECA), that directly extracts the optimal features retaining most of the data entropy by means of compacting the information in very few features (often in just one or two). The proposed method produces features which have higher expressive power. In particular, it is based on the independent component analysis framework, and introduces an extra rotation to the eigen decomposition, which is optimized via gradient-ascent search. This maximum entropy preservation suggests that OKECA features are more efficient than KECA features for density estimation. In addition, a critical issue in both the methods is the selection of the kernel parameter, since it critically affects the resulting performance. Here, we analyze the most common kernel length-scale selection criteria. The results of both the methods are illustrated in different synthetic and real problems. Results show that OKECA returns projections with more expressive power than KECA, the most successful rule for estimating the kernel parameter is based on maximum likelihood, and OKECA is more robust to the selection of the length-scale parameter in kernel density estimation.

This paper deals with the development and implementation of a cloud screening algorithm for image time series, with the focus on the forthcoming Sentinel-2 satellites to be launched under the ESA Copernicus Programme. The proposed methodology is based on kernel ridge regression and exploits the temporal information to detect anomalous changes that correspond to cloud covers. The huge data volumes to be processed when dealing with high temporal, spatial, and spectral resolution datasets motivate the implementation of the algorithm within distributed computer resources. In consequence, an operational cloud screening service has been specifically designed and implemented in the frame of the Sentinels Synergy Framework (SenSyF). The effectiveness of the proposed method is successfully illustrated using a time series dataset with a 5-day revisit derived from SPOT-4 at high resolution, which has been collected by ESA in preparation for the exploitation of the Sentinel-2 mission.

Earth observation through remote sensing images allows the accurate characterization and identification of materials on the surface from space and airborne platforms. Multiple and heterogeneous image sources can be available for the same geographical region: multispectral, hyperspectral, radar, multitemporal, and multiangular images can today be acquired over a given scene. These sources can be combined/fused to improve classification of the materials on the surface. Even if this type of systems is generally accurate, the field is about to face new challenges: the upcoming constellations of satellite sensors will acquire large amounts of images of different spatial, spectral, angular, and temporal resolutions. In this scenario, multimodal image fusion stands out as the appropriate framework to address these problems. In this paper, we provide a taxonomical view of the field and review the current methodologies for multimodal classification of remote sensing images. We also highlight the most recent advances, which exploit synergies with machine learning and signal processing: sparse methods, kernel-based fusion, Markov modeling, and manifold alignment. Then, we illustrate the different approaches in seven challenging remote sensing applications: 1) multiresolution fusion for multispectral image classification; 2) image downscaling as a form of multitemporal image fusion and multidimensional interpolation among sensors of different spatial, spectral, and temporal resolutions; 3) multiangular image classification; 4) multisensor image fusion exploiting physically-based feature extractions; 5) multitemporal image classification of land covers in incomplete, inconsistent, and vague image sources; 6) spatiospectral multisensor fusion of optical and radar images for change detection; and 7) cross-sensor adaptation of classifiers. The adoption of these techniques in operational settings will help to monitor our planet from space in the very near future.

This paper presents a novel semisupervised kernel partial least squares (KPLS) algorithm for nonlinear feature extraction to tackle both land-cover classification and biophysical parameter retrieval problems. The proposed method finds projections of the original input data that align with the target variable (labels) and incorporates the wealth of unlabeled information to deal with low-sized or underrepresented data sets. The method relies on combining two kernel functions: the standard radial-basis-function kernel based on labeled information and a generative, i.e., probabilistic, kernel directly learned by clustering the data many times and at different scales across the data manifold. The construction of the kernel is very simple and intuitive: Two samples should belong to the same class if they consistently belong to the same clusters at different scales. The effectiveness of the proposed method is successfully illustrated in multi- and hyperspectral remote sensing image classification and biophysical parameter estimation problems. Accuracy improvements in the range between +5% and 15% over standard principal component analysis (PCA), +4% and 15% over kernel PCA, and +3% and 10% over KPLS are obtained on several images. The average gain in the root-mean-square error of +5% and reductions in bias estimates of +3% are obtained for biophysical parameter retrieval compared to standard PCA feature extraction.

This paper presents a multitemporal feature extraction method based on kernels that is particularly designed for change detection. The method provides features that maximize specific changes between two dates while minimizing sources of errors, such as residual land-cover changes and misregistration errors, in the time series. The extracted features computed in the kernel feature space can deal with non-linear relations between samples at different dates. Moreover, no supervised information is required to find the changes of interest for the selected dates in the time series. The effectiveness of the proposed method is successfully illustrated in a cloud masking application using a Landsat time series. Results show that the proposed method provides the most discriminative features in terms of cloud detection when confronted with state of the art linear and nonlinear unsupervised feature extraction algorithms. In particular, extracted features with the proposed method enable automatic cloud detection in multispectral time series.

This paper presents a synergistic cloud detection algorithm that has been developed for processing simultaneous observations from AATSR and MERIS sensors on-board ENVISAT. The main objective of this work is to explore sensor synergies in order to increase the cloud detection accuracy and provide a reliable cloud mask. This is of paramount importance in the framework of the ESA climate change initiative for clouds (Cloud CCI), which aims to provide long time series of cloud properties at a global scale from satellite data. The cloud detection algorithm is based on an ensemble of artificial neural networks, where the outputs of different dedicated models are combined to provide more accurate and robust predictions. The performance of the method has been tested on a large number of real images, and provides higher classification accuracy than other methods, especially when spatial information from the images is included in the classifiers.

This letter introduces a simple method for including invariances in support-vector-machine (SVM) remote sensing image classification. We design explicit invariant SVMs to deal with the particular characteristics of remote sensing images. The problem of including data invariances can be viewed as a problem of encoding prior knowledge, which translates into incorporating informative support vectors (SVs) that better describe the classification problem. The proposed method essentially generates new (synthetic) SVs from the obtained by training a standard SVM with the available labeled samples. Then, original and transformed SVs are used for training the virtual SVM introduced in this letter. We first incorporate invariances to rotations and reflections of image patches for improving contextual classification. Then, we include an invariance to object scale in patch-based classification. Finally, we focus on the challenging problem of including illumination invariances to deal with shadows in the images. Very good results are obtained when few labeled samples are available for classification. The obtained classifiers reveal enhanced sparsity and robustness. Interestingly, the methodology can be applied to any maximum-margin method, thus constituting a new research opportunity.

Many remote sensing data processing problems are inherently constituted by several tasks that can be solved either individually or jointly. For instance, each image in a multitemporal classification setting could be taken as an individual task. Here, the relation to previous acquisitions should be properly considered because of the nonstationary behavior of temporal, spatial, and angular image features which gives rise to distribution changes. This phenomenon is known as covariate shift. Additionally, when labeled data are scarce or expensive to obtain, the small sample-set problem arises, which makes solving the problems independently in each domain difficult. Multitask learning (MTL) aims at jointly solving a set of prediction problems by sharing information across tasks. This paper introduces MTL in remote sensing data classification. The proposed methods alleviate the data set shift by imposing cross-information in the classifiers through matrix regularization. We consider the support vector machine (SVM) as the core learner and two different regularization schemes: 1) the inclusion of relational operators between tasks and 2) the pairwise Euclidean distance of the predictors in the Hilbert space. These methods rely on simple and intuitive modifications of the kernel used in the standard SVM. Experiments are conducted in three challenging remote sensing problems: cloud screening from multispectral images, land-mine detection using radar data, and multitemporal and multisource image classification. The pairwise method consistently outperforms standard independent and aggregate approaches by about +2% to 4% in all problems at no additional cost. Also, the solutions found give us information about the distribution shift among tasks.

We present an adaptation algorithm focused on the description of the data changes under different acquisition conditions. When considering a source and a destination domain, the adaptation is carried out by transforming one data set to the other using an appropriate nonlinear deformation. The eventually nonlinear transform is based on vector quantization and graph matching. The transfer learning mapping is defined in an unsupervised manner. Once this mapping has been defined, the samples in one domain are projected onto the other, thus allowing the application of any classifier or regressor in the transformed domain. Experiments on challenging remote sensing scenarios, such as multitemporal very high resolution image classification and angular effects compensation, show the validity of the proposed method to match-related domains and enhance the application of cross-domains image processing techniques.

Feature extraction is of paramount importance for an accurate classification of remote sensing images. Techniques based on data transformations are widely used in this context. However, linear feature extraction algorithms, such as the principal component analysis and partial least squares, can address this problem in a suboptimal way because the data relations are often nonlinear. Kernel methods may alleviate this problem only when the structure of the data manifold is properly captured. However, this is difficult to achieve when small-size training sets are available. In these cases, exploiting the information contained in unlabeled samples together with the available training data can significantly improve data description by defining an effective semisupervised nonlinear feature extraction strategy. We present a novel semisupervised Kernel Partial Least Squares (KPLS) algorithm for non-linear feature extraction. The method relies on combining two kernel functions: the standard RBF kernel using labeled information and a generative kernel directly learned by clustering the data. The effectiveness of the proposed method is successfully illustrated in multi- and hyper-spectral remote sensing image classification: accuracy improvements between +15 - 20% over standard PCA and +10% over advanced kernel PCA and KPLS for both images is obtained. Matlab code is available at http://isp.uv.es for the interested readers.

This paper introduces a simple method to include invariances in support vector machine (SVM) for remote sensing image classification. We rely on the concept of virtual support vectors, by which the SVM is trained with both the selected support vectors and synthetic examples encoding the invariance of interest. The algorithm is very simple and effective, as demonstrated in two particularly interesting examples: invariance to the presence of shadows and to rotations in patchbased image segmentation. The improved accuracy (around +6% both in OA and Cohen's κ statistic), along with the simplicity of the approach encourage its use and extension to encode other invariances and other remote sensing data analysis applications.

This paper evaluates nonlinear retrieval methods to derive atmospheric properties from hyperspectral infrared sounding spectra, with emphasis on the retrieval of temperature, humidity, and ozone atmospheric profiles. We concentrate on the Infrared Atmospheric Sounding Interferometer (IASI) onboard the MetOp-A satellite data for the future Meteosat Third Generation Infrared Sounder (MTG-IRS). The methods proposed in this work are compared in terms of both accuracy and speed with the current MTG-IRS L2 processing concept, which processes MetOp-IASI and proxy MTG-IRS data. The official chain consists of a principal component extraction, typically referred to as empirical orthogonal functions (EOF) and a subsequent canonical linear regression. This research proposes the evaluation of some other methodological advances considering: 1) other linear feature extraction methods instead of EOF, such as partial least squares; and 2) the linear combination of nonlinear regression models in the form of committee of experts. The nonlinear regression models considered in this work are artificial neural networks and kernel ridge regression as nonparametric multioutput powerful regression tools. Results show that, in general, nonlinear models yield better results than linear retrieval for both MetOp-IASI and MTG-IRS synthetic and real data. Averaged gains throughout the column of +1.8 K and +2.2 K are obtained for temperature profile estimation from MetOp-IASI and IRS data, respectively. Similar gains are obtained for the estimation of dew point temperatures. In both variables, these improvements are more noticeable in lower atmospheric layers. The combination of models makes the retrieval more robust, improves the accuracy, and decreases the estimated bias. The nonlinear statistical approach is successfully compared to optimal estimation (OE) in terms of accuracy, bias and computational cost. These results confirm the potential of statistical nonlinear inversion techniques for the retrieval of atmospheric profiles.

This paper proposes the use of kernel ridge regression (KRR) to derive surface and atmospheric properties from hyperspectral infrared sounding spectra. We focus on the retrieval of temperature and humidity atmospheric profiles from Infrared Atmospheric Sounding Interferometer (MetOp-IASI) data, and provide confidence maps on the predictions. In addition, we propose a scheme for the identification of anomalies by supervised classification of discrepancies with the ECMWF estimates. For the retrieval, we observed that KRR clearly outperformed linear regression. Looking at the confidence maps, we observed that big discrepancies are mainly due to the presence of clouds and low emissivities in desert areas. For the identification of anomalies, we observed that the confidence intervals provided by the KRR may help in discarding big errors. High detection accuracy (around 90%) is achieved by a support vector machine, which largely outperforms standard linear and nonlinear classifiers.

This paper proposes the kernel entropy component analysis (KECA) for clustering remote sensing data. The method generates nonlinear features that reveal structure related to the Renyi entropy of the input space data set. Unlike other kernel feature extraction methods, the top eigenvalues and eigenvectors of the kernel matrix are not necessarily chosen. Data are interestingly mapped with a distinct angular structure, which is exploited to derive a new angle-based spectral clustering algorithm based on the mapped data. An out-of-sample extension of the method is also presented to deal with test data. We focus on cloud screening from MERIS images. Several images are considered to account for the high variability of the problem. Good results show the suitability of the proposal.

This paper introduces a nonlinear feature extraction method based on kernels for remote sensing data analysis. The proposed approach is based on the minimum noise fraction (MNF) transform, which maximizes the signal variance while also minimizing the estimated noise variance. We here propose an alternative kernel MNF (KMNF) in which the noise is explicitly estimated in the reproducing kernel Hilbert space. This enables KMNF dealing with non-linear relations between the noise and the signal features jointly. Results show that the proposed KMNF provides the most noise-free features when confronted with PCA, MNF, KPCA, and the previous version of KMNF. Extracted features with the explicit KMNF also improve hyperspectral image classification.

Earth observation satellites currently provide a large volume of images at different scales. Most of these satellites provide global coverage with a revisit time that usually depends on the instrument characteristics and performance. Typically, medium-spatial-resolution instruments provide better spectral and temporal resolutions than mapping-oriented high-spatial-resolution multispectral sensors. However, in order to monitor a given area of interest, users demand images with the best resolution available, which cannot be reached using a single sensor. In this context, image fusion may be effective to merge information from different data sources. In this letter, an image fusion approach based on multiresolution and multisource spatial unmixing is used to obtain a composite image with the spectral and temporal characteristics of medium-spatial-resolution instrument along with the spatial resolution of high-spatial-resolution image. A time series of Landsat/TM and ENVISAT/MERIS Full Resolution images acquired in the 2004 European Space Agency (ESA) Spectra Barrax Campaign illustrates the method's capabilities. The qualitative and quantitative assessments of the product images are given. The proposed methodology is general enough to be applied to similar sensors, such as the multispectral instruments which will fly on board the ESA GMES Sentinel-2 and Sentinel-3 upcoming satellite series.

Earth observation satellites provide a valuable source of data which when conveniently processed can be used to better understand the Earth system dynamics. In this regard, one of the prerequisites for the analysis of satellite image time series is that the images are spatially coregistered so that the resulting multitemporal pixel entities offer a true temporal view of the area under study. This implies that all the observations must be mapped to a common system of grid cells. This process is known as gridding and, in practice, two common grids can be used as a reference: 1) a grid defined by some kind of external data set (e.g., an existing land-cover map) or 2) a grid defined by one of the images of the time series. The aim of this paper is to study the impact that gridding has on the quality of satellite time series. More precisely, the impact of the so-called gridding artifacts is quantified using a time series of 12 images acquired over The Netherlands by the Medium Resolution Imaging Spectrometer (MERIS). First, the impact of selecting a reference grid is evaluated in terms of geolocation errors and pixel overlap. Then, the effect of observation geometry is studied as nongeostationary satellites, like MERIS, can acquire images from the same area from a number of orbits. Finally, a high-resolution land-cover data set is used to account for temporal information consistency (pixel homogeneity in terms of land-cover composition). Results have shown an average pixel overlap with the nearest pixel between 20% and 41% depending on the selected reference grid and on the differences in observation geometry. These results indicate that inappropriate gridding might result in collocated time series that are not adequate for temporal studies at pixel level (particularly over nonhomogeneous areas) and that, in any case, it is interesting to identify areas with low pixel overlap in order to further analyze the reliability of the products derived over these areas.

Data from current medium-spatial-resolution imaging spectroradiometers are used for land-cover mapping and land-cover change detection at regional to global scales. However, few landscapes are homogeneous at these scales, and this creates the so-called mixed-pixel problem. In this context, this study explores the use of the linear spectral mixture model to extract subpixel land-cover composition from medium-spatial-resolution data. In particular, a time series of MEdium Resolution Imaging Spectrometer (MERIS) full-resolution (FR; pixel size of 300 m) images acquired over The Netherlands is used to illustrate this study. The Netherlands was selected because of the following: 1) the fragmentation of its landscapes and 2) the availability of a high-spatial-resolution land-cover data set (LGN5) which can be used as a reference. The question then is to what extent a multitemporal unmixing of MERIS FR data delivers land-cover information comparable with the one provided by the LGN5. To this end, fully constrained linear spectral unmixing is applied to each individual MERIS image and to the multitemporal composite. The unmixing results are validated at both subpixel and per-pixel scales and at two thematic aggregation levels (12 and 4 land-cover classes). The obtained results indicate that the described unmixing approach yields moderate results for the 12-class case and good results for the 4-class case. These results might be explained by MERIS preprocessing steps, gridding effects, vegetation phenophases, and spectral class separability.

Nowadays, the increasing quantity of applications using images from Earth Observation satellites makes demanding better spatial, spectral and temporal resolutions. Nevertheless, due to the technical constraint of a trade off between spatial and spectral resolutions, and between spatial resolution and coverage, high spatial resolution is related with low spectral and temporal resolutions and vice versa. Data fusion methods are a good solution to combine information from multiple sensors in order to obtain image products with better characteristics. In this paper, we propose an image fusion approach based on a multi-resolution and multi-source unmixing. The proposed methodology yields a composite image with the spatial resolution of the higher resolution image (downscaling) while retaining the spectral and temporal characteristics of the medium spatial resolution image. The approach is tested in the specific cases of ENVISAT/MERIS and Landsat/TM instruments, but is general enough to be applied to other sensor combination.

Remote sensing image classification constitutes a challenging problem since very few labeled pixels are typically available from the analyzed scene. In such situations, labeled data extracted from other images modeling similar problems might be used to improve the classification accuracy. However, when training and test samples follow even slightly different distributions, classification is very difficult. This problem is known assample selection bias. In this paper, we propose a new method to combine labeled and unlabeled pixels to increase classification reliability and accuracy. A semisupervised support vector machine classifier based on the combination of clustering and themean mapkernel is proposed. The method reinforces samples in the same cluster belonging to the same class by combining sample and cluster similarities implicitly in the kernel space. Asoftversion of the method is also proposed where only the most reliable training samples, in terms of likelihood of the image data distribution, are used. Capabilities of the proposed method are illustrated in a cloud screening application using data from the MEdium Resolution Imaging Spectrometer (MERIS) instrument onboard the European Space Agency ENVISAT satellite. Cloud screening constitutes a clear example of sample selection bias since cloud features change to a great extent depending on the cloud type, thickness, transparency, height, and background. Good results are obtained and show that the method is particularly well suited for situations where the available labeled information does not adequately describe the classes in the test data.

This paper presents two semisupervised one-class support vector machine (OC-SVM) classifiers for remote sensing applications. Inone-classimage classification, one tries to detect pixels belonging to one of the classes in the image and reject the others. When few labeled pixels of only one class are available, obtaining a reliable classifier is a difficult task. In the particular case of SVM-based classifiers, this task is even harder because the free parameters of the model need to be finely adjusted, but no clear criterion can be adopted. In order to improve the OC-SVM classifier accuracy and alleviate the problem of free-parameter selection, the information provided by unlabeled samples present in the scene can be used. In this paper, we present two state-of-the-art algorithms for semisupervised one-class classification for remote sensing classification problems. The first proposed algorithm is based on modifying the OC-SVM kernel by modeling the data marginal distribution with the graph Laplacian built with both labeled and unlabeled samples. The second one is based on a simple modification of the standard SVM cost function which penalizes more the errors made when classifying samples of the target class. The good performance of the proposed methods is illustrated in four challenging remote sensing image classification scenarios where the goal is to detect one of the classes present on the scene. In particular, we present results for multisource urban monitoring, hyperspectral crop detection, multispectral cloud screening, and change-detection problems. Experimental results show the suitability of the proposed techniques, particularly in cases with few or poorly representative labeled samples.

The project CHRIS/Proba Toolbox for BEAM (CHRIS-Box) has been developed in order to support users of data from the CHRIS sensor onboard of ESA's Proba platform. BEAM and the CHRIS-Box are user tools which ESA/ESRTN are providing free of charge to the Earth Observation Community. The CHRIS-Box software provides extensions for BEAM that allows accomplishing the following tasks: a) Noise reduction to remove the vertical striping and other noise present in CHRIS response-corrected images; b) Cloud screening to mark cloudy pixels in CHRIS noise-corrected images; the cloud screening algorithm provides cloud probability and abundances for each pixel; c) Atmospheric correction that provides surface reflectance without external information; and d) Geometric correction that provides geographic coordinates for each pixel of a CHRIS image.

We evaluate Gaussian Processes (GPs) for the estimation of biophysical parameters from acquired multispectral data. The standard GP formulation is used, and all hyperparameters (kernel parameters and noise variance) are optimized by maximizing the marginal likelihood. This gives rise to a fully-adaptive GP to data characteristics, both in terms of signal and noise properties. The good numerical results in the estimation of oceanic chlorophyll concentration and leaf membrane state confirm GPs as adequate, alternative non-parametric methods for biophysical parameter estimation. GPs are also analyzed by scrutinizing the predictive variance, the estimated noise variance, and the relevance of each feature after optimization.

This paper presents a cloud screening algorithm based on ensemble methods that exploits the combined information from both MERIS and AATSR instruments on board ENVISAT in order to improve current cloud masking products for both sensors. The first step is to analyze the synergistic use of MERIS and AATSR images in order to extract some physically-based features increasing the separability of clouds and surface. Then, several artificial neural networks are trained using different sets of input features and different sets of training samples depending on acquisition and surface conditions. Finally, outputs of the trained neural networks are combined at the decision level to construct a more accurate and robust ensemble of classifiers. The proposed classifier is tested on more than 80 coregistered MERIS/AATSR images providing better classification accuracy than the official cloud flags and available operational cloud screening algorithms for MERIS and AATSR. Moreover, thanks to the synergy of both sensors, it correctly classifies critical cloud-screening problems such as snow and ice covers over land and sun-glint over ocean.

This letter presents two kernel-based methods for semisupervised regression. The methods rely on building a graph or hypergraph Laplacian with both the available labeled and unlabeled data, which is further used to deform the training kernel matrix. The deformed kernel is then used for support vector regression (SVR). Given the high computational burden involved, we present two alternative formulations based on the Nystrom method and the incomplete Cholesky factorization to achieve operational processing times. The semisupervised SVR algorithms are successfully tested in multiplatform leaf area index estimation and oceanic chlorophyll concentration prediction. Experiments are carried out with both multispectral and hyperspectral data, demonstrating good generalization capabilities when a low number of labeled samples are available, which is usually the case in biophysical parameter retrieval.

Two kernel-based methods for semi-supervised regression are presented. The methods rely on building a graph or hypergraph Laplacian with both the labeled and unlabeled data, which is further used to deform the training kernel matrix. The deformed kernel is then used for support vector regression (SVR). The semi-supervised SVR methods are sucessfully tested in LAI estimation and ocean chlorophyll concentration prediction from remotely sensed images.

This paper presents a semi-supervised classifier based on the combination of the expectation-maximization (EM) algorithm for Gaussian mixture models (GMM) and the mean map kernel. The proposed method uses the most reliable samples in terms of maximum likelihood to compute a kernel function that accurately reflects the similarity between clusters in the kernel space. The proposed method improves classification accuracy in situations where the available labeled information does not properly describe the classes in the test image.

FLEX (FLuorescence EXperiment) is a candidate mission for the European Space Agency (ESA) Earth Explorer program. The main objective of the mission is the measurement the chlorophyll fluorescence signal emitted by vegetation at the red and far-red spectral regions (roughly 630-770 nm). The current FLEX mission design includes different instruments intended to provide the appropriate characterization of those atmospheric and surface parameters necessary for the retrieval and interpretation of the fluorescence signal. The complete processing chain for the derivation of fluorescence and reflectance products from the radiance data acquired by the different instruments included in the FLEX pay-load is described in this paper. Six processing modules have been implemented: cloud screening, aerosol optical thickness (AOT) retrieval, automatic spectral characterisation, columnar water vapor (CWV) retrieval, fluorescence retrieval and reflectance retrieval. The processing chain has been tested against a scene-based simulated data set which reproduces FLEX instruments and realistic atmospheric conditions.

This letter presents a modification to the established Fraunhofer line discrimination (FLD) method for improving the accuracy of the solar-induced chlorophyll fluorescence (ChF) retrieval over terrestrial vegetation. The FLD method relies on the decoupling of reflected and ChF emitted radiation by the evaluation of measurements inside and outside the absorption bands. The improved FLD method introduces two correction coefficients that relate the values of the fluorescence and the reflectance inside and outside the absorption band. The new method uses the full spectral information around the absorption band to derive these coefficients. A sensitivity analysis has been performed to evaluate the impact of the correction coefficients on the accuracy of the ChF estimation. The new formulation has been tested for the$\hbox{O}_{2}$A-band on synthetic data obtaining lower errors in comparison to the standard FLD and has been successfully applied to real measurements at canopy level.

This letter presents a semisupervised method based on kernel machines and graph theory for remote sensing image classification. The support vector machine (SVM) is regularized with the unnormalized graph Laplacian, thus leading to the Laplacian SVM (LapSVM). The method is tested in the challenging problems of urban monitoring and cloud screening, in which an adequate exploitation of the wealth of unlabeled samples is critical. Results obtained using different sensors, and with low number of training samples, demonstrate the potential of the proposed LapSVM for remote sensing image classification.

The multitemporal classification of remote sensing images is a challenging problem, in which the efficient combination of different sources of information (e.g., temporal, contextual, or multisensor) can improve the results. In this paper, we present a general framework based on kernel methods for the integration of heterogeneous sources of information. Using the theoretical principles in this framework, three main contributions are presented. First, a novel family of kernel-based methods for multitemporal classification of remote sensing images is presented. The second contribution is the development of nonlinear kernel classifiers for the well-known difference and ratioing change detection methods by formulating them in an adequate high-dimensional feature space. Finally, the presented methodology allows the integration of contextual information and multisensor images with different levels of nonlinear sophistication. The binary support vector (SV) classifier and the one-class SV domain description classifier are evaluated by using both linear and nonlinear kernel functions. Good performance on synthetic and real multitemporal classification scenarios illustrates the generalization of the framework and the capabilities of the proposed algorithms.

The Fraunhofer Line Discrimination (FLD) principle is established as a good method for remote sensing of solar induced chlorophyll fluorescence. Some improvements to the method are analysed in order to determine and reduce the sources of error in the estimation of the fluorescence emission. A sensitivity analysis has been performed over simulated data generated from real diurnal cycle measurements.

Vegetation monitoring is a key issue in Earth Observation due to its relation with the global CO2cycle. Chlorophyll fluorescence (ChF) emitted by the vegetation is an accurate indicator of the plant status and their photosynthetic activity. This work analyses the diurnal evolution of the ChF emission spectrum and the fluorescence yield in order to determine the best conditions for remote sensing of ChF from a satellite platform. The ChF evolution is studied at leaf level during several diurnal cycles, in simulated conditions, for two species under different stress conditions. The analysis of the signal levels gives an estimation of the values of ChF emission which could be observed from a remote sensing platform, and determines the best overpass time for this observation.

This paper introduces the use of Relevance Vector Machines (RVM) for remote sensing hyperspectral image classification. We also include the Mahalanobis kernel in the formulation of the RVM to take into account the covariance of the features in the classification process. Experimental results in different scenarios confirm the accuracy and robustness of the proposed method, and also the ease of free parameters tuning.

This work evaluates a new semi-supervised classification framework based on kernel methods and graph theory. In particular, the support vector machine (SVM) is further regularized with the un-normalized graph Laplacian, thus leading to the proposed Laplacian SVM. The method is tested in the challenging problem of cloud screening where the objective is to identify clouds in multispectral images acquired by space-borne sensors working in the visible and near-infrared spectral range. Preliminary results obtained using MERIS/ENVISAT data show the potential of the proposed Laplacian SVM in several scenarios.

This paper presents simple but powerful combination methods of dedicated one-class classifiers (OCCs) for efficient remote sensing image classification. The mean and product combination rules are applied to the probabilistic outputs generated by OCCs, and the performance is illustrated in a urban monitoring application in which multi-sensor (optical and SAR) data and multi-source (spectral and contextual) features are available. Two OCCs are used as core parts: the classical mixture of Gaussians (MoG) and the support vector domain description (SVDD) classifier. The obtained results by combining SVDD classifier outputs show a clear improvement in the accuracy, and more robustness to high dimensional samples compared to both MoG and stacked approaches.

This paper presents a methodology for cloud screening of multispectral images acquired with the Medium Resolution Imaging Spectrometer (MERIS) instrument on-board the Environmental Satellite (ENVISAT). The method yields both a discrete cloud mask and a cloud-abundance product from MERIS level-1b data on a per-pixel basis. The cloud-screening method relies on the extraction of meaningful physical features (e.g., brightness and whiteness), which are combined with atmospheric-absorption features at specific MERIS-band locations (oxygen and water-vapor absorptions) to increase the cloud-detection accuracy. All these features are inputs to an unsupervised classification algorithm; the cloud-probability output is then combined with a spectral unmixing procedure to provide a cloud-abundance product instead of binary flags. The method is conceived to be robust and applicable to a broad range of actual situations with high variability of cloud types, presence of ground covers with bright and white spectra, and changing illumination conditions or observation geometry. The presented method has been shown to outperform the MERIS level-2 cloud flag in critical cloud-screening situations, such as over ice/snow covers and around cloud borders. The proposed modular methodology constitutes a general framework that can be applied to multispectral images acquired by spaceborne sensors working in the visible and near-infrared spectral range with proper spectral information to characterize atmospheric-oxygen and water-vapor absorptions.

This letter presents a framework of composite kernel machines for enhanced classification of hyperspectral images. This novel method exploits the properties of Mercer's kernels to construct a family of composite kernels that easily combine spatial and spectral information. This framework of composite kernels demonstrates: 1) enhanced classification accuracy as compared to traditional approaches that take into account the spectral information only: 2) flexibility to balance between the spatial and spectral information in the classifier; and 3) computational efficiency. In addition, the proposed family of kernel classifiers opens a wide field for future developments in which spatial and spectral information can be easily integrated.

We propose the use of support vector machines (SVMs) for automatic hyperspectral data classification and knowledge discovery. In the first stage of the study, we use SVMs for crop classification and analyze their performance in terms of efficiency and robustness, as compared to extensively used neural and fuzzy methods. Efficiency is assessed by evaluating accuracy and statistical differences in several scenes. Robustness is analyzed in terms of: (1) suitability to working conditions when a feature selection stage is not possible and (2) performance when different levels of Gaussian noise are introduced at their inputs. In the second stage of this work, we analyze the distribution of the support vectors (SVs) and perform sensitivity analysis on the best classifier in order to analyze the significance of the input spectral bands. For classification purposes, six hyperspectral images acquired with the 128-band HyMAP spectrometer during the DAISEX-1999 campaign are used. Six crop classes were labeled for each image. A reduced set of labeled samples is used to train the models, and the entire images are used to assess their performance. Several conclusions are drawn: (1) SVMs yield better outcomes than neural networks regarding accuracy, simplicity, and robustness; (2) training neural and neurofuzzy models is unfeasible when working with high-dimensional input spaces and great amounts of training data; (3) SVMs perform similarly for different training subsets with varying input dimension, which indicates that noisy bands are successfully detected; and (4) a valuable ranking of bands through sensitivity analysis is achieved.

In this paper, we propose a procedure to reduce data dimensionality while preserving relevant information for posterior crop cover classification. The huge amount of data involved in hyperspectral image processing is one of the main problems in order to apply pattern recognition techniques. We propose a dimensionality reduction strategy that eliminates redundant information and a subsequent selection of the most discriminative features based on classification and regression trees (CART). CART allow feature selection based on the classification success, it is a non-linear method and specially allows knowledge discovery. The main advantage of our proposal relies on model interpretability, since we can get qualitative information by analyzing the surrogate and main splits of the tree. This method is tested with a crop cover recognition application of six hyperspectral images from the same area acquired with the 128-bands HyMap spectrometer. Even though CART do not provide the best results in classification it is useful for a previous pre-processing step of feature selection. Finally, we analyze the selected bands of the input space in order to gain knowledge on the problem and to give a physical interpretation of results.

In this paper, we propose a procedure to reduce dimensionality of hyperspectral data while preserving relevant information for posterior crop cover classification. One of the main problems with hyperspectral image processing is the huge amount of data involved. In addition, pattern recognition methods are sensitive to problems associated to high dimensionality feature spaces (referred to as Hughes phenomenon of curse of dimensionality). We propose a dimensionality reduction strategy that eliminates redundant information by means of local correlation criterion between contiguous spectral bands; and a subsequent selection of the most discriminative features based on a Sequential Float Feature Selection algorithm. This method is tested with a crop cover recognition application of six hyperspectral images from the same area acquired with the 128-bands HyMap spectrometer during the DAISEX99 campaign. In the experiments, we analyze the dependence on the dimension and employed metrics. The results obtained using the Gaussian Maximum Likelihood improve the classification accuracy and confirm the validity of the proposed approach. Finally, we analyze the selected bands of the input space on order to gain knowledge on the problem and to give a physical interpretation of the results.

A new approach to the classification of hyperspectral images is proposed. The main problem with supervised methods is that the learning process heavily depends on the quality of the training data set. In remote sensing, the training set is useful only for simultaneous images or for images with the same classes taken under the same conditions; and, even worse, the training set is frequently not available. On the other hand, unsupervised methods are not sensitive to the number of labelled samples since they work on the whole image. Nevertheless, relationship between clusters and classes is not ensured. In this context, we propose a combined strategy of supervised and unsupervised learning methods that avoids these drawbacks and automates the classification process. The method is based on the general formulation of the expectation-maximization (EM) algorithm. This method is applied to crop cover recognition of six hyperspectral images from the same area acquired with the HyMap spectrometer during the DAISEX-99 campaign. For classification purposes, six different classes are considered. Classification accuracy results are compared to common methods: ISODATA, Learning Vector Quantization, Gaussian Maximum Likelihood, Expectation-Maximization, and Neural Networks. The good performance confirms the validity of the proposed approach in terms of accuracy and robustness.

A novel fuzzy-based activation function for artificial neural networks is proposed. This approach provides easy hardware implementation and straightforward interpretability in the basis of IF-THEN rules. Backpropagation learning with the new activation function also has low computational complexity. Several application examples ( XOR gate, chaotic time-series prediction, channel equalization, and independent component analysis) support the potential of the proposed scheme.