Core image processing tasks, such as super-resolution, denoising, deblurring, pansharpening, and atmospheric correction, underpin all optical remote sensing (RS) pipelines. Errors at this stage propagate through downstream applications, distorting land-cover maps, change detection, and climate records. Classical physics-based models capture sensor optics, radiometry, and geometry but struggle with complex noise and scene variability. In contrast, deep learning (DL) methods offer powerful data-driven solutions yet often act as closed boxes, ignoring physical constraints and overfitting to spurious patterns. Hybrid DL (HDL) approaches bridge this gap by integrating physical models with neural architectures, combining interpretability and data adaptivity. This article surveys the emerging landscape of HDL methods in RS image processing, outlining their theoretical foundations, motivations, and design philosophies. We categorize fusion strategies, from model-embedded schemes (e.g., plug-and-play (PnP) and unrolling) to model-guided learning (e.g., deep image prior (DIP) and unsupervised frameworks), and discuss how they enhance trust, robustness, and physical consistency in RS image analysis.

We propose a regularisation strategy of classical machine learning algorithms rooted in causality that ensures robustness against distribution shifts. Building upon the anchor regression framework, we demonstrate how incorporating a straightforward regularisation term into the loss function of classical multivariate analysis algorithms, such as (orthonormalized) partial least squares, reduced-rank regression, and multiple linear regression, enables out-of-distribution generalisation. Our framework allows users to efficiently verify the compatibility of a loss function with the regularisation strategy. Estimators for selected algorithms are provided, showcasing consistency and efficacy in synthetic and real-world climate science problems. The empirical validation highlights the versatility of anchor regularisation, emphasizing its compatibility with multivariate analysis approaches and its role in enhancing replicability while guarding against distribution shifts. The extended anchor framework advances causal inference methodologies, addressing the need for reliable out-of-distribution generalisation.

Flag manifolds encode nested sequences of subspaces and serve as powerful structures for various computer vision and machine learning applications. Despite their utility in tasks such as dimensionality reduction, motion averaging, and subspace clustering, current applications are often restricted to extracting flags using common matrix decomposition methods like the singular value decomposition. Here, we address the need for a general algorithm to factorize and work with hierarchical datasets. In particular, we propose a novel, flag-based method that decomposes arbitrary hierarchical real-valued data into a hierarchy-preserving flag representation in Stiefel coordinates. Our work harnesses the potential of flag manifolds in applications including denoising, clustering, and few-shot learning.

We propose a novel approach for learning causal response representations. Our method aims to extract directions in which a multidimensional outcome is most directly caused by a treatment variable. By bridging conditional independence testing with causal representation learning, we formulate an optimisation problem that maximises the evidence against conditional independence between the treatment and outcome, given a conditioning set. This formulation employs flexible regression models tailored to specific applications, creating a versatile framework. The problem is addressed through a generalised eigenvalue decomposition. We show that, under mild assumptions, the distribution of the largest eigenvalue can be bounded by a known $F$-distribution, enabling testable conditional independence. We also provide theoretical guarantees for the optimality of the learned representation in terms of signal-to-noise ratio and Fisher information maximisation. Finally, we demonstrate the empirical effectiveness of our approach in simulation and real-world experiments. Our results underscore the utility of this framework in uncovering direct causal effects within complex, multivariate settings.

Information theory is an outstanding framework for measuring uncertainty, dependence, and relevance in data and systems. It has several desirable properties for real-world applications: naturally deals with multivariate data, can handle heterogeneous data, and the measures can be interpreted. However, it has not been adopted by a wider audience because obtaining information from multidimensional data is a challenging problem due to the curse of dimensionality. We propose an indirect way of estimating information based on a multivariate iterative Gaussianization transform. The proposed method has a multivariate-to-univariate property: it reduces the challenging estimation of multivariate measures to a composition of marginal operations applied in each iteration of the Gaussianization. Therefore, the convergence of the resulting estimates depends on the convergence of well-understood univariate entropy estimates, and the global error linearly depends on the number of times the marginal estimator is invoked. We introduce Gaussianization-based estimates for Total Correlation, Entropy, Mutual Information, and Kullback-Leibler Divergence. Results on artificial data show that our approach is superior to previous estimators, particularly in high-dimensional scenarios. We also illustrate the method's performance in different fields to obtain interesting insights. We make the tools and datasets publicly available to provide a test bed for analyzing future methodologies.

Masked image modeling (MIM), a common selfsupervised learning (SSL) technique, has been extensively studied for remote sensing image processing. Nevertheless, its effectiveness for hyperspectral imagery (HSI) remains underexplored due to the distinct data structures and high dimensionality. This paper aims to provide a detailed understanding of MIM from different perspectives of representation learning and statistical analysis for HSI classification tasks. Our study reveals that the MIM paradigm injects inductive bias in the attention mechanism of the transformer model, which is advantageous for capturing the local discrepancies between the spectra. We also show that MIM can increase the diversity of the attention heads in every layer, which is beneficial for the model in extracting more discriminative features from different spectral bands. The similarity of representations from various layers further proves this. Furthermore, our investigation highlights how MIM introduces a dynamic perspective to spectral representations, enabling the model to learn more robust and discriminative features. The final numerical experiments indicate that a moderate mask ratio can enhance the performance of downstream tasks. This suggests that designing a more targeted masking strategy might be necessary to achieve higher and more stable gains in downstream task performance. Without bells and whistles, the vanilla MIM improves the overall classification accuracy by an average of 2.69% over its SL counterpart. We hope that our findings can advance the understanding of MIM in HSI and inspire the design of a more stable SSL paradigm for HSI processing.

Satellite remote sensing is the primary source of global aerosol observations, providing essential data for understanding aerosol-climate interactions and constraining global climate models. To solve the inverse problem at the heart of the retrieval process, traditional algorithms must make simplifications and often cannot quantify uncertainty. In this study, we explore the use of invertible neural networks (INNs) for retrieving aerosol optical depth (AOD) from spectral top-of-atmosphere (TOA) reflectance. INNs can handle the inherent uncertainty of underdetermined inverse problems. They model the forward and inverse processes simultaneously while learning additional random latent variables used to recover full nonparametric posterior distributions for the inverse predictions. We develop location-specific INNs for MODIS sensor data, training on synthetic datasets generated by combining atmospheric reflectance from MODIS dark target (DT) lookup tables (LUTs) and surface reflectance from a MODIS bidirectional reflectance product. The INNs successfully emulate the forward problem and achieve accurate AOD inversion results on synthetic test sets (RMSE$\approx ~0.05$). The posterior distributions obtained are reliable (mean absolute calibration error (MACE)$\approx ~2.5$%), efficiently providing informative predictive uncertainty estimates. In addition, the INNs’ invertible architecture is found to promote physically consistent predictions and uncertainties. To further validate them in a real-world context, the INNs are applied to MODIS L1B reflectance observations to produce full-resolution AOD estimates with pixel-level uncertainties. The retrievals are compared to collocated ground measurements from the Aeronet network. The INNs obtain good accuracy in all tested locations in line with the operational DT AOD product (RMSE$\approx ~0.1$, 74% within DT expected error (EE) bounds). The INNs are also able to retrieve AOD over bright surfaces where DT cannot be applied. Despite uncovered limitations out-of-distribution, the INNs show consistent skill in target domains across diverse land surfaces. The INNs’ unique modeling and uncertainty quantification features have the potential to enhance aerosol and climate studies in various real-world contexts.

Principal component analysis (PCA), along with its ex-tensions to manifolds and outlier contaminated data, have been indispensable in computer vision and machine learning. In this work, we present a unifying formalism for PCA and its variants, and introduce a framework based on the flags of linear subspaces, i.e. a hierarchy of nested linear subspaces of increasing dimension, which not only allows for a common implementation but also yields novel variants, not explored previously. We begin by generalizing traditional PCA methods that either maximize variance or minimize reconstruction error. We expand these interpre-tations to develop a wide array of new dimensionality re-duction algorithms by accounting for outliers and the data manifold. To devise a common computational approach, we recast robust and dual forms of peA as optimization problems on flag manifolds. We then integrate tangent space approximations of principal geodesic analysis (tangent-PCA) into this flag-based framework, creating novel robust and dual geodesic PCA variations. The remarkable flexibility offered by the ‘flagification’ introduced here enables even more algorithmic variants identified by specific flag types. Last but not least, we propose an effective convergent solver for these flag-formulations employing the Stiefel manifold. Our empirical results on both real-world and synthetic sce-narios, demonstrate the superiority of our novel algorithms, especially in terms of robustness to outliers on manifolds.

Remote sensing data management and its use for classification and inference purposes is at the forefront of research tasks nowadays. There are, however, some inherent drawbacks and difficulties when dealing with, and understanding how satellite information is provided (particularly when referring to multiband/multispectral satellite platforms) and how different and disparate datasets related to soil content can be used and merged with this imagery.We present WALGREEN. The aim of this tool is to provide a secure environment to handle the whole process to use polygons or, geographical coordinates in tiff/geotiff images, have real-time access to images, save and get soil organic carbon real measurements, and generate datasets for machine learning training and inferential methods. We also aim to providing a framework to preprocess soil organic carbon information from different but accepted sources, like the Land Use/Cover Area frame statistical Survey database, so that even without real measurements, researchers may be able to start training different machine learning methodologies.

In the face of climate change-induced droughts, vulnerable regions encounter severe threats to food security, demanding urgent humanitarian assistance. This paper introduces a causal inference framework for the Horn of Africa, aiming to assess the impact of cash-based interventions on food crises. Our contributions include identifying causal relationships within the malnutrition system, harmonizing a comprehensive database including socio-economic, weather and remote sensing data, and estimating the causal effect of cash-based interventions on malnutrition. On a country level, our results revealed no significant effects, likely due to limited sample size, suboptimal data quality, and an imperfect causal graph resulting from our limited understanding of multidisciplinary systems like malnutrition. Instead, on a district level, results revealed significant effects, further implying the context-specific nature of the system. This underscores the need to enhance data collection and refine causal models with domain experts for more effective future interventions and policies, improving transparency and accountability in humanitarian aid.

Displacement of human populations due to more extreme weather hazards is a global phenomenon that leads to significant human and economic losses. Mobility related to slow-onset events such as droughts is particularly challenging to model because the start and duration of droughts are uncertain, and their effects are often intertwined with other contextual factors, such as conflicts, political stability, and undependable market prices, that are difficult to measure. Moreover, the collection of in situ socioeconomic data poses a significant challenge, where the use of alternative data sources to warn and plan for impending waves of displacement effectively could help. This study investigates the use of social media as an additional input feature to enhance drought-induced displacement predictions. A methodology for identifying human displacement based on tweet activity is proposed. Results from displacement models based on interpretable machine learning, socioeconomic data, and weather indicators consistently show the benefit of including Twitter data when applied at the district level in Somalia. The proposed approach could enhance drought-induced displacement predictions, thereby helping anticipatory action and the planning of humanitarian interventions.

This paper provides an overview of the ACM SIGKDD 2024 AI4Science4AI special day. It includes information about the organizers, invited speakers, keynote speakers, the event agenda, and insights from related workshops. The AI4Science4AI special day aims to bring together experts in artificial intelligence (AI) and science to discuss the latest developments, challenges, and future directions.

Droughts pose significant challenges for accurate monitoring due to their complex spatiotemporal characteristics. Data-driven machine learning (ML) models have shown promise in detecting extreme events when enough well-annotated data is available. However, droughts do not have a unique and precise definition, which leads to noise in human-annotated events and presents an imperfect learning scenario for deep learning models. This article introduces a 3-D convolutional neural network (CNN) designed to address the complex task of drought detection, considering spatiotemporal dependencies and learning with noisy and inaccurate labels. Motivated by the shortcomings of traditional drought indices, we leverage supervised learning with labeled events from multiple sources, capturing the shared conceptual space among diverse definitions of drought. In addition, we employ several strategies to mitigate the negative effect of noisy labels (NLs) during training, including a novel label correction (LC) method that relies on model outputs, enhancing the robustness and performance of the detection model. Our model significantly outperforms state-of-the-art drought indices when detecting events in Europe between 2003 and 2015, achieving an AUROC of 72.28%, an AUPRC of 7.67%, and an ECE of 16.20%. When applying the proposed LC method, these performances improve by +5%, +15%, and +59%, respectively. Both the proposed model and the robust learning methodology aim to advance drought detection by providing a comprehensive solution to label noise and conceptual variability.

Statistical regression methods are widely used in remote sensing applications but tend to lack physical interpretability. In this paper, we introduce a methodological framework to improve model emulation and its understanding with machine learning feature selection. Our wrapper-forward feature selection method seamlessly integrates physics knowledge into model emulation, improving the trade-off between accuracy and interpretability. We illustrate our methodology by applying it to atmospheric radiative transfer models in the context of global sensitivity analysis (GSA) and emulation. Our approach consistently aligns with variance-based GSA, pinpointing the critical features of aerosol properties, solar zenith angle, and water vapor. While our physically-based emulators yield only a modest accuracy improvement of 0.2% over conventional Gaussian Processes emulators, its introduction signifies a step forward to physics-aware machine learning-based emulation. The emulator performance remains steadfast, unaffected by substantial changes, further underscoring the reliability of our approach.

In this study, we examined the effectiveness of integrating satellite-based crop biophysical parameters, meteorological conditions, and soil properties for the end and mid-season cotton yield prediction in the continental United States (CONUS) region. We employed six machine learning algorithms: decision tree (DT), random forest (RF), adaptive boosting (Ad-aBoost), gradient boosting (GB), light gradient boosting machine (LightGBM), and extreme gradient boosting machine (XGBoost). By employing this rigorous approach to hyperparameter tuning based on Bayesian optimization, the XGBoost method was found as the best method for both mid-season and end-season cotton yield prediction. Furthermore, we investigated the global importance of temporal and static features using the Shapley Additive Global importancE (SAGE) method to understand the driving factors of cotton yield prediction. As a result of global feature importance analysis, precipitation (P), enhanced vegetation index (EVI), and leaf area index (LAI) were found as the most important temporal features, while silt and pH were found as the most important soil properties.

We introduce Mesogeos, a large-scale multi-purpose dataset for wildfire modeling in the Mediterranean. Mesogeos integrates variables representing wildfire drivers (meteorology, vegetation, human activity) and historical records of wildfire ignitions and burned areas for 17 years (2006-2022). It is designed as a cloud-friendly spatio-temporal dataset, namely a datacube, harmonizing all variables in a grid of 1km x 1km x 1-day resolution. The datacube structure offers opportunities to assess machine learning (ML) usage in various wildfire modeling tasks. We extract two ML-ready datasets that establish distinct tracks to demonstrate this potential: (1) short-term wildfire danger forecasting and (2) final burned area estimation given the point of ignition. We define appropriate metrics and baselines to evaluate the performance of models in each track. By publishing the datacube, along with the code to create the ML datasets and models, we encourage the community to foster the implementation of additional tracks for mitigating the increasing threat of wildfires in the Mediterranean.

Despite their success deep neural networks still lack interpretability and are regarded as black boxes. This hampers a wider adoption in applications with societal, environmental or economical implications, and motivated a variety of techniques for explaining their outputs. Such explanations are however typically produced after model training so there is no guarantee that models learn faithful attributions, a goal they were not trained for. We evaluate the impact of different penalty terms in the loss function that promote explainable feature attributions, and that can be learned during training in an unsupervised way. We show that explainability-constrained models produce better saliency maps based on multiple metrics and tests. Regularizers imposing locality, fidelity and symmetry properties lead to the best performances in terms of MoRF and ROAR scores.

Cotton is under the threat of climate and ecosystem change, and has an essential role in the global textile industry. This makes its yield prediction essential for both economics and sustainability. The potential cotton yield can be predicted by integrating climatic factors, soil parameters, and biophysical parameters observed by high temporal & spatial resolution remote sensing satellites. This study used a multisource dataset to create an explainable and accurate predictive model for cotton yield prediction over the continental US (CONUS). A recently proposed glass-box method called Explainable Boosting Machine (EBM), which provides transparency, reliability, and ease of interpretation, was implemented. Accuracy performance was compared with common machine learning (ML) methods for predicting cotton yields. The EBM showed higher accuracy against other glass-box methods and competitive results with black-box models. With the help of the EBM, the importance of individual features and their pairwise interactions was revealed without applying any post-hoc methods. The study findings showed that the precipitation (P), enhanced vegetation index (EVI), and leaf area index (LAI) are the three most important dynamic features. The dynamic features are the driver of the created model with 78% of the overall feature importance, followed by pairwise interactions of the features with 16% contribution. Lastly, static features contribute 6% to the overall feature importance. The study highlights the importance of using multi-source data and interactions of the input features and providing an interpretable model to understand the inner dynamics of cotton yield predictions.

Food security is at stake, with climate change heavily impacting agriculture and food production. In the present context of extreme events and changing conditions, developing advanced crop yield models can learn from all available information, and providing interpretable predictions for decision-making is of paramount relevance. This work explores the potential and limitations of developing interpretable crop yield models using long short-term memory (LSTM) neural networks, which typically excel at extracting information from time series. LSTMs were designed and trained with multisource satellite and meteorological time series over Continental US (CONUS) and corn, soybean, and wheat yield data from the US Department of Agriculture. Two recent attribution methods are used to interpret and extract knowledge from the developed models: integrated gradients (IG), based on back-propagation, and Shapley (SHAP) values, based on perturbations. Our results show that: 1) LSTM models achieved high accuracy ($\text {R}^{2}>0.56$); 2) multisource combinations outperformed single-variable models in all crop models; 3) both attribution methods were equivalent in detecting essential drivers and their contribution; 4) satellite estimates of enhanced vegetation index (EVI) and vegetation optical depth (VOD) together with meteorological estimates of maximum temperature (TMX) were the most relevant input features for crop yield estimations; and finally and 5) we discovered critical periods of the crop growth cycle for the corn, soybean, and wheat models. The suggested strategy constitutes an important step toward modeling and understanding crop production systems and advancing in automatic data-driven and accountable field management.

Earth observation from satellites offers the possibility to monitor our planet with unprecedented accuracy. Radiative transfer models (RTMs) encode the energy transfer through the atmosphere, and are used to model and understand the Earth system, as well as to estimate the parameters that describe the status of the Earth from satellite observations by inverse modeling. However, performing inference over such simulators is a challenging problem. RTMs are nonlinear, non-differentiable and computationally costly codes, which adds a high level of difficulty in inference. In this paper, we introduce two computational techniques to infer not only point estimates of biophysical parameters but also their joint distribution. One of them is based on a variational autoencoder approach and the second one is based on a Monte Carlo Expectation Maximization (MCEM) scheme. We compare and discuss benefits and drawbacks of each approach. We also provide numerical comparisons in synthetic simulations and the real PROSAIL model, a popular RTM that combines land vegetation leaf and canopy modeling. We analyze the performance of the two approaches for modeling and inferring the distribution of three key biophysical parameters for quantifying the terrestrial biosphere.

Clouds play a key role in regulating climate change but are difficult to simulate within Earth system models (ESMs). Improving the representation of clouds is one of the key tasks toward more robust climate change projections. This study introduces a new machine-learning-based framework relying on satellite observations to improve understanding of the representation of clouds and their relevant processes in climate models. The proposed method is capable of assigning distributions of established cloud types to coarse data. It facilitates a more objective evaluation of clouds in ESMs and improves the consistency of cloud process analysis. The method is built on satellite data from the Moderate Resolution Imaging Spectroradiometer (MODIS) instrument labeled by deep neural networks with cloud types defined by the World Meteorological Organization (WMO), using cloud-type labels from CloudSat as ground truth. The method is applicable to datasets with information about physical cloud variables comparable to MODIS satellite data and at sufficiently high temporal resolution. We apply the method to alternative satellite data from the Cloud_cci project (ESA Climate Change Initiative), coarse-grained to typical resolutions of climate models. The resulting cloud-type distributions are physically consistent and the horizontal resolutions typical of ESMs are sufficient to apply our method. We recommend outputting crucial variables required by our method for future ESM data evaluation. This will enable the use of labeled satellite data for a more systematic evaluation of clouds in climate models.

Atmospheric radiative transfer models (RTMs) are widely used in satellite data processing to correct for the scattering and absorption effects caused by aerosols and gas molecules in the Earth’s atmosphere. As the complexity of RTMs grows and the requirements for future Earth Observation missions become more demanding, the conventional lookup-table (LUT) interpolation approach faces important challenges. Emulators have been suggested as an alternative to LUT interpolation, but they are still too slow for operational satellite data processing. Our research introduces a solution that harnesses the power of multifidelity methods to improve the accuracy and runtime of Gaussian process (GP) emulators. We investigate the impact of the number of fidelity layers, dimensionality reduction, and training dataset size on the performance of multifidelity GP emulators. We find that an optimal multifidelity emulator can achieve relative errors in surface reflectance below 0.5% and performs atmospheric correction of hyperspectral PRISMA satellite data (one million pixels) in a few minutes. Additionally, we provide a suite of functions and tools for automating the creation and generation of atmospheric RTM emulators.

Feature selection is essential in various fields of science and engineering, from remote sensing to computer vision. Reducing data dimensionality by removing redundant features and selecting the most informative ones improves machine learning algorithms’ performance, especially in supervised classification tasks, while lowering storage needs. Graph-embedding techniques have recently been found efficient for feature selection since they preserve the geometric structure of the original feature space while embedding data into a low-dimensional subspace. However, the main drawback is the high computational cost of solving an eigenvalue decomposition problem, especially for large-scale problems. This paper addresses this issue by combining the graph embedding framework and representation theory for a novel feature selection method. Inspired by the high dimensional model representation, the feature transformation is assumed to be a linear combination of a set of univariate orthogonal functions carried out in the graph embedding framework. As a result, an explicit embedding function is created, which can be utilised to embed out-of-samples into low-dimensional space and provide a feature relevance score. The significant contribution of the proposed method is to divide ann-dimensional generalised eigenvalue problem intonsmall-sized eigenvalue problems. With this property, the computational complexity of the graph embedding is significantly reduced, resulting in a scalable feature selection method, which could be easily parallelized too. The performance of the proposed method is compared favourably to its counterparts in high-dimensional hyperspectral image processing in terms of classification accuracy, feature stability, and computational time.

This work introduces a novel method that makes use of machine learning (ML) techniques to classify hyper- and multi spectral observations into optical water types (OWTs). Classification was done using$k$-means clustering, which was followed by a feature relevance step based on the sensitivity analysis (SA) of the predictive mean and variance function of a Gaussian process (GP) regression model. The method was used both in training and predictive mode. The latter allows applying the approach for new unlabeled observations, so that the OWTs and the associated relevant features can automatically be assessed. The methods were studied on hyperspectral synthesized and in situ Arctic data, and were further evaluated on a test image acquired over Arctic seas. Good empirical results encourage wide adoption of the methodology to be applied in operational processing and assessment of water types.

Retrieval of physical parameters is of paramount relevance for Earth monitoring. Statistical (machine) learning approaches have been successfully introduced in the community, because they can learn nonlinear functional relations from observational data with no strong a priori assumptions and parametric forms. However, these methods still have two relevant problems: they only consider only one nonlinear feature map of the data, which can be limiting in complex problems where inputs (e.g., radiances) and outputs (e.g., state vectors) have strong nonlinear relations, and in most of the cases, models do not incorporate the structure of the output (dependent) variables. This article proposes a kernel method that solves the two aforementioned problems for physical parameter retrieval: first, it performs multioutput regression with the desired number of connected mappings, and second, it incorporates the output variables structure via a dedicated kernel. The proposed method has a closed-form solution, and thus, neither kernel dimensionality reduction nor preimaging is necessary unlike in previous structured kernel methods. Through the definition of appropriate kernel feature mappings, we also derive a pragmatic deep structured kernel ridge regression (KRR). The method is characterized statistically using a Gaussian process (GP) treatment and providing guarantees based on the concepts of leverage scores (LSs) and effective dimension: both explain that including an output structure acts as a powerful regularizer. We illustrate the method’s performance in toy examples and remote sensing parameter estimation problems involving vegetation parameters [chlorophyll, leaf area index (LAI), and fractional vegetation cover (FVC)] from compact high-resolution imaging spectrometer (CHRIS) images and the atmospheric temperature, moisture, and ozone profiles from infrared atmospheric sounding interferometer (IASI) data.

Anomaly detection (AD) is a field of intense research in remote sensing (RS) image processing. Identifying low probability events in RS images is a challenging problem given the high dimensionality of the data, especially when no (or little) information about the anomaly is availablea priori. While a plenty of methods are available, the vast majority of them do not scale well to large datasets and require the choice of some (very often critical) hyperparameters. Therefore, unsupervised and computationally efficient detection methods become strictly necessary, especially now with the data deluge problem. In this article, we propose an unsupervised method for detecting anomalies and changes in RS images by means of a multivariate Gaussianization methodology that allows to estimate multivariate densities accurately, a long-standing problem in statistics, and machine learning. The methodology transforms arbitrarily complex multivariate data into a multivariate Gaussian distribution. Since the transformation is differentiable, by applying the change of variables formula, one can estimate the probability at any point of the original domain. The assumption is straightforward: pixels with low estimated probability are considered anomalies. Our method can describe any multivariate distribution, makes an efficient use of memory and computational resources, and is parameter-free. We show the efficiency of the method in experiments involving both AD and change detection (CD) in different RS image sets. For AD, we propose two approaches. The first is using directly the Gaussianization transform and the second is using a hybrid model that combines Gaussianization and the Reed–Xiaoli (RX) method typically used in AD. For CD, we take advantage of the Gaussianization transform and attribute the change to pixels with low probability compared to the first image, instead of those with high difference value typically employed in RS. Results show that our approach outperforms other linear and nonlinear methods in terms of detection power in both anomaly and CD scenarios, showing robustness and scalability to dimensionality and sample sizes.

Satellite-derived soil moisture (SM) products have become an important information source for the study of land surface processes in hydrology and land monitoring. Characterizing and estimating soil memory and persistence from satellite observations is of paramount relevance, and has deep implications in ecology, water management, and climate modeling. In this work, we address the problem of SM persistence estimation from microwave sensors using several autocorrelation metrics that, unlike traditional approaches, build on accurate estimates of the autocorrelation function from nonuniformly sampled time series. We show how the choice of the autocorrelation estimator can have a dramatic impact on the SM persistence metrics derived thereof, particularly given the nonuniform nature of satellite observations, yet this fact has been overlooked to a large extent in literature. We give empirical evidence of performance using ground-based SM measurements, as well as L-band Soil Moisture and Ocean Salinity (SMOS) and C-band [Advanced Microwave Scanning Radiometer-2 (AMSR2), Advanced Scatterometer (ASCAT)] remotely sensed SM data. Experiments along transects allow us to scrutinize the inter-method consistency and the spatial–temporal characteristics of autocorrelation estimators. This motivates the introduction of novel measures of spatial–temporal autocorrelation and allows us to retrieve improved persistence estimates. Results over the Iberian Peninsula indicate the SM persistence patterns captured by L- and C-band microwave sensors over semiarid regions exhibit spatially concurrent patterns of persistence and support their combination in long-term data records. We conclude that accounting for the nonuniform nature of the satellite time series using robust autocorrelation estimations allows providing improved measures and spatial descriptions of SM persistence.

Atmospheric radiative transfer models (RTMs) simulate the light propagation in the Earth's atmosphere. With the evolution of RTMs, their increase in complexity makes them impractical in routine processing such as atmospheric correction. To overcome their computational burden, standard practice is to interpolate a multidimensional lookup table (LUT) of prestored simulations. However, accurate interpolation relies on large LUTs, which still implies large computation times for their generation and interpolation. In recent years, emulation has been proposed as an alternative to LUT interpolation. Emulation approximates the RTM outputs by a statistical regression model trained with a low number of RTM runs. However, a concern is whether the emulator reaches sufficient accuracy for atmospheric correction. Therefore, we have performed a systematic assessment of key aspects that impact the precision of emulating MODTRAN: 1) regression algorithm; 2) training database size; 3) dimensionality reduction (DR) method and a number of components; and 4) spectral resolution. The Gaussian processes regression (GPR) was found the most accurate emulator. The principal component analysis remains a robust DR method and nearly 20 components reach sufficient precision. Based on a database of 1000 samples covering a broad range of atmospheric conditions, GPR emulators can reconstruct the simulated spectral data with relative errors below 1% for the 95th percentile. These emulators reduce the processing time from days to minutes, preserving sufficient accuracy for atmospheric correction and providing model uncertainties and derivatives. We provide a set of guidelines and tools to design and generate accurate emulators for satellite data processing applications.

The modeling of Earth observation (EO) data is a challenging problem, typically approached by either purely mechanistic or purely data-driven methods. Mechanistic models encode the domain knowledge and physical rules governing the system. Such models, however, need the correct specification of all interactions between variables in the problem and the appropriate parameterization is a challenge in itself. On the other hand, machine learning approaches are flexible data-driven tools, able to approximate arbitrarily complex functions, but lack interpretability and struggle when data are scarce or in extrapolation regimes. In this article, we argue thathybrid learning schemesthat combine both approaches can address all these issues efficiently. We introduceGaussian process (GP) convolution modelsfor hybrid modeling in EO problems. We specifically propose the use of a class of GP convolution models calledlatent force models(LFMs) for EO time series modeling, analysis, and understanding. LFMs are hybrid models that incorporate physical knowledge encoded in differential equations into a multioutput GP model. LFMs can transfer information across time series, cope with missing observations, infer explicit latent functions forcing the system, and learn parameterizations which are very helpful for system analysis and interpretability. We illustrate the performance in two case studies. First, we consider time series of soil moisture (SM) from active Advanced Scatterometer (ASCAT) and passive [SM and ocean salinity (SMOS), advanced microwave scanning radiometer-2 (AMSR2)] microwave satellites. We show how assuming a first-order differential equation as governing equation, the model automatically estimates the e-folding time or decay rate related to SM persistence and discovers latent forces related to precipitation. In the second case study, we show how the model can fill in gaps of leaf area index (LAI) and Fraction of Absorbed Photosynthetically Active Radiation (fAPAR) from moderate resolution imaging spectroradiometer (MODIS) optical time series by exploiting their relations across different spatial and temporal domains. The proposed hybrid methodology reconciles the two main approaches in remote-sensing parameter estimation by blending statistical learning and mechanistic modeling.

Learning the manifold structure of remote sensing images is of paramount relevance for modeling and understanding processes, as well as encapsulating the high dimensionality in a reduced set of informative features for subsequent classification, regression, or unmixing. Manifold learning methods have shown excellent performance when dealing with hyperspectral image (HSI) analysis, but, unless specifically designed, they cannot provide an explicit embedding map readily applicable to out-of-sample (OOS) data. A common assumption to deal with the problem is that the transformation between the high-dimensional input space and the latent space (typically low) is linear. This is a particularly strong assumption, especially when dealing with HSIs due to the well-known nonlinear nature of the data. To address this problem, a manifold learning method based on high-dimensional model representation (HDMR) is proposed, which enables a nonlinear embedding function to project OOS samples into the latent space. The proposed method is compared to manifold learning methods along with their linear counterparts and achieves promising performance in terms of classification accuracy for a representative set of HSIs.

Machine learning models alone are excellent approximators, but very often do not respect the most elementary laws of physics, like mass or energy conservation, so consistency and confidence are compromised. In this paper we describe the main challenges ahead in the field, and introduce several ways to live in the Physics and machine learning interplay: encoding differential equations from data, constraining data-driven models with physics-priors and dependence constraints, improving parameterizations, emulating physical models, and blending data-driven and process-based models. This is a collective long-term AI agenda towards developing and applying algorithms capable of discovering knowledge in the Earth system.

The present study focuses on retrieving soil and canopy temperatures, which are key parameters to estimate soil moisture and vegetation optical depth from multi-frequency microwaves information. Several retrieval algorithms assume that canopy and vegetation temperatures are similar in thermal equilibrium conditions, while others separate their contributions, as SMOS-IC, one of the consolidated retrieval algorithms for the Soil Moisture and Ocean Salinity (SMOS) satellite mission. Soil and canopy temperatures in SMOS-IC are modelled from the ECMWF (European Centre for Medium-Range Weather Forecasts) centre. Both SMOS and the Soil Moisture Active Passive (SMAP) missions are currently the only passive L-band (1.4 GHz) missions in operation, but their lifetime is limited. In this context, the upcoming Copernicus Imaging Microwave Radiometer (CIMR) mission will provide continuity on L-band measurements with complementary information in a range of microwave frequencies, from 1.4 to 36.5 GHz. This study uses in situ soil moisture information from the International Soil Moisture Network (ISMN) as input in the SMOS-IC algorithm to retrieve vegetation optical depth (VOD) and soil/canopy effective temperature (TGC). The retrieved effective temperature is then compared with modelled temperatures from ECMWF and with data from the Advanced Microwave Scanning Radiometer 2 (AMSR2), which acquires the higher frequency bands (C, X, Ka, and Ku) present in the future CIMR mission. Results confirm the potential of all high-frequency bands to estimate TGC, with C and X-bands being the most correlated. This study is a first approach to evaluate how microwave multi-frequency information can help modelling soil and canopy temperatures in the SMOS-IC retrieval algorithm, from which the upcoming CIMR mission may benefit.

Image classification has become one of the most common applications in remote sensing yielding to the creation of a variety of operational thematic maps at multiple spatio-temporal scales. The information contained in these maps summarizes key characteristics related with the physical environment and provides fundamental information of the Earth for vegetation monitoring or land use status over time. However, high spatial resolution land cover maps are usually only produced for specific small regions or in an image tile. We present a general methodology to obtain a high spatial resolution land cover maps using Landsat spectral information, the powerful Google Earth Engine platform, and operational coarse classification schemes such as the MODIS (MOD12) land cover. After the experimental analysis for different regions, we conclude that the method allows to successfully learn the MODIS Plant Functional Type classification scheme at 500 m pixel resolution which greatly improves the level of spatial detail when the machine learning model is applied to Landsat pixel resolution (30 m) reflectance data.

Agriculture monitoring, and in particular food security, requires near real-time information on crop growing conditions for early detection of possible production deficits. In this work, we propose the use of Gaussian processes (GPs). together with in-situ, EO and ERA-Interim climate reanalysis data for crop yield forecasting. Country-level agricultural survey data from FAOSTAT are used for quantitative assessment. The study is conducted in the framework of the ASAP (Anomaly hot Spots of Agricultural Production) early warning decision support system of the European Commission, which aims at providing timely information about possible crop production anomalies worldwide. After grouping countries with similar growing season periods, we We show that GP models allow predicting the yield of the different planted crops within such groups with coefficient of determination R2 ranging from 0.5 to 0.95. For each country and crop, a better fit that the mean of the data is obtained in all cases, and the errors obtained are comparable to the ones obtained for the group. The proposed modelling framework can be potentially adopted in ASAP operations to forecast national level crop yield and production.

This work introduces the use of Gaussian processes (GPs) for the estimation and understanding of crop development and yield using multisensor satellite observations and meteorological data. The proposed methodology combines synergistic information on canopy greenness, biomass, soil, and plant water content from optical and microwave sensors with the atmospheric variables typically measured at meteorological stations. A composite covariance is used in the GP model to account for varying scales, nonstationary, and nonlinear processes. The GP model reports noticeable gains in terms of accuracy with respect to other machine learning approaches for the estimation of corn, wheat, and soybean yields consistently for four years of data across continental U.S. (CONUS). Sparse GPs allow obtaining fast and compact solutions up to a limit, where heavy sparsity compromises the credibility of confidence intervals. We further study the GP interpretability by sensitivity analysis, which reveals that remote sensing parameters accounting for soil moisture and greenness mainly drive the model predictions. GPs finally allow us to identify climate extremes and anomalies impacting crop productivity and their associated drivers.

Current anomaly detection (AD) algorithms are typically challenged by either accuracy or efficiency. More accurate nonlinear detectors are typically slow and not scalable. In this letter, we propose two families of techniques to improve the efficiency of the standard kernel Reed-Xiaoli (KRX) method for AD by approximating the kernel function with either the data-independent random Fourier features or the data-dependent basis with the Nyström approach. We compare all methods for both real multi- and hyperspectral images. We show that the proposed efficient methods have a lower computational cost, and they perform similar to (or outperform) the standard KRX algorithm thanks to their implicit regularization effect. Last but not least, the Nyström approach has an improved power of detection.

Monte Carlo (MC) algorithms are widely used for Bayesian inference in statistics, signal processing, and machine learning. In this work, we introduce an Markov Chain Monte Carlo (MCMC) technique driven by a particle filter. The resulting scheme is a generalization of the so-called Particle Metropolis-Hastings (PMH) method, where a suitable Markov chain of sets of weighted samples is generated. We also introduce a marginal version for the goal of jointly inferring dynamic and static variables. The proposed algorithms outperform the corresponding standard PMH schemes, as shown by numerical experiments.

Many fields of science and engineering require the use of complex and computationally expensive models to understand the involved processes in the system of interest. Nevertheless, due to the high cost involved, the required study becomes a cumbersome process. This paper introduces an interpolation procedure which belongs to the family of active learning algorithms, in order to construct cheap surrogate models of such costly complex systems. The proposed technique is sequential and adaptive, and is based on the optimization of a suitable acquisition function. We illustrate its efficiency in a toy example and for the construction of an emulator of an atmosphere modeling system.

Modeling and understanding the Earth system is a constant and challenging scientific endeavour. When a clear mechanistic model is unavailable, complex or uncertain, learning from data can be an alternative. While machine learning has provided excellent methods for detection and retrieval, understanding the governing equations of the system from observational data seems an elusive problem. In this paper we introduce sparse regression to uncover a set of governing equations in the form of a system of ordinary differential equations (ODEs). The presented method is used to explicitly describe variable relations by identifying the most expressive and simplest ODEs explaining data to model relevant components of the biosphere.

Machine learning and deep learning in particular have made a huge impact in many fields of science and engineering. In the last decade, advanced deep learning methods have been developed and applied to remote sensing and geoscientific data problems extensively. Applications on classification and parameter retrieval are making a difference: methods are very accurate, can handle large amounts of data, and can deal with spatial and temporal data structures efficiently. Nevertheless, several important challenges need still to be addressed. First, current standard deep architectures cannot deal with long-range dependencies so distant driving processes (in space or time) are not captured, and they cannot cope with non-Euclidean spaces efficiently. Second, as other data-driven techniques, deep learning models do not necessarily respect physical or causal relations. Finally, deep learning models are still obscure and resistant to interpretability. Advances are needed to cope with arbitrary signal structures and data relations, physical plausibility and interpretability. This paper discusses about ways forward to develop new DL methods for the Earth sciences in all three directions.

High spatial resolution vegetation products are fundamental in different fields, such as improving the understanding of crop seasonality at regional scales. Here, two new vegetation products such as the Leaf Area Index (LAI) and the Fraction of Absorbed Photosynthetically Active Radiation (FAPAR) are downscaled at continental scales. A novel HIghly Scalable Temporal Adaptive Reflectance Fusion Model (HIS-TARFM) is used to generate the gap-free time series of Landsat surface reflectance data by fusing MODIS and Landsat reflectance for the contiguous United States. An artificial neural network is trained to capture the relationship between the gap free Landsat surface reflectance and the MODIS LAI/FAPAR products and allows to predict both biophysical variables at 30 meters spatial resolution. The results confirm that both vegetation products largely agree with the test dataset, providing low error and high explained variance.

In this work we propose the use of Long Short-Term Memory (LSTM) Recurrent Neural Networks for multivariate time series of satellite data for crop yield estimation. Recurrent nets allow exploiting the temporal dimension efficiently, but interpretability is hampered by the typically overparameterized models. The focus of the study is to understand LSTM models by looking at the hidden units distribution, the impact of increasing network complexity, and the relative importance of the input covariates. We extracted time series of three variables describing the soil-vegetation status in agroe-cosystems -soil moisture, VOD and EVI- from optical and microwave satellites, as well as available in situ surveys on crops across Continental U.S. to perform the experiments. Firstly, the models were validated in error terms. Secondly, the trained models were visualized and, thirdly, some useful statistics were extracted from the hidden unit activation heatmaps, accounting for redundancy and cluttering of activation responses. Results reveal how networks assign most of the relevance to soil moisture and focus on two phenological stages of crop growth.

Manifold learning methods are very efficient methods for hyperspectral image (HSI) analysis but, unless specifically designed, they cannot provide an explicit embedding map readily applicable to out-of-sample data. A common assumption to deal with the problem is that the transformation between the high input dimensional space and the (typically low) latent space is linear. This is a particularly strong assumption, especially when dealing with hyperspectral images due to the well-known nonlinear nature of the data. To address this problem, a manifold learning method based on High Dimensional Model Representation (HDMR) is proposed, which enables to present a nonlinear embedding function to project out-of-sample samples into the latent space. The proposed method is compared to its linear counterparts and achieves promising performance in terms of classification accuracy of hyperspectral images.

Gaussian processes (GPs) are a class of Kernel methods that have shown to be very useful in geoscience and remote sensing applications for parameter retrieval, model inversion, and emulation. They are widely used because they are simple, flexible, and provide accurate estimates. GPs are based on a Bayesian statistical framework which provides a posterior probability function for each estimation. Therefore, besides the usual prediction (given in this case by the mean function), GPs come equipped with the possibility to obtain a predictive variance (i.e., error bars, confidence intervals) for each prediction. Unfortunately, the GP formulation usually assumes that there is no noise in the inputs, only in the observations. However, this is often not the case in earth observation problems where an accurate assessment of the measuring instrument error is typically available, and where there is huge interest in characterizing the error propagation through the processing pipeline. In this letter, we demonstrate how one can account for input noise estimates using a GP model formulation which propagates the error terms using the derivative of the predictive mean function. We analyze the resulting predictive variance term and show how they more accurately represent the model error in a temperature prediction problem from infrared sounding data.

Remote sensing observations, products, and simulations are fundamental sources of information to monitor our planet and its climate variability. Uncovering the main modes of spatial and temporal variability in Earth data is essential to analyze and understand the underlying physical dynamics and processes driving the Earth System. Dimensionality reduction methods can work with spatio-temporal data sets and decompose the information efficiently. Principal component analysis (PCA), also known as empirical orthogonal functions (EOFs) in geophysics, has been traditionally used to analyze climatic data. However, when nonlinear feature relations are present, PCA/EOF fails. In this article, we propose a nonlinear PCA method to deal with spatio-temporal Earth system data. The proposed method, called rotated complex kernel PCA (ROCK-PCA for short), works in reproducing kernel Hilbert spaces to account for nonlinear processes, operates in the complex kernel domain to account for both space and time features, and adds an extra rotation for improved flexibility. The result is an explicitly resolved spatio-temporal decomposition of the Earth data cube. The method is unsupervised and computationally very efficient. We illustrate its ability to uncover spatio-temporal patterns using synthetic experiments and real data. Results of the decomposition of three essential climate variables are shown: satellite-based global gross primary productivity (GPP), soil moisture (SM), and reanalysis sea surface temperature (SST) data. The ROCK-PCA method allows identifying their annual and seasonal oscillations, as well as their nonseasonal trends and spatial variability patterns. The main modes of variability of GPP and SM match expected distributions of land-cover and eco-hydrological zones, respectively; the interannual component of SM is shown to be highly correlated with El Niño Southern Oscillation (ENSO) phenomenon; and the SST annual oscillation is perfectly uncoupled in magnitude and phase from the global warming trend and ENSO anomalies, as well as from their mutual interactions. We provide the working source code of the presented method for the interested reader in https://github.com/DiegoBueso/ROCK-PCA.

In many remote sensing applications, one wants to estimate variables or parameters of interest from observations. When the target variable is available at a resolution that matches the remote sensing observations, standard algorithms, such as neural networks, random forests, or the Gaussian processes, are readily available to relate the two. However, we often encounter situations where the target variable is only available at the group level, i.e., collectively associated with a number of remotely sensed observations. This problem setting is known in statistics and machine learning as multiple instance learning (MIL) or distribution regression (DR). This article introduces a nonlinear (kernel-based) method for DR that solves the previous problems without making any assumption on the statistics of the grouped data. The presented formulation considers distribution embeddings in reproducing kernel Hilbert spaces and performs standard least squares regression with the empirical means therein. A flexible version to deal with multisource data of different dimensionality and sample sizes is also presented and evaluated. It allows working with the native spatial resolution of each sensor, avoiding the need for matchup procedures. Noting the large computational cost of the approach, we introduce an efficient version via random Fourier features to cope with millions of points and groups. Real experiments involve the Soil Moisture Active Passive (SMAP) vegetation optical depth (VOD) data for the estimation of crop production in the U.S. Corn Belt and the Moderate Resolution Imaging Spectroradiometer (MODIS) and Multi-angle Imaging SpectroRadiometer (MISR) reflectances for the estimation of aerosol optical depth (AOD). An exhaustive empirical evaluation of the method is done against naive (linear and nonlinear) approaches based on input-space means as well as previously presented methods for MIL. We provide source code of our methods in http://isp.uv.es/code/dr.html.

In this paper, we analyze the effect of spatial and spectral compression on the performance of statistically based retrieval. Although the quality of the information is not completely preserved during the coding process, experiments reveal that a certain amount of compression may yield a positive impact on the accuracy of retrievals. We unveil two strategies, both with interesting benefits: either to apply a very high compression, which still maintains the same retrieval performance as that obtained for uncompressed data; or to apply a moderate to high compression, which improves the performance. As a second contribution of this paper, we focus on the origins of these benefits. On the one hand, we show that a certain amount of noise is removed during the compression stage, which benefits the retrievals performance. On the other hand, we analyze the effect of compression on spectral/spatial regularization (smoothing). We quantify the amount of information shared among the spatial neighbors for the different methods and compression ratios. We also propose a simple strategy to specifically exploit spectral and spatial relations and find that, when these relations are taken into account beforehand, the benefits of compression are reduced. These experiments suggest that compression can be understood as an indirect way to regularize the data and exploit spatial neighbors information, which improves the performance of pixelwise statistics-based retrieval algorithms.

Anomalous change detection (ACD) is an important problem in remote sensing image processing. Detecting not only pervasive but also anomalous or extreme changes has many applications for which methodologies are available. This paper introduces a nonlinear extension of a full family of anomalous change detectors. In particular, we focus on algorithms that utilize Gaussian and elliptically contoured (EC) distribution and extend them to their nonlinear counterparts based on the theory of reproducing kernels' Hilbert space. We illustrate the performance of the kernel methods introduced in both pervasive and ACD problems with real and simulated changes in multispectral and hyperspectral imagery with different resolutions (AVIRIS, Sentinel-2, WorldView-2, and Quickbird). A wide range of situations is studied in real examples, including droughts, wildfires, and urbanization. Excellent performance in terms of detection accuracy compared to linear formulations is achieved, resulting in improved detection accuracy and reduced false-alarm rates. Results also reveal that the EC assumption may be still valid in Hilbert spaces. We provide an implementation of the algorithms as well as a database of natural anomalous changes in real scenarios http://isp.uv.es/kacd.html.

Establishing causal relations between random variables from observational data is perhaps the most important challenge in today's science. In remote sensing and geosciences, this is of special relevance to better understand the earth's system and the complex interactions between the governing processes. In this paper, we focus on an observational causal inference, and thus, we try to estimate the correct direction of causation using a finite set of empirical data. In addition, we focus on the more complex bivariate scenario that requires strong assumptions and no conditional independence tests can be used. In particular, we explore the framework of (nondeterministic) additive noise models, which relies on the principle of independence between the cause and the generating mechanism. A practical algorithmic instantiation of such principle only requires: 1) two regression models in the forward and backward directions and 2) the estimation of statistical independence between the obtained residuals and the observations. The direction leading to more independent residuals is decided to be the cause. We instead propose a criterion that uses the sensitivity (derivative) of the dependence estimator, the sensitivity criterion allows to identify samples most affecting the dependence measure, and hence, the criterion is robust to spurious detections. We illustrate the performance in a collection of 28 geoscience causal inference problems, a database of radiative transfer models simulations and machine learning emulators in vegetation parameter modeling involving 182 problems, and assessing the impact of different regression models in a carbon cycle problem. The criterion achieves the state-of-the-art detection rates in all cases, and it is generally robust to noise sources and distortions. The presented approach confirms the validity in observational bivariate problems in the earth sciences.

Physically based radiative transfer models (RTMs) are widely used in Earth observation to understand the radiation processes occurring on the Earth's surface and their interactions with water, vegetation, and atmosphere. Through continuous improvements, RTMs have increased in accuracy and representativity of complex scenes at expenses of an increase in complexity and computation time, making them impractical in various remote sensing applications. To overcome this limitation, the common practice is to precompute large lookup tables (LUTs) for their later interpolation. To further reduce the RTM computation burden and the error in LUT interpolation, we have developed a method to automatically select the minimum and optimal set of input-output points (nodes) to be included in an LUT. We present the gradient-based automatic LUT generator algorithm (GALGA), which relies on the notion of an acquisition function that incorporates: 1) the Jacobian evaluation of an RTM and 2) the information about the multivariate distribution of the current nodes. We illustrate the capabilities of GALGA in the automatic construction and optimization of MODTRAN-based LUTs of different dimensions of the input variables space. Our results indicate that when compared with a pseudorandom homogeneous distribution of the LUT nodes, GALGA reduces:1) the LUT size by >24%; 2) the computation time by 27%; and 3) the maximum interpolation relative errors by at least 10%. It is concluded that an automatic LUT design might benefit from the methodology proposed in GALGA to reduce interpolation errors and computation time in computationally expensive RTMs.

Soil moisture (SM) is a key state variable of the hydrological cycle, needed to monitor the effects of a changing climate on natural resources. Soil moisture is highly variable in space and time, presenting seasonalities, anomalies and long-term trends, but also, and important nonlinear behaviours. Here, we introduce a novel fast and nonlinear complex PCA method to analyze the spatio-temporal patterns of the Earth's surface SM. We use global SM estimates acquired during the period 2010-2017 by ESA's SMOS mission. Our approach unveils both time and space modes, trends and periodicities unlike standard PCA decompositions. Results show the distribution of the total SM variance among its different components, and indicate the dominant modes of temporal variability in surface soil moisture for different regions. The relationship of the derived SM spatio-temporal patterns with E1 Niño Southern Oscillation (ENSO) conditions is also explored.

This paper introduces a novel statistical regression framework that allows the incorporation of consistency constraints. A linear and nonlinear (kernel-based) formulation are introduced, and both imply closed-form analytical solutions. The models exploit all the information from a set of drivers while being maximally independent of a set of auxiliary, protected variables. We successfully illustrate the performance in the estimation of chlorophyll content.

In this work we describe the methodology applied for the retrieval of global LAI, FAPAR and FVC from Advanced Very High Resolution Radiometer (AVHRR) on board the Meteorological-Operational (MetOp) polar orbiting satellites also known as EUMETSAT Polar System (EPS). A novel approach has been developed for the joint retrieval of three parameters (LAI, FVC, and FAPAR) instead of training one model per parameter. The method relies on multi-output Gaussian Processes Regression (GPR) trained over PROSAIL EPS simulations. A sensitivity analysis is performed to assess several sources of uncertainties in retrievals and maximize the positive impact of modeling the noise in training simulations. We describes the main features of the operational processing chain along with the current status of the global EPS vegetation products, including details about its overall quality and preliminary assessment of the products based on intercomparsion with equivalent (MODIS, PROBA-V) satellite vegetation products.

In this work we propose a method to find anomalous changes in remote sensing images based on the chronochrome approach. A regressor between images is used to discover the most influential points in the observed data. Typically, the pixels with largest residuals are decided to be anomalous changes. In order to find the anomalous pixels we consider the Cook distance and propose its nonlinear extension using random Fourier features as an efficient nonlinear measure of impact. Good empirical performance is shown over different multispectral images both visually and quantitatively evaluated with ROC curves.

Gaussian Processes (GPs) are a class of kernel methods that have shown to be very useful in geoscience applications. They are widely used because they are simple, flexible and provide very accurate estimates for nonlinear problems, especially in parameter retrieval. An addition to a predictive mean function, GPs come equipped with a useful property: the predictive variance function which provides confidence intervals for the predictions. The GP formulation usually assumes that there is no input noise in the training and testing points, only in the observations. However, this is often not the case in Earth observation problems where an accurate assessment of the instrument error is usually available. In this paper, we showcase how the derivative of a GP model can be used to provide an analytical error propagation formulation and we analyze the predictive variance and the propagated error terms in a temperature prediction problem from infrared sounding data.

The Infrared Atmospheric Sounding Interferometer (IASI) on board the MetOp satellite series provides important measurements for Numerical Weather Prediction (NWP). Retrieving accurate atmospheric parameters from the raw data provided by IASI is a large challenge, but necessary in order to use the data in NWP models. Statistical models performance is compromised because of the extremely high spectral dimensionality and the high number of variables to be predicted simultaneously across the atmospheric column. All this poses a challenge for selecting and studying optimal models and processing schemes. Earlier work has shown non-linear models such as kernel methods and neural networks perform well on this task, but both schemes are computationally heavy on large quantities of data. Kernel methods do not scale well with the number of training data, and neural networks require setting critical hyperparameters. In this work we follow an alternative pathway: we study transfer learning in convolutional neural nets (CNN s) to alleviate the retraining cost by departing from proxy solutions (either features or networks) obtained from previously trained models for related variables. We show how features extracted from the IASI data by a CNN trained to predict a physical variable can be used as inputs to another statistical method designed to predict a different physical variable at low altitude. In addition, the learned parameters can be transferred to another CNN model and obtain results equivalent to those obtained when using a CNN trained from scratch requiring only fine tuning.

In this work we evaluate multi-output (MO) Gaussian Process (GP) models based on the linear model of coregionalization (LMC) for estimation of biophysical parameter variables under a gap filling setup. In particular, we focus on LAI and fAPAR over rice areas. We show how this problem cannot be solved with standard single-output (SO) GP models, and how the proposed MO-GP models are able to successfully predict these variables even in high missing data regimes, by implicitly performing an across-domain information transfer.

Products derived from a single multispectral sensor are hampered by a limited spatial, spectral or temporal resolutions. Image fusion in general and downscaling/blending in particular allow to combine different multiresolution datasets. We present here an optimal interpolation approach to generate smoothed and gap-free time series of Landsat reflectance data. We fuse MODIS (moderate-resolution imaging spectroradiometer) and Landsat data globally using the Google Earth Engine (GEE) platform. The optimal interpolator exploits GEE ability to ingest large amounts of data (Landsat climatologies) and uses simple linear operations that scale easily in the cloud. The approach shows very good results in practice, as tested over five sites with different vegetation types and climatic characteristics in the contiguous US.

This work tackles the target detection problem through the well-known global RX method. The RX method models the clutter as a multivariate Gaussian distribution, and has been extended to nonlinear distributions using kernel methods. While the kernel RX can cope with complex clutters, it requires a considerable amount of computational resources as the number of clutter pixels gets larger. Here we propose random Fourier features to approximate the Gaussian kernel in kernel RX and consequently our development keep the accuracy of the nonlinearity while reducing the computational cost which is now controlled by an hyperparameter. Results over both synthetic and real-world image target detection problems show space and time efficiency of the proposed method while providing high detection performance.

We here introduce an automatic Digital Terrain Model (DTM) extraction method. The proposed sparsity-driven DTM extractor (SD-DTM) takes a high-resolution Digital Surface Model (DSM) as an input and constructs a high-resolution DTM using the variational framework. To obtain an accurate DTM, an iterative approach is proposed for the minimization of the target variational cost function. Accuracy of the SD-DTM is shown in a real-world DSM data set. We show the efficiency and effectiveness of the approach both visually and quantitatively via residual plots in illustrative terrain types.

Measurements of soil moisture are needed for a better global understanding of the land surface-climate feedbacks at both the local and the global scale. Satellite sensors operating in the low frequency microwave spectrum (from 1 to 10 GHz) have proven to be suitable for soil moisture retrievals. These sensors now cover nearly 4 decades thus allowing for global multi-mission climate data records. In this paper, we assess the possibility of using L-band (SMOS) and C-band (AMSR2, ASCAT) remotely sensed soil moisture time series for the global estimation of soil moisture persistence. A multi -output Gaussian process regression model is applied to ensure spatio-temporal coverage of the satellite data sets. It allows a robust computation of temporal autocorrelation and e- folding times. Results over a selection of catchments reveals general agreement between the response of in-situ and satellite microwave observations to hydrological processes. The response of the uppermost-modeled soil moisture layer of GLDAS-1-Noah agrees well with that of the observations, whereas major differences are displayed by MERRA2 reanalysis. The temporal dynamics of the three microwave sensors are shown to be consistent, close to in-situ and to GLDAS-1- Noah, which supports their combination for the global estimation soil moisture persistence.

Water quality parameters are derived applying several machine learning regression methods on the Case2eXtreme dataset (C2X). The used data are based on Hydrolight in-water radiative transfer simulations at Sentinel-3 OLCI wavebands, and the application is done exclusively for absorbing waters with high concentrations of coloured dissolved organic matter (CDOM). The regression approaches are: regularized linear, random forest, Kernel ridge, Gaussian process and support vector regressors. The validation is made with and an independent simulation dataset. A comparison with the OLCI Neural Network Swarm (ONSS) is made as well. The best approached is applied to a sample scene and compared with the standard OLCI product delivered by EUMETSAT/ESA.

This paper introduces deep Gaussian processes (DGPs) for geo-physical parameter retrieval. Unlike the standard full GP model, the DGP accounts for complicated (modular, hierarchical) processes, provides a efficient solution that scales well to large datasets, and improves prediction accuracy over standard full and sparse GP models. We give empirical evidence of performance for estimation of surface dew point temperature from infrared sounding data.

This paper introduces a methodology to construct emulators of costly radiative transfer models (RTMs). The proposed methodology is sequential and adaptive, and it is based on the notion of acquisition functions in Bayesian optimization. Here, instead of optimizing the unknown underlying RTM function, one aims to achieve accurate approximations. The Automatic Multi-Output Gaussian Process Emulator (AMO-GAPE) methodology combines the interpolation capabilities of Gaussian processes (GPs) with the accurate design of an acquisition function that favors sampling in low density regions and flatness of the interpolation function. We illustrate the promising capabilities of the method for the construction of an emulator for a standard leaf-canopy RTM.

We present a deep learning model with temporal memory to detect clouds in image time series acquired by the Seviri imager mounted on the Meteosat Second Generation (MSG) satellite. The model provides pixel-level cloud maps with related confidence and propagates information in time via a recurrent neural network structure. With a single model, we are able to outline clouds along all year and during day and night with high accuracy.

End-to-end mission performance simulators (E2ES) are suitable tools to accelerate satellite mission development from concet to deployment. One core element of these E2ES is the generation of synthetic scenes that are observed by the various instruments of an Earth Observation mission. The generation of these scenes rely on Radiative Transfer Models (RTM) for the simulation of light interaction with the Earth surface and atmosphere. However, the execution of advanced RTMs is impractical due to their large computation burden. Classical interpolation and statistical emulation methods of pre-computed Look-Up Tables (LUT) are therefore common practice to generate synthetic scenes in a reasonable time. This work evaluates the accuracy and computation cost of interpolation and emulation methods to sample the input LUT variable space. The results on MONDTRAN-based top-of-atmosphere radiance data show that Gaussian Process emulators produced more accurate output spectra than linear interpolation at a fraction of its time. It is concluded that emulation can function as a fast and more accurate alternative to interpolation for LUT parameter space sampling.

This letter introduces warped Gaussian process (WGP) regression in remote sensing applications. WGP models output observations as a parametric nonlinear transformation of a GP. The parameters of such a prior model are then learned via standard maximum likelihood. We show the good performance of the proposed model for the estimation of oceanic chlorophyll content from multispectral data, vegetation parameters (chlorophyll, leaf area index, and fractional vegetation cover) from hyperspectral data, and in the detection of the causal direction in a collection of 28 bivariate geoscience and remote sensing causal problems. The model consistently performs better than the standard GP and the more advanced heteroscedastic GP model, both in terms of accuracy and more sensible confidence intervals.

Current remote sensing image classification problems have to deal with an unprecedented amount of heterogeneous and complex data sources. Upcoming missions will soon provide large data streams that will make land cover/use classification difficult. Machine-learning classifiers can help at this, and many methods are currently available. A popular kernel classifier is the Gaussian process classifier (GPC), since it approaches the classification problem with a solid probabilistic treatment, thus yielding confidence intervals for the predictions as well as very competitive results to the state-of-the-art neural networks and support vector machines. However, its computational cost is prohibitive for large-scale applications, and constitutes the main obstacle precluding wide adoption. This paper tackles this problem by introducing two novel efficient methodologies for GP classification. We first include the standard random Fourier features approximation into GPC, which largely decreases its computational cost and permits large-scale remote sensing image classification. In addition, we propose a model which avoids randomly sampling a number of Fourier frequencies and alternatively learns the optimal ones within a variational Bayes approach. The performance of the proposed methods is illustrated in complex problems of cloud detection from multispectral imagery and infrared sounding data. Excellent empirical results support the proposal in both computational cost and accuracy.

Solving inverse problems is central in geosciences and remote sensing. The radiative transfer models (RTMs) represent mathematically the physical laws that rule the phenomena in remote sensing applications (forward models). The numerical inversion of the RTM equations is a challenging and computationally demanding problem. For this reason, often the application of a simpler statistical regression is preferred. In general, the regression models predict the biophysical parameter of interest from the corresponding received radiance, learning a mapping from in situ data. However, this approach does not employ the physical information encoded in the RTMs. An alternative strategy, which attempts to include the physical knowledge, consists in learning a regression model trained using simulated data by an RTM code. In this paper, we introduce a nonlinear nonparametric regression model that combines the benefits of the two aforementioned approaches. The inversion is performed considering jointly both real observations and RTM-simulated data. The proposed joint Gaussian process (JGP) provides a solid framework for exploiting the regularities between the two types of data, in order to perform inverse modeling. The JGP automatically detects the relative quality of the simulated and real data, and combines them properly. This occurs by learning an additional hyperparameter with respect to a standard Gaussian process model, so that the novel scheme is at the same time simple and robust, i.e., capable of adapting to different scenarios. The advantages of the JGP method compared with benchmark strategies are shown considering synthetic and real data in different experiments. Specifically, we consider leaf area index retrieval from Landsat data combined with simulated data generated by the PROSAIL model.

Passive Millimeter Wave Images (PMMWIs) are being increasingly used to identify and localize objects concealed under clothing. Taking into account the quality of these images and the unknown position, shape, and size of the hidden objects, large data sets are required to build successful classification/detection systems. Kernel methods, in particular Gaussian Processes (GPs), are sound, flexible, and popular techniques to address supervised learning problems. Unfortunately, their computational cost is known to be prohibitive for large scale applications. In this work, we present a novel approach to PMMWI classification based on the use of Gaussian Processes for large data sets. The proposed methodology relies on linear approximations to kernel functions through random Fourier features. Model hyperparameters are learned within a variational Bayes inference scheme. Our proposal is well suited for real-time applications, since its computational cost at training and test times is much lower than the original GP formulation. The proposed approach is tested on a unique, large, and real PMMWI database containing a broad variety of sizes, types, and locations of hidden objects.

This paper introduces a methodology for predicting the year of plantation (YOP) from remote sensing data. The application has important implications in forestry management and inventorying. We exploit hyperspectral and LiDAR data in combination with state-of-the-art machine learning classifiers. In particular, we present a complete processing chain to extract spectral, textural and morphological features from both sensory data. Features are then combined and fed a Gaussian Process Classifier (GPC) trained to predict YOP in a forest area in North Carolina (US). The GPC algorithm provides accurate YOP estimates, reports spatially explicit maps and associated confidence maps, and provides sensible feature rankings.

This paper presents the development and implementation of a cloud detection algorithm for Proba-V. Accurate and automatic detection of clouds in satellite scenes is a key issue for a wide range of remote sensing applications. With no accurate cloud masking, undetected clouds are one of the most significant sources of error in both sea and land cover biophysical parameter retrieval. The objective of the algorithms presented in this paper is to detect clouds accurately providing a cloud flag per pixel. For this purpose, the method exploits the information of Proba-V using statistical machine learning techniques to identify the clouds present in Proba-V products. The effectiveness of the proposed method is successfully illustrated using a large number of real Proba-V images.

Emissivity is one of the most important parameters to improve the determination of the troposphere properties (thermodynamic properties, aerosols and trace gases concentration) and it is essential to estimate the radiative budget. With the second generation of infrared sounders, we can estimate emissivity spectra at high spectral resolution, which gives us a global view and long-term monitoring of continental surfaces. Statistically, this is an ill-posed retrieval problem, with as many output variables as inputs. We here propose nonlinear multi-output statistical regression based on kernel methods to estimate spectral emissivity given the radiances. Kernel methods can cope with high-dimensional input-output spaces efficiently. We give empirical evidence of models performance on Infrared Atmospheric Sounding Interferometer (IASI) simulated data. Kernel regression model largely improves previous least squares linear regression model quantitatively, with an average reduction of 25% in mean-square error.

In this paper we present a combined strategy for the retrieval of atmospheric profiles from infrared sounders. The approach considers the spatial information and a noise-dependent dimensionality reduction approach. The extracted features are fed into a canonical linear regression. We compare Principal Component Analysis (PCA) and Minimum Noise Fraction (MNF) for dimensionality reduction, and study the compactness and information content of the extracted features. Assessment of the results is done on a big dataset covering many spatial and temporal situations. PCA is widely used for these purposes but our analysis shows that one can gain significant improvements of the error rates when using MNF instead. In our analysis we also investigate the relationship between error rate improvements when including more spectral and spatial components in the regression model, aiming to uncover the trade-off between model complexity and error rates.

This paper introduces an automatic methodology to construct emulators for costly radiative transfer models (RTMs). The proposed method is sequential and adaptive, and it is based on the notion of the acquisition function by which instead of optimizing the unknown RTM underlying function we propose to achieve accurate approximations. The Automatic Gaussian Process Emulator (AGAPE) methodology combines the interpolation capabilities of Gaussian processes (GPs) with the accurate design of an acquisition function that favors sampling in low density regions and flatness of the interpolation function. We illustrate the good capabilities of the method in toy examples and for the construction of an optimal look-up-table for atmospheric correction based on MODTRAN5.

Convolutional neural networks (CNN) have proven to be state of the art methods for many image classification tasks and their use is rapidly increasing in remote sensing problems. One of their major strengths is that, when enough data is available, CNN perform an end-to-end learning without the need of custom feature extraction methods. In this work, we study the use of different CNN architectures for cloud masking of Proba-V multispectral images. We compare such methods with the more classical machine learning approach based on feature extraction plus supervised classification. Experimental results suggest that CNN are a promising alternative for solving cloud masking problems.

This paper presents an efficient methodology for approximating kernel functions in Gaussian process classification (GPC). Two models are introduced. We first include the standard random Fourier features (RFF) approximation into GPC, which largely improves the computational efficiency and permits large scale remote sensing data classification. In addition, we develop a novel approach which avoids randomly sampling a number of Fourier frequencies, and alternatively learns the optimal ones using a variational Bayes approach. The performance of the proposed methods is illustrated in complex problems of cloud detection from multispectral imagery.

Establishing causal relations between random variables from observational data is perhaps the most important challenge in today's Science. In remote sensing and geosciences this is of special relevance to better understand the Earth's system and the complex and elusive interactions between processes. In this paper we explore a framework to derive cause-effect relations from pairs of variables via regression and dependence estimation. We propose to focus on the sensitivity (curvature) of the dependence estimator to account for the asymmetry of the forward and inverse densities of approximation residuals. Results in a large collection of 28 geoscience causal inference problems demonstrate the good capabilities of the method.

The ESA's SMOS and the NASA's SMAP missions, launched in 2009 and 2015, respectively, are the first two missions having on-board L-band microwave sensors, which are very sensitive to the water content in soils and vegetation. Focusing on the vegetation signal at L-band, we have implemented an inversion approach for SMAP that allows deriving vegetation optical depth (VOD, a microwave parameter related to biomass and plant water content) alongside soil moisture, without reliance on ancillary optical information on vegetation. This work aims at using this new observational data to monitor the phenology of crops in major global agro-ecosystems and enhance present agricultural monitoring and prediction capabilities. Core agricultural regions have been selected worldwide covering major crops (corn, soybean, wheat, rice). The complementarity and synergies between the microwave vegetation signal, sensitive to biomass water-uptake dynamics, and optical indices, sensitive to canopy greenness, are explored. Results reveal the value of L-band VOD as an independent ecological indicator for global terrestrial biosphere studies.1

The coloured dissolved organic matter (CDOM) concentration is the standard measure of humic substance in natural waters. CDOM measurements by remote sensing is calculated using the absorption coefficient (a) at a certain wavelength (e.g. ≈ 440nm). This paper presents a comparison of four machine learning methods for the retrieval of CDOM from remote sensing signals: regularized linear regression (RLR), random forest (RF), kernel ridge regression (KRR) and Gaussian process regression (GPR). Results are compared with the established polynomial regression algorithms. RLR is revealed as the simplest and most efficient method, followed closely by its nonlinear counterpart KRR.

Solving inverse problems is central in geosciences and remote sensing. Very often a mechanistic physical model of the system exists that solves the forward problem. Inverting the implied radiative transfer model (RTM) equations numerically implies, however, challenging and computationally demanding problems. Statistical models tackle the inverse problem and predict the biophysical parameter of interest from radiance data, exploiting either in situ data or simulated data from an RTM. We introduce a novel nonlinear and nonparametric statistical inversion model which incorporates both real observations and RTM-simulated data. The proposed Joint Gaussian Process (JGP) provides a solid framework for exploiting the regularities between the two types of data, in order to perform inverse modeling. Advantages of the JGP method over competing strategies are shown on both a simple toy example and in leaf area index (LAI) retrieval from Landsat data combined with simulated data generated by the PROSAIL model.

Automatic land cover classification from satellite image time series is of paramount relevance to assess vegetation and crop status, with important implications in agriculture, biofuels, and food. However, due to the high cost and human resources needed to characterize and classify land cover through field campaigns, a recurrent limiting factor is the lack of available labeled data. On top of this, the biophysical-geophysical variables exhibit particular temporal structures that need to be exploited. Land cover classification based on image time series is very complex because of the data manifold distortions through time. We propose the use of the kernel manifold alignment (KEMA) method for domain adaptation of remote sensing time series before classification. KEMA is nonlinear and semisupervised and reduces to solve a simple generalized eigenproblem. We give empirical evidence of performance through classification of biophysical (leaf area index, fraction of absorbed photosynthetically active radiation, fractional vegetation cover, and normalized difference vegetation index) time series on a global scale.

The infrared atmospheric sounding interferometer (IASI) is flying on board of the Metop satellite series, which is part of the EUMETSAT Polar System. Products obtained from IASI data represent a significant improvement in the accuracy and quality of the measurements used for meteorological models. Notably, the IASI collects rich spectral information to derive temperature and moisture profiles, among other relevant trace gases, essential for atmospheric forecasts and for the understanding of weather. Here, we investigate the impact of near-lossless and lossy compression on IASI L1C data when statistical retrieval algorithms are later applied. We search for those compression ratios that yield a positive impact on the accuracy of the statistical retrievals. The compression techniques help reduce certain amount of noise on the original data and, at the same time, incorporate spatial-spectral feature relations in an indirect way without increasing the computational complexity. We observed that compressing images, at relatively low bit rates, improves results in predicting temperature and dew point temperature, and we advocate that some amount of compression prior to model inversion is beneficial. This research can benefit the development of current and upcoming retrieval chains in infrared sounding and hyperspectral sensors.

This brief addresses two main issues of the standard kernel entropy component analysis (KECA) algorithm: the optimization of the kernel decomposition and the optimization of the Gaussian kernel parameter. KECA roughly reduces to a sorting of the importance of kernel eigenvectors by entropy instead of variance, as in the kernel principal components analysis. In this brief, we propose an extension of the KECA method, named optimized KECA (OKECA), that directly extracts the optimal features retaining most of the data entropy by means of compacting the information in very few features (often in just one or two). The proposed method produces features which have higher expressive power. In particular, it is based on the independent component analysis framework, and introduces an extra rotation to the eigen decomposition, which is optimized via gradient-ascent search. This maximum entropy preservation suggests that OKECA features are more efficient than KECA features for density estimation. In addition, a critical issue in both the methods is the selection of the kernel parameter, since it critically affects the resulting performance. Here, we analyze the most common kernel length-scale selection criteria. The results of both the methods are illustrated in different synthetic and real problems. Results show that OKECA returns projections with more expressive power than KECA, the most successful rule for estimating the kernel parameter is based on maximum likelihood, and OKECA is more robust to the selection of the length-scale parameter in kernel density estimation.

Kernel-based machine learning regression algorithms (MLRAs) are potentially powerful methods for being implemented into operational biophysical variable retrieval schemes. However, they face difficulties in coping with large training data sets. With the increasing amount of optical remote sensing data made available for analysis and the possibility of using a large amount of simulated data from radiative transfer models (RTMs) to train kernel MLRAs, efficient data reduction techniques will need to be implemented. Active learning (AL) methods enable to select the most informative samples in a data set. This letter introduces six AL methods for achieving optimized biophysical variable estimation with a manageable training data set, and their implementation into a Matlab-based MLRA toolbox for semiautomatic use. The AL methods were analyzed on their efficiency of improving the estimation accuracy of the leaf area index and chlorophyll content based on PROSAIL simulations. Each of the implemented methods outperformed random sampling, improving retrieval accuracy with lower sampling rates. Practically, AL methods open opportunities to feed advanced MLRAs with RTM-generated training data for the development of operational retrieval models.

This paper introduces the use of single-layer and deep convolutional networks for remote sensing data analysis. Direct application to multi- and hyperspectral imagery of supervised (shallow or deep) convolutional networks is very challenging given the high input data dimensionality and the relatively small amount of available labeled data. Therefore, we propose the use of greedy layerwise unsupervised pretraining coupled with a highly efficient algorithm for unsupervised learning of sparse features. The algorithm is rooted on sparse representations and enforces both population and lifetime sparsity of the extracted features, simultaneously. We successfully illustrate the expressive power of the extracted representations in several scenarios: classification of aerial scenes, as well as land-use classification in very high resolution or land-cover classification from multi- and hyperspectral images. The proposed algorithm clearly outperforms standard principal component analysis (PCA) and its kernel counterpart (kPCA), as well as current state-of-the-art algorithms of aerial classification, while being extremely computationally efficient at learning representations of data. Results show that single-layer convolutional networks can extract powerful discriminative features only when the receptive field accounts for neighboring pixels and are preferred when the classification requires high resolution and detailed results. However, deep architectures significantly outperform single-layer variants, capturing increasing levels of abstraction and complexity throughout the feature hierarchy.

Gaussian Process Regression (GPR) for machine learning has lately been successfully introduced for chlorophyll content mapping from remotely sensed data. The method provides a fast, stable and accurate prediction of biophysical parameters. However, since GPR is a non-linear kernel regression method, the relevance of the features are not accessible. In this paper, we introduce a probabilistic approach for feature sensitivity analysis (SA) of the GPR in order to reveal the relative importance of the features (bands) being used in the regression process. We evaluated the SA on GPR ocean chlorophyll content prediction. The method revealed the importance of the spectral bands, thus allowing the discrimination between Case-1 water and Case-2 water conditions.

This paper proposes a full Earth observation processing chaing for biophysical parameter estimation at local scales. In particular, we focus on the Leaf Area Index (LAI) as an essential climate variable required for the monitoring and modeling of land surfaces at local scale. The main goal of this study is tied to the use of optical satellite images to retrieve Earth Observation (EO) biophysical parameters able to describe the spatio-temporal changes in agro-ecosystems at local scale. The objective of this work is two-fold: (i) to set up and update the EO products processing chain at high resolution (local) scale; and (ii) derive multitemporal LAI maps at 30 m resolution to be fed into a crop model. The processing chain includes the combination of surface reflectance products from Landat 7 ETM+ and Landsat 8 OLI. The results of the processing chain used and the retrieval approach are encouraging for crop monitoring at local scale.

This paper studies the level of complementary information conveyed by extremely high resolution LiDAR and optical images. We pursue this goal following an indirect approach via unsupervised spatial-spectral feature extraction. We used a recently presented unsupervised convolutional neural network trained to enforce both population and lifetime spar-sity in the feature representation. We derived independent and joint feature representations, and analyzed the sparsity scores and the discriminative power. Interestingly, the obtained results revealed that the RGB+LiDAR representation is no longer sparse, and the derived basis functions merge color and elevation yielding a set of more expressive colored edge filters. The joint feature representation is also more discriminative when used for clustering and topological data visualization.

The accurate estimation of carbon and heat fluxes at global scale is paramount for future policy decisions in the context of global climate change. This paper analyzes the relative relevance of potential remote sensing and meteorological drivers of global carbon and energy fluxes over land. The study is done in an indirect way via upscaling both Gross Primary Production (GPP) and latent energy (LE) using Gaussian Process regression (GPR). In summary, GPR is successfully compared to multivariate linear regression (RMSE gain of +4.17% in GPP and +7.63% in LE) and kernel ridge regression (+2.91% in GPP and +3.07% in LE). The best GP models are then studied in terms of explanatory power based on the analysis of the lengthscales of the anisotropic covariance function, sensitivity maps of the predictive mean, and the robustness to distortions in the input variables. It is concluded that GPP is predominantly mediated by several vegetation indices and land surface temperature (LST), while LE is mostly driven by LST, global radiation and vegetation indices.

This paper deals with the development and implementation of a cloud screening algorithm for image time series, with the focus on the forthcoming Sentinel-2 satellites to be launched under the ESA Copernicus Programme. The proposed methodology is based on kernel ridge regression and exploits the temporal information to detect anomalous changes that correspond to cloud covers. The huge data volumes to be processed when dealing with high temporal, spatial, and spectral resolution datasets motivate the implementation of the algorithm within distributed computer resources. In consequence, an operational cloud screening service has been specifically designed and implemented in the frame of the Sentinels Synergy Framework (SenSyF). The effectiveness of the proposed method is successfully illustrated using a time series dataset with a 5-day revisit derived from SPOT-4 at high resolution, which has been collected by ESA in preparation for the exploitation of the Sentinel-2 mission.

Manifold alignment has become very popular in recent literature. Aligning data distributions prior to product generation is an appealing strategy, since it allows to provide data spaces that are more similar to each other, regardless of the subsequent use of the transformed data. We propose a methodology that finds a common representation among data spaces from different sensors using geographic image correspondences, or semantic ties. To cope with the strong deformations between the data spaces considered, we propose to add nonlinearities by expanding the input space with Gaussian Radial Basis Function (RBF) features with respect to the centroids of a partitioning of the data. Such features allow us to cope with nonlinear transformations, while keeping a simple and efficient linear formulation. The proposed method is multi-domain and does not require co-registration, rather only a partial degree of spatial overlap. We test it on a challenging problem of multisensor classification transferring a model trained on a WorldView 2 image to predict land cover of a 3-bands orthophoto and show that we can transfer the model with an accuracy comparable to the one that would have been obtained by a model trained on the target image with an image-specific ground truth.

Kernel methods constitute a family of powerful machine learning algorithms, which have found wide use in remote sensing and geosciences. However, kernel methods are still not widely adopted because of the high computational cost when dealing with large scale problems, such as the inversion of radiative transfer models. This paper introduces the method of random kitchen sinks (RKS) for fast statistical retrieval of bio-geo-physical parameters. The RKS method allows to approximate a kernel matrix with a set of random bases sampled from the Fourier domain. We extend their use to other bases, such as wavelets, stumps, and Walsh expansions. We show that kernel regression is now possible for datasets with millions of examples and high dimensionality. Examples on atmospheric parameter retrieval from infrared sounders and biophysical parameter retrieval by inverting PROSAIL radiative transfer models with simulated Sentinel-2 data show the effectiveness of the technique.

This paper focuses on biophysical parameter retrieval based on Gaussian Processes (GPs). Very often an arbitrary transformation is applied to the observed variable (e.g. chlorophyll content) to better pose the problem. This standard practice essentially tries to linearize/uniformize the distribution by applying non-linear link functions like the logarithmic, the exponential or the logistic functions. In this paper, we propose to use a GP model that automatically learns the optimal transformation directly from the data. The so-called warped GP regression (WGPR) presented in [1] models output observations as a parametric nonlinear transformation of a GP. The parameters of such prior model are then learned via standard maximum likelihood. We show the good performance of the proposed model for the estimation of oceanic chlorophyll content, which outperforms the regular GPR and a more advanced heteroscedastic GPR model.

Physically-based radiative transfer models (RTMs) help in understanding the processes occurring on the Earth's surface and their interactions with vegetation and atmosphere. However, advanced RTMs can take a long computational time, which makes them unfeasible in many real applications. To overcome this problem, it has been proposed to substitute RTMs through so-called emulators. Emulators are statistical models that approximate the functioning of RTMs. They are advantageous in real practice because of the computational efficiency and excellent accuracy and flexibility for extrapolation. We here present an `Emulator toolbox' that enables analyzing three multi-output machine learning regression algorithms (MO-MLRAs) on their ability to approximate an RTM. As a proof of concept, a case study on emulating sun-induced fluorescence (SIF) is presented. The toolbox is foreseen to open new opportunities in the use of advanced RTMs, in which both consistent physical assumptions and data-driven machine learning algorithms live together.

Leaf area index (LAI) is a key biophysical parameter used to determine foliage cover and crop growth in environmental studies. Smartphones are nowadays ubiquitous sensor devices with high computational power, moderate cost, and high-quality sensors. A smartphone app, which is called PocketLAI, was recently presented and tested for acquiring ground LAI estimates. In this letter, we explore the use of state-of-the-art nonlinear Gaussian process regression (GPR) to derive spatially explicit LAI estimates over rice using ground data from PocketLAI and Landsat 8 imagery. GPR has gained popularity in recent years because of its solid Bayesian foundations that offer not only high accuracy but also confidence intervals for the retrievals. We show the first LAI maps obtained with ground data from a smartphone combined with advanced machine learning. This letter compares LAI predictions and confidence intervals of the retrievals obtained with PocketLAI with those obtained with classical instruments, such as digital hemispheric photography (DHP) and LI-COR LAI-2000. This letter shows that all three instruments obtained comparable results, but PocketLAI is far cheaper. The proposed methodology hence opens a wide range of possible applications at moderate cost.

Earth observation through remote sensing images allows the accurate characterization and identification of materials on the surface from space and airborne platforms. Multiple and heterogeneous image sources can be available for the same geographical region: multispectral, hyperspectral, radar, multitemporal, and multiangular images can today be acquired over a given scene. These sources can be combined/fused to improve classification of the materials on the surface. Even if this type of systems is generally accurate, the field is about to face new challenges: the upcoming constellations of satellite sensors will acquire large amounts of images of different spatial, spectral, angular, and temporal resolutions. In this scenario, multimodal image fusion stands out as the appropriate framework to address these problems. In this paper, we provide a taxonomical view of the field and review the current methodologies for multimodal classification of remote sensing images. We also highlight the most recent advances, which exploit synergies with machine learning and signal processing: sparse methods, kernel-based fusion, Markov modeling, and manifold alignment. Then, we illustrate the different approaches in seven challenging remote sensing applications: 1) multiresolution fusion for multispectral image classification; 2) image downscaling as a form of multitemporal image fusion and multidimensional interpolation among sensors of different spatial, spectral, and temporal resolutions; 3) multiangular image classification; 4) multisensor image fusion exploiting physically-based feature extractions; 5) multitemporal image classification of land covers in incomplete, inconsistent, and vague image sources; 6) spatiospectral multisensor fusion of optical and radar images for change detection; and 7) cross-sensor adaptation of classifiers. The adoption of these techniques in operational settings will help to monitor our planet from space in the very near future.

The transform in image coding aims to remove redundancy among data coefficients so that they can be independently coded, and to capture most of the image information in few coefficients. While the second goal ensures that discarding coefficients will not lead to large errors, the first goal ensures that simple (point-wise) coding schemes can be applied to the retained coefficients with optimal results. Principal Component Analysis (PCA) provides the best independence and data compaction for Gaussian sources. Yet, non-linear generalizations of PCA may provide better performance for more realistic non-Gaussian sources. Principal Polynomial Analysis (PPA) generalizes PCA by removing the non-linear relations among components using regression, and was analytically proved to perform better than PCA in dimensionality reduction. We explore here the suitability of reversible PPA for lossless compression of hyperspectral images. We found that reversible PPA performs worse than PCA due to the high impact of the rounding operation errors and to the amount of side information. We then propose two generalizations: Backwards PPA, where polynomial estimations are performed in reverse order, and Double-Sided PPA, where more than a single dimension is used in the predictions. Both yield better coding performance than canonical PPA and are comparable to PCA.

The re-use of available labeled samples to classify newly acquired data is a hot topic in pattern analysis and machine learning. Classification algorithms developed with data from one domain cannot be directly used in another related domain, unless the data representation or the classifier have been adapted to the new data distribution. This is crucial in satellite/airborne image analysis: when confronted to domain shifts issued from changes in acquisition or illumination conditions, image classifiers tend to become inaccurate. In this paper, we introduce a method to align data manifolds that represent the same land cover classes, but have undergone spectral distortions. The proposed method relies on a semi-supervised manifold alignment technique and relaxes the requirement of labeled data in all domains by exploiting centrality measures over graphs to match the manifolds. Experiments on multispectral pixel classification at very high spatial resolution show the potential of the method.

Due to technological constraints, hyperspectral earth observation imagery are often a mosaic of overlapping flight lines collected in different passes over the area of interest. This causes variations in aqcuisition conditions such that the reflected spectrum can vary significantly between these flight lines. Partly, this problem is solved by atmospherical correction, but residual spectral differences often remain. A probabilistic domain adaptation framework based on graph matching using Hidden Markov Random Fields was recently proposed for transforming hyperspectral data from one image to better correspond to the other. This paper investigates the use of scale and angle invariant textural features for improving the performance of the used Hidden Markov Random Field matching framework in the case of hyperspectral flight lines. These textural features are derived from the filtering of VHR optical imagery with a bank of Gabor filters with varying orientation, scale and frequency and subsequently rendering them invariant to scale and frequency by applying the 2D DFT on the filter responses in the scale and frequency space.

This paper presents a new framework for manifold learning based on a sequence of principal polynomials that capture the possibly nonlinear nature of the data. The proposed Principal Polynomial Analysis (PPA) generalizes PCA by modeling the directions of maximal variance by means of curves, instead of straight lines. Contrarily to previous approaches, PPA reduces to performing simple univariate regressions, which makes it computationally feasible and robust. Moreover, PPA shows a number of interesting analytical properties. First, PPA is a volume-preserving map, which in turn guarantees the existence of the inverse. Second, such an inverse can be obtained in closed form. Invertibility is an important advantage over other learning methods, because it permits to understand the identified features in the input domain where the data has physical meaning. Moreover, it allows to evaluate the performance of dimensionality reduction in sensible (input-domain) units. Volume preservation also allows an easy computation of information theoretic quantities, such as the reduction in multi-information after the transform. Third, the analytical nature of PPA leads to a clear geometrical interpretation of the manifold: it allows the computation of Frenet-Serret frames (local features) and of generalized curvatures at any point of the space. And fourth, the analytical Jacobian allows the computation of the metric induced by the data, thus generalizing the Mahalanobis distance. These properties are demonstrated theoretically and illustrated experimentally. The performance of PPA is evaluated in dimensionality and redundancy reduction, in both synthetic and real datasets from the UCI repository.

An accurate estimation of biophysical variables is the key to monitor our Planet. Leaf chlorophyll content helps in interpreting the chlorophyll fluorescence signal from space, whereas oceanic chlorophyll concentration allows us to quantify the healthiness of the oceans. Recently, the family of Bayesian nonparametric methods has provided excellent results in these situations. A particularly useful method in this framework is the Gaussian process regression (GPR). However, standard GPR assumes that the variance of the noise process is independent of the signal, which does not hold in most of the problems. In this letter, we propose a nonstandard variational approximation that allows accurate inference in signal-dependent noise scenarios. We show that the so-called variational heteroscedastic GPR (VHGPR) is an excellent alternative to standard GPR in two relevant Earth observation examples, namely, Chl vegetation retrieval from hyperspectral images and oceanic Chl concentration estimation from in situ measured reflectances. The proposed VHGPR outperforms the tested empirical approaches, as well as statistical linear regression (both least squares and least absolute shrinkage and selection operator), neural nets, and kernel ridge regression, and the homoscedastic GPR, in terms of accuracy and bias, and proves more robust when a low number of examples is available.

Solar irradiation prediction is an important problem in geosciences with direct applications in renewable energy. Recently, a high number of machine learning techniques have been introduced to tackle this problem, mostly based on neural networks and support vector machines. Gaussian process regression (GPR) is an alternative nonparametric method that provided excellent results in other biogeophysical parameter estimation. In this letter, we evaluate GPR for the estimation of solar irradiation. Noting the nonstationary temporal behavior of the signal, we develop a particular time-based composite covariance to account for the relevant seasonal signal variations. We use a unique meteorological data set acquired at a radiometric station that includes both measurements and radiosondes, as well as numerical weather prediction models. We show that the so-called temporal GPR outperforms ten state-of-the-art statistical regression algorithms (even when including time information) in terms of accuracy and bias, and it is more robust to the number of predictions used.

This paper presents a novel semisupervised kernel partial least squares (KPLS) algorithm for nonlinear feature extraction to tackle both land-cover classification and biophysical parameter retrieval problems. The proposed method finds projections of the original input data that align with the target variable (labels) and incorporates the wealth of unlabeled information to deal with low-sized or underrepresented data sets. The method relies on combining two kernel functions: the standard radial-basis-function kernel based on labeled information and a generative, i.e., probabilistic, kernel directly learned by clustering the data many times and at different scales across the data manifold. The construction of the kernel is very simple and intuitive: Two samples should belong to the same class if they consistently belong to the same clusters at different scales. The effectiveness of the proposed method is successfully illustrated in multi- and hyperspectral remote sensing image classification and biophysical parameter estimation problems. Accuracy improvements in the range between +5% and 15% over standard principal component analysis (PCA), +4% and 15% over kernel PCA, and +3% and 10% over KPLS are obtained on several images. The average gain in the root-mean-square error of +5% and reductions in bias estimates of +3% are obtained for biophysical parameter retrieval compared to standard PCA feature extraction.

In recent years, kernel methods, in particular support vector machines (SVMs), have been successfully introduced to remote sensing image classification. Their properties make them appropriate for dealing with a high number of image features and a low number of available labeled spectra. The introduction of alternative approaches based on (parametric) Bayesian inference has been quite scarce in the more recent years. Assuming a particular prior data distribution may lead to poor results in remote sensing problems because of the specificities and complexity of the data. In this context, the emerging field of nonparametric Bayesian methods constitutes a proper theoretical framework to tackle the remote sensing image classification problem. This paper exploits the Bayesian modeling and inference paradigm to tackle the problem of kernel-based remote sensing image classification. This Bayesian methodology is appropriate for both finite- and infinite-dimensional feature spaces. The particular problem of active learning is addressed by proposing an incremental/active learning approach based on three different approaches: 1) the maximum differential of entropies; 2) the minimum distance to decision boundary; and 3) the minimum normalized distance. Parameters are estimated by using the evidence Bayesian approach, the kernel trick, and the marginal distribution of the observations instead of the posterior distribution of the adaptive parameters. This approach allows us to deal with infinite-dimensional feature spaces. The proposed approach is tested on the challenging problem of urban monitoring from multispectral and synthetic aperture radar data and in multiclass land cover classification of hyperspectral images, in both purely supervised and active learning settings. Similar results are obtained when compared to SVMs in the supervised mode, with the advantage of providing posterior estimates for classification and automatic parameter learning. Comparison with random sampling as well as standard active learning methods such as margin sampling and entropy-query-by-bagging reveals a systematic overall accuracy gain and faster convergence with the number of queries.

We introduce a method for manifold alignment of different modalities (or domains) of remote sensing images. The problem is recurrent when a set of multitemporal, multisource, multisensor, and multiangular images is available. In these situations, images should ideally be spatially coregistered, corrected, and compensated for differences in the image domains. Such procedures require massive interaction of the user, involve tuning of many parameters and heuristics, and are usually applied separately. Changes of sensors and acquisition conditions translate into shifts, twists, warps, and foldings of the (typically nonlinear) manifolds where images lie. The proposed semisupervised manifold alignment (SS-MA) method aligns the images working directly on their manifolds and is thus not restricted to images of the same resolutions, either spectral or spatial. SS-MA pulls close together samples of the same class while pushing those of different classes apart. At the same time, it preserves the geometry of each manifold along the transformation. The method builds a linear invertible transformation to a latent space where all images are alike and reduces to solving a generalized eigenproblem of moderate size. We study the performance of SS-MA in toy examples and in real multiangular, multitemporal, and multisource image classification problems. The method performs well for strong deformations and leads to accurate classification for all domains. A MATLAB implementation of the proposed method is provided at http://isp. uv.es/code/ssma.htm.

This brief presents a methodology to develop recursive filters in reproducing kernel Hilbert spaces. Unlike previous approaches that exploit the kernel trick on filtered and then mapped samples, we explicitly define the model recursivity in the Hilbert space. For that, we exploit some properties of functional analysis and recursive computation of dot products without the need of preimaging or a training dataset. We illustrate the feasibility of the methodology in the particular case of the γ-filter, which is an infinite impulse response filter with controlled stability and memory depth. Different algorithmic formulations emerge from the signal model. Experiments in chaotic and electroencephalographic time series prediction, complex nonlinear system identification, and adaptive antenna array processing demonstrate the potential of the approach for scenarios where recursivity and nonlinearity have to be readily combined.

This paper presents a multitemporal feature extraction method based on kernels that is particularly designed for change detection. The method provides features that maximize specific changes between two dates while minimizing sources of errors, such as residual land-cover changes and misregistration errors, in the time series. The extracted features computed in the kernel feature space can deal with non-linear relations between samples at different dates. Moreover, no supervised information is required to find the changes of interest for the selected dates in the time series. The effectiveness of the proposed method is successfully illustrated in a cloud masking application using a Landsat time series. Results show that the proposed method provides the most discriminative features in terms of cloud detection when confronted with state of the art linear and nonlinear unsupervised feature extraction algorithms. In particular, extracted features with the proposed method enable automatic cloud detection in multispectral time series.

This paper presents a synergistic cloud detection algorithm that has been developed for processing simultaneous observations from AATSR and MERIS sensors on-board ENVISAT. The main objective of this work is to explore sensor synergies in order to increase the cloud detection accuracy and provide a reliable cloud mask. This is of paramount importance in the framework of the ESA climate change initiative for clouds (Cloud CCI), which aims to provide long time series of cloud properties at a global scale from satellite data. The cloud detection algorithm is based on an ensemble of artificial neural networks, where the outputs of different dedicated models are combined to provide more accurate and robust predictions. The performance of the method has been tested on a large number of real images, and provides higher classification accuracy than other methods, especially when spatial information from the images is included in the classifiers.

In this paper, we propose a method to match multitemporal sequences of hyperspectral images using Hidden Markov Random Fields. Based on the matching of the data manifold, the algorithm matches the reflectance spectra of the classes, thus allowing the reuse of labeled examples acquired on one image to classify the other. This allows valorization of spectra collected in situ to other acquisitions than the one they were acquired for, without user supervision, prior knowledge of the class reflectance in the new domain or global information about atmospheric conditions.

Accurate estimation of biophysical variables is the key to monitor our Planet. In particular, leaf chlorophyll content helps in interpreting the chlorophyll fluorescence signal from space, which is an accurate indicator of the actual state of the vegetation beyond greenness. Recently, the family of Bayesian nonparametric methods has provided excellent results in these situations. A particularly useful method in this framework is the Gaussian Processes regression (GP). However, standard GP assumes that the variance of the noise process is independent of the signal, which does not hold in most of the problems. In this paper, we propose a non-standard variational approximation that allows accurate inference in signal-dependent noise scenarios. We show that the so-called Variational Heteroscedastic Gaussian Process (VHGP) regression is an excellent alternative to standard GP for the retrieval of vegetation chlorophyll content from hyperspectral images. In general VHGP outperforms GP (and many other empirical and machine learning techniques) in accuracy and bias, and reveals more robust when a low number of examples is available.

In this paper, we introduce a non-linear and multidimensional generalization of the Structural SIMilarity index (SSIM) for quality assessment of hyperspectral images. We exploit well-known properties of functional analysis and estimate means, variances, and correlation in proper reproducing kernel Hilbert spaces (rkHs). The so-called Kernel SSIM (KSSIM) is shown to generalize the conventional SSIM and the recently introduced Q4and Qnmetrics for remote sensing applications, and naturally works with multidimensional images. For the experimentation, we built a database of different distortions commonly encountered in remote sensing images. KSSIM shows an improved agreement with classification results compared to standard similarity metrics, and high consistency for different noise sources and levels.

The analysis of multi-modal and multi-sensor images is nowadays of paramount importance for Earth Observation (EO) applications. There exist a variety of methods that aim at fusing the different sources of information to obtain a compact representation of such datasets. However, for change detection existing methods are often unable to deal with heterogeneous image sources and very few consider possible nonlinearities in the data. Additionally, the availability of labeled information is very limited in change detection applications. For these reasons, we present the use of a semi-supervised kernel-based feature extraction technique. It incorporates a manifold regularization accounting for the geometric distribution and jointly addressing the small sample problem. An exhaustive example using Landsat 5 data illustrates the potential of the method for multi-sensor change detection.

This letter introduces a simple method for including invariances in support-vector-machine (SVM) remote sensing image classification. We design explicit invariant SVMs to deal with the particular characteristics of remote sensing images. The problem of including data invariances can be viewed as a problem of encoding prior knowledge, which translates into incorporating informative support vectors (SVs) that better describe the classification problem. The proposed method essentially generates new (synthetic) SVs from the obtained by training a standard SVM with the available labeled samples. Then, original and transformed SVs are used for training the virtual SVM introduced in this letter. We first incorporate invariances to rotations and reflections of image patches for improving contextual classification. Then, we include an invariance to object scale in patch-based classification. Finally, we focus on the challenging problem of including illumination invariances to deal with shadows in the images. Very good results are obtained when few labeled samples are available for classification. The obtained classifiers reveal enhanced sparsity and robustness. Interestingly, the methodology can be applied to any maximum-margin method, thus constituting a new research opportunity.

Many remote sensing data processing problems are inherently constituted by several tasks that can be solved either individually or jointly. For instance, each image in a multitemporal classification setting could be taken as an individual task. Here, the relation to previous acquisitions should be properly considered because of the nonstationary behavior of temporal, spatial, and angular image features which gives rise to distribution changes. This phenomenon is known as covariate shift. Additionally, when labeled data are scarce or expensive to obtain, the small sample-set problem arises, which makes solving the problems independently in each domain difficult. Multitask learning (MTL) aims at jointly solving a set of prediction problems by sharing information across tasks. This paper introduces MTL in remote sensing data classification. The proposed methods alleviate the data set shift by imposing cross-information in the classifiers through matrix regularization. We consider the support vector machine (SVM) as the core learner and two different regularization schemes: 1) the inclusion of relational operators between tasks and 2) the pairwise Euclidean distance of the predictors in the Hilbert space. These methods rely on simple and intuitive modifications of the kernel used in the standard SVM. Experiments are conducted in three challenging remote sensing problems: cloud screening from multispectral images, land-mine detection using radar data, and multitemporal and multisource image classification. The pairwise method consistently outperforms standard independent and aggregate approaches by about +2% to 4% in all problems at no additional cost. Also, the solutions found give us information about the distribution shift among tasks.

When dealing with supervised target detection, the acquisition of labeled samples is one of the most critical phases: the samples must be yet representative of the class of interest, but must also be found among a vast majority of non-target examples. Moreover, the efficiency of the search is also an issue, since the samples labeled as background are not used by target detectors such as the support vector data description (SVDD). In this work we propose a competitive and effective approach to identify the most relevant training samples for one-class classification based on the use of an active learning strategy. The SVDD classifier is first trained with insufficient target examples. It is then used to detect the most informative samples to be labeled by a user through active learning techniques. By selecting unlabeled samples in a smart way and by adopting a diversity criterion, it is possible to obtain an accurate description of the class of interest with a relatively small number of training samples. The performance of the proposed method is illustrated in a change detection scenario and is validated by comparison with state-of-art active learning techniques originally developed for multiclass problems.

Feature extraction is of paramount importance for an accurate classification of remote sensing images. Techniques based on data transformations are widely used in this context. However, linear feature extraction algorithms, such as the principal component analysis and partial least squares, can address this problem in a suboptimal way because the data relations are often nonlinear. Kernel methods may alleviate this problem only when the structure of the data manifold is properly captured. However, this is difficult to achieve when small-size training sets are available. In these cases, exploiting the information contained in unlabeled samples together with the available training data can significantly improve data description by defining an effective semisupervised nonlinear feature extraction strategy. We present a novel semisupervised Kernel Partial Least Squares (KPLS) algorithm for non-linear feature extraction. The method relies on combining two kernel functions: the standard RBF kernel using labeled information and a generative kernel directly learned by clustering the data. The effectiveness of the proposed method is successfully illustrated in multi- and hyper-spectral remote sensing image classification: accuracy improvements between +15 - 20% over standard PCA and +10% over advanced kernel PCA and KPLS for both images is obtained. Matlab code is available at http://isp.uv.es for the interested readers.

This paper introduces a simple method to include invariances in support vector machine (SVM) for remote sensing image classification. We rely on the concept of virtual support vectors, by which the SVM is trained with both the selected support vectors and synthetic examples encoding the invariance of interest. The algorithm is very simple and effective, as demonstrated in two particularly interesting examples: invariance to the presence of shadows and to rotations in patchbased image segmentation. The improved accuracy (around +6% both in OA and Cohen's κ statistic), along with the simplicity of the approach encourage its use and extension to encode other invariances and other remote sensing data analysis applications.

In this letter, an unsupervised kernel-based approach to change detection is introduced. Nonlinear clustering is utilized to partition in two a selected subset of pixels representing both changed and unchanged areas. Once the optimal clustering is obtained, the learned representatives of each group are exploited to assign all the pixels composing the multitemporal scenes to the two classes of interest. Two approaches based on different assumptions of the difference image are proposed. The first accounts for the difference image in the original space, while the second defines a mapping describing the difference image directly in feature spaces. To optimize the parameters of the kernels, a novel unsupervised cost function is proposed. An evidence of the correctness, stability, and superiority of the proposed solution is provided through the analysis of two challenging change-detection problems.

Mechanisms of human color vision are characterized by two phenomenological aspects: the system is nonlinear and adaptive to changing environments. Conventional attempts to derive these features from statistics use separate arguments for each aspect. The few statistical explanations that do consider both phenomena simultaneously follow parametric formulations based on empirical models. Therefore, it may be argued that the behavior does not come directly from the color statistics but from the convenient functional form adopted. In addition, many times the whole statistical analysis is based on simplified databases that disregard relevant physical effects in the input signal, as, for instance, by assuming flat Lambertian surfaces. In this work, we address the simultaneous statistical explanation of the nonlinear behavior of achromatic and chromatic mechanisms in a fixed adaptation state and the change of such behavior (i.e., adaptation) under the change of observation conditions. Both phenomena emerge directly from the samples through a single data-driven method: the sequential principal curves analysis (SPCA) with local metric. SPCA is a new manifold learning technique to derive a set of sensors adapted to the manifold using different optimality criteria. Here sequential refers to the fact that sensors (curvilinear dimensions) are designed one after the other, and not to the particular (eventually iterative) method to draw a single principal curve. Moreover, in order to reproduce the empirical adaptation reported under D65 and A illuminations, a new database of colorimetrically calibrated images of natural objects under these illuminants was gathered, thus overcoming the limitations of available databases. The results obtained by applying SPCA show that the psychophysical behavior on color discrimination thresholds, discount of the illuminant, and corresponding pairs in asymmetric color matching emerge directly from realistic data regularities, assuming no a priori functional form. These results provide stronger evidence for the hypothesis of a statistically driven organization of color sensors. Moreover, the obtained results suggest that the nonuniform resolution of color sensors at this low abstraction level may be guided by an error-minimization strategy rather than by an information-maximization goal.

This paper evaluates nonlinear retrieval methods to derive atmospheric properties from hyperspectral infrared sounding spectra, with emphasis on the retrieval of temperature, humidity, and ozone atmospheric profiles. We concentrate on the Infrared Atmospheric Sounding Interferometer (IASI) onboard the MetOp-A satellite data for the future Meteosat Third Generation Infrared Sounder (MTG-IRS). The methods proposed in this work are compared in terms of both accuracy and speed with the current MTG-IRS L2 processing concept, which processes MetOp-IASI and proxy MTG-IRS data. The official chain consists of a principal component extraction, typically referred to as empirical orthogonal functions (EOF) and a subsequent canonical linear regression. This research proposes the evaluation of some other methodological advances considering: 1) other linear feature extraction methods instead of EOF, such as partial least squares; and 2) the linear combination of nonlinear regression models in the form of committee of experts. The nonlinear regression models considered in this work are artificial neural networks and kernel ridge regression as nonparametric multioutput powerful regression tools. Results show that, in general, nonlinear models yield better results than linear retrieval for both MetOp-IASI and MTG-IRS synthetic and real data. Averaged gains throughout the column of +1.8 K and +2.2 K are obtained for temperature profile estimation from MetOp-IASI and IRS data, respectively. Similar gains are obtained for the estimation of dew point temperatures. In both variables, these improvements are more noticeable in lower atmospheric layers. The combination of models makes the retrieval more robust, improves the accuracy, and decreases the estimated bias. The nonlinear statistical approach is successfully compared to optimal estimation (OE) in terms of accuracy, bias and computational cost. These results confirm the potential of statistical nonlinear inversion techniques for the retrieval of atmospheric profiles.

We propose a semiautomatic procedure to generate land cover maps from remote sensing images. The proposed algorithm starts by building a hierarchical clustering tree, and exploits the most coherent pixels with respect to the available class information. For a given amount of labeled pixels, the algorithm returns both classification and confidence maps. Since the quality of the map depends of the number and informativeness of the labeled pixels, active learning methods are used to select the most informative samples to increase confidence in class membership. Experiments on four different data sets, accounting for hyperspectral and multispectral images at different spatial resolutions, confirm the effectiveness of the proposed approach, and how active learning techniques reduce the uncertainty of the classification maps. Specifically, more accurate results with fewer labeled samples are obtained. Inclusion of spatial information in the classifiers drastically improves the classification accuracy, leading to faster convergence curves and tighter confidence intervals. In conclusion, the presented algorithm provides efficient image classification and, at the same time, yields a confidence map that may be very useful in many Earth observation applications.

This paper evaluates state-of-the-art parametric and nonparametric approaches for the estimation of leaf chlorophyll content$(Chl)$, leaf area index, and fractional vegetation cover from space. The parametric approach involves comparison of established and generic narrowband vegetation indices (VIs) and the Normalized Area Over reflectance Curve method, which calculates the continuum spectral region sensitive to$Chl$. However, as not all available bands take part in these spectral algorithms, it remains unclear whether optimal estimations are achieved. Alternatively, the nonparametric approach is based on Gaussian process (GP) techniques and allows inclusion of all bands. GP builds a nonlinear regression as a linear combination of spectra mapped to a high-dimensional space. Moreover, GP provides an indication of the most contributing bands for each parameter, a weight for the most relevant spectra contained in the training data set, and a confidence estimate of the retrieval. GP has previously demonstrated to be competitive in accuracy with support vector regression and neural networks. Results from hyperspectral Compact High Resolution Imaging Spectrometer data over the Spanish Barrax test site show that GP outperformed the VIs in assessing the vegetation properties when using at least four out of the 62 bands. GP identified most contributing bands in the red and red edge and, to a lower extent, in the blue and NIR parts of the spectrum. Since the proposed GP method is able to build robust relationships between the parameter of interest and only a few bands, it is a promising approach for multispectral data as well.

This paper presents a methodology to develop recursive filters in reproducing kernel Hilbert spaces (RKHS). Unlike previous approaches that exploit the kernel trick on filtered and then mapped samples, we explicitly define model recursivity in the Hilbert space. The method exploits some properties of functional analysis and recursive computation of dot products without the need of pre-imaging. We illustrate the feasibility of the methodology in the particular case of the gamma filter, an infinite impulse response (IIR) filter with controlled stability and memory depth. Different algorithmic formulations emerge from the signal model. Experiments in chaotic and electroencephalographic time series prediction scenarios demonstrate the potentiality of the approach.

This paper proposes the use of kernel ridge regression (KRR) to derive surface and atmospheric properties from hyperspectral infrared sounding spectra. We focus on the retrieval of temperature and humidity atmospheric profiles from Infrared Atmospheric Sounding Interferometer (MetOp-IASI) data, and provide confidence maps on the predictions. In addition, we propose a scheme for the identification of anomalies by supervised classification of discrepancies with the ECMWF estimates. For the retrieval, we observed that KRR clearly outperformed linear regression. Looking at the confidence maps, we observed that big discrepancies are mainly due to the presence of clouds and low emissivities in desert areas. For the identification of anomalies, we observed that the confidence intervals provided by the KRR may help in discarding big errors. High detection accuracy (around 90%) is achieved by a support vector machine, which largely outperforms standard linear and nonlinear classifiers.

This paper proposes the kernel entropy component analysis (KECA) for clustering remote sensing data. The method generates nonlinear features that reveal structure related to the Renyi entropy of the input space data set. Unlike other kernel feature extraction methods, the top eigenvalues and eigenvectors of the kernel matrix are not necessarily chosen. Data are interestingly mapped with a distinct angular structure, which is exploited to derive a new angle-based spectral clustering algorithm based on the mapped data. An out-of-sample extension of the method is also presented to deal with test data. We focus on cloud screening from MERIS images. Several images are considered to account for the high variability of the problem. Good results show the suitability of the proposal.

This paper introduces a nonlinear feature extraction method based on kernels for remote sensing data analysis. The proposed approach is based on the minimum noise fraction (MNF) transform, which maximizes the signal variance while also minimizing the estimated noise variance. We here propose an alternative kernel MNF (KMNF) in which the noise is explicitly estimated in the reproducing kernel Hilbert space. This enables KMNF dealing with non-linear relations between the noise and the signal features jointly. Results show that the proposed KMNF provides the most noise-free features when confronted with PCA, MNF, KPCA, and the previous version of KMNF. Extracted features with the explicit KMNF also improve hyperspectral image classification.

Inspired by the concept of Principal Curves, in this paper, we define Principal Polynomials as a non-linear generalization of Principal Components to overcome the conditional mean independence restriction of PCA. Principal Polynomials deform the straight Principal Components by minimizing the regression error (or variance) in the corresponding orthogonal subspaces. We propose to use a projection on a series of these polynomials to set a new nonlinear data representation: the Principal Polynomial Analysis (PPA). We prove that the dimensionality reduction error in PPA is always lower than in PCA. Lower truncation error and increased independence suggest that unsupervised PPA features can be better suited to image classification than those identified by other unsupervised techniques. We analyze the performance of Linear Discriminant Analysis in the feature space after dimensionality reduction using the proposed PPA, the classical PCA, and locally linear embedding (LLE). Experiments on very high resolution data confirm the suitability of PPA to describe nonlinear manifolds found in remote sensing data.

This paper presents a family of metrics for assessing image similarity. The methods use the Hilbert-Schmidt Independence Criterion (HSIC) to estimate nonlinear statistical dependence between multidimensional images. The proposed methods have very good theoretical and practical properties. We illustrate the performance in evaluating the quality of natural photographic images, hyperspectral images under different noise levels, in synthetic multiresolution problems, and real pansharpening products.

A semiautomatic procedure to generate classification maps of remote sensing images is proposed. Starting from a hierarchical unsupervised classification, the algorithm exploits the few available labeled pixels to assign each cluster to the most probable class. For a given amount of labeled pixels, the algorithm returns a classified segmentation map, along with confidence levels of class membership for each pixel. Active learning methods are used to select the most informative samples to increase confidence in the class membership. Experiments on a AVIRIS hyperspectral image confirm the effectiveness of the method, especially when used with active learning query functions and spatial regularization.

In this paper we propose an unsupervised approach to change detection by computing the difference image directly in the feature spaces. The resulting difference kernel, that is a combination of kernels computed on the coregistered and radiometrically matched input images, is used to train a nonlinear partitioning algorithm. In order to apply the kernel k-means, issues related to the initialization and to the tuning of parameters (e.g. the Gaussian RBF bandwidth) are considered. To validate the proposed unsupervised algorithm, two multitemporal VHR remote sensing images are used.

Earth observation satellites currently provide a large volume of images at different scales. Most of these satellites provide global coverage with a revisit time that usually depends on the instrument characteristics and performance. Typically, medium-spatial-resolution instruments provide better spectral and temporal resolutions than mapping-oriented high-spatial-resolution multispectral sensors. However, in order to monitor a given area of interest, users demand images with the best resolution available, which cannot be reached using a single sensor. In this context, image fusion may be effective to merge information from different data sources. In this letter, an image fusion approach based on multiresolution and multisource spatial unmixing is used to obtain a composite image with the spectral and temporal characteristics of medium-spatial-resolution instrument along with the spatial resolution of high-spatial-resolution image. A time series of Landsat/TM and ENVISAT/MERIS Full Resolution images acquired in the 2004 European Space Agency (ESA) Spectra Barrax Campaign illustrates the method's capabilities. The qualitative and quantitative assessments of the product images are given. The proposed methodology is general enough to be applied to similar sensors, such as the multispectral instruments which will fly on board the ESA GMES Sentinel-2 and Sentinel-3 upcoming satellite series.

Hyperspectral data lossy compression has not yet achieved global acceptance in the remote sensing community, mainly because it is generally perceived that using compressed images may affect the results of posterior processing stages. This possible negative effect, however, has not been accurately characterized so far. In this letter, we quantify the impact of lossy compression on two standard approaches for hyperspectral data exploitation: spectral unmixing, and supervised classification using support vector machines. Our experimental assessment reveals that different stages of the linear spectral unmixing chain exhibit different sensitivities to lossy data compression. We have also observed that, for certain compression techniques, a higher compression ratio may lead to more accurate classification results. Even though these results may seem counterintuitive, this work explains these observations in light of the spatial regularization and/or whitening that most compression techniques perform and further provides recommendations on best practices when applying lossy compression prior to hyperspectral data classification and/or unmixing.

This letter proposes a multioutput support vector regression (M-SVR) method for the simultaneous estimation of different biophysical parameters from remote sensing images. General retrieval problems require multioutput (and potentially nonlinear) regression methods. M-SVR extends the single-output SVR to multiple outputs maintaining the advantages of a sparse and compact solution by using an$\varepsilon$-insensitive cost function. The proposed M-SVR is evaluated in the estimation of chlorophyll content, leaf area index and fractional vegetation cover from a hyperspectral compact high-resolution imaging spectrometer images. The achieved improvement with respect to the single-output regression approach suggests that M-SVR can be considered a convenient alternative for nonparametric biophysical parameter estimation and model inversion.

Earth observation satellites provide a valuable source of data which when conveniently processed can be used to better understand the Earth system dynamics. In this regard, one of the prerequisites for the analysis of satellite image time series is that the images are spatially coregistered so that the resulting multitemporal pixel entities offer a true temporal view of the area under study. This implies that all the observations must be mapped to a common system of grid cells. This process is known as gridding and, in practice, two common grids can be used as a reference: 1) a grid defined by some kind of external data set (e.g., an existing land-cover map) or 2) a grid defined by one of the images of the time series. The aim of this paper is to study the impact that gridding has on the quality of satellite time series. More precisely, the impact of the so-called gridding artifacts is quantified using a time series of 12 images acquired over The Netherlands by the Medium Resolution Imaging Spectrometer (MERIS). First, the impact of selecting a reference grid is evaluated in terms of geolocation errors and pixel overlap. Then, the effect of observation geometry is studied as nongeostationary satellites, like MERIS, can acquire images from the same area from a number of orbits. Finally, a high-resolution land-cover data set is used to account for temporal information consistency (pixel homogeneity in terms of land-cover composition). Results have shown an average pixel overlap with the nearest pixel between 20% and 41% depending on the selected reference grid and on the differences in observation geometry. These results indicate that inappropriate gridding might result in collocated time series that are not adequate for temporal studies at pixel level (particularly over nonhomogeneous areas) and that, in any case, it is interesting to identify areas with low pixel overlap in order to further analyze the reliability of the products derived over these areas.

Data from current medium-spatial-resolution imaging spectroradiometers are used for land-cover mapping and land-cover change detection at regional to global scales. However, few landscapes are homogeneous at these scales, and this creates the so-called mixed-pixel problem. In this context, this study explores the use of the linear spectral mixture model to extract subpixel land-cover composition from medium-spatial-resolution data. In particular, a time series of MEdium Resolution Imaging Spectrometer (MERIS) full-resolution (FR; pixel size of 300 m) images acquired over The Netherlands is used to illustrate this study. The Netherlands was selected because of the following: 1) the fragmentation of its landscapes and 2) the availability of a high-spatial-resolution land-cover data set (LGN5) which can be used as a reference. The question then is to what extent a multitemporal unmixing of MERIS FR data delivers land-cover information comparable with the one provided by the LGN5. To this end, fully constrained linear spectral unmixing is applied to each individual MERIS image and to the multitemporal composite. The unmixing results are validated at both subpixel and per-pixel scales and at two thematic aggregation levels (12 and 4 land-cover classes). The obtained results indicate that the described unmixing approach yields moderate results for the 12-class case and good results for the 4-class case. These results might be explained by MERIS preprocessing steps, gridding effects, vegetation phenophases, and spectral class separability.

Most signal processing problems involve the challenging task of multidimensional probability density function (PDF) estimation. In this paper, we propose a solution to this problem by using a family of rotation-based iterative Gaussianization (RBIG) transforms. The general framework consists of the sequential application of a univariate marginal Gaussianization transform followed by an orthonormal transform. The proposed procedure looks for differentiable transforms to a known PDF so that the unknown PDF can be estimated at any point of the original domain. In particular, we aim at a zero-mean unit-covariance Gaussian for convenience. RBIG is formally similar to classical iterative projection pursuit algorithms. However, we show that, unlike in PP methods, the particular class of rotations used has no special qualitative relevance in this context, since looking for interestingness is not a critical issue for PDF estimation. The key difference is that our approach focuses on the univariate part (marginal Gaussianization) of the problem rather than on the multivariate part (rotation). This difference implies that one may select the most convenient rotation suited to each practical application. The differentiability, invertibility, and convergence of RBIG are theoretically and experimentally analyzed. Relation to other methods, such as radial Gaussianization, one-class support vector domain description, and deep neural networks is also pointed out. The practical performance of RBIG is successfully illustrated in a number of multidimensional problems such as image synthesis, classification, denoising, and multi-information estimation.

Nowadays, the increasing quantity of applications using images from Earth Observation satellites makes demanding better spatial, spectral and temporal resolutions. Nevertheless, due to the technical constraint of a trade off between spatial and spectral resolutions, and between spatial resolution and coverage, high spatial resolution is related with low spectral and temporal resolutions and vice versa. Data fusion methods are a good solution to combine information from multiple sensors in order to obtain image products with better characteristics. In this paper, we propose an image fusion approach based on a multi-resolution and multi-source unmixing. The proposed methodology yields a composite image with the spatial resolution of the higher resolution image (downscaling) while retaining the spectral and temporal characteristics of the medium spatial resolution image. The approach is tested in the specific cases of ENVISAT/MERIS and Landsat/TM instruments, but is general enough to be applied to other sensor combination.

This paper introduces a nonlinear measure of dependence between random variables in the context of remote sensing data analysis. The Hilbert-Schmidt Independence Criterion (HSIC) is a kernel method for evaluating statistical dependence. HSIC is based on computing the Hilbert-Schmidt norm of the cross-covariance operator of mapped samples in the corresponding Hilbert spaces. The HSIC empirical estimator is very easy to compute and has good theoretical and practical properties. We exploit the capabilities of HSIC to explain nonlinear dependences in two remote sensing problems: temperature estimation and chlorophyll concentration prediction from spectra. Results show that, when the relationship between random variables is nonlinear or when few data are available, the HSIC criterion outperforms other standard methods, such as the linear correlation or mutual information.

In this paper, we consider active sampling to label pixels grouped with hierarchical clustering. The objective of the method is to match the data relationships discovered by the clustering algorithm with the user's desired class semantics. The first is represented as a complete tree to be pruned and the second is iteratively provided by the user. The active learning algorithm proposed searches the pruning of the tree that best matches the labels of the sampled points. By choosing the part of the tree to sample from according to current pruning's uncertainty, sampling is focused on most uncertain clusters. This way, large clusters for which the class membership is already fixed are no longer queried and sampling is focused on division of clusters showing mixed labels. The model is tested on a VHR image in a multiclass classification setting. The method clearly outperforms random sampling in a transductive setting, but cannot generalize to unseen data, since it aims at optimizing the classification of a given cluster structure.

This letter introduces a nonlinear measure of independence between random variables for remote sensing supervised feature selection. The so-called Hilbert–Schmidt independence criterion (HSIC) is a kernel method for evaluating statistical dependence and it is based on computing the Hilbert–Schmidt norm of the cross-covariance operator of mapped samples in the corresponding Hilbert spaces. The HSIC empirical estimator is easy to compute and has good theoretical and practical properties. Rather than using this estimate for maximizing the dependence between the selected features and the class labels, we propose the more sensitive criterion of minimizing the associated HSIC$p$-value. Results in multispectral, hyperspectral, and SAR data feature selection for classification show the good performance of the proposed approach.

This letter presents a graph kernel for spatio-spectral remote sensing image classification with support vector machines (SVMs). The method considers higher order relations in the neighborhood (beyond pairwise spatial relations) to iteratively compute a kernel matrix for SVM learning. The proposed kernel is easy to compute and constitutes a powerful alternative to existing approaches. The capabilities of the method are illustrated in several multi- and hyperspectral remote sensing images acquired over both urban and agricultural areas.

This letter presents advanced classification methods for very high resolution images. Efficient multisource information, both spectral and spatial, is exploited through the use of composite kernels in support vector machines. Weighted summations of kernels accounting for separate sources of spectral and spatial information are analyzed and compared to classical approaches such as pure spectral classification or stacked approaches using all the features in a single vector. Model selection problems are addressed, as well as the importance of the different kernels in the weighted summation.

Remote sensing image classification constitutes a challenging problem since very few labeled pixels are typically available from the analyzed scene. In such situations, labeled data extracted from other images modeling similar problems might be used to improve the classification accuracy. However, when training and test samples follow even slightly different distributions, classification is very difficult. This problem is known assample selection bias. In this paper, we propose a new method to combine labeled and unlabeled pixels to increase classification reliability and accuracy. A semisupervised support vector machine classifier based on the combination of clustering and themean mapkernel is proposed. The method reinforces samples in the same cluster belonging to the same class by combining sample and cluster similarities implicitly in the kernel space. Asoftversion of the method is also proposed where only the most reliable training samples, in terms of likelihood of the image data distribution, are used. Capabilities of the proposed method are illustrated in a cloud screening application using data from the MEdium Resolution Imaging Spectrometer (MERIS) instrument onboard the European Space Agency ENVISAT satellite. Cloud screening constitutes a clear example of sample selection bias since cloud features change to a great extent depending on the cloud type, thickness, transparency, height, and background. Good results are obtained and show that the method is particularly well suited for situations where the available labeled information does not adequately describe the classes in the test data.

This paper presents two semisupervised one-class support vector machine (OC-SVM) classifiers for remote sensing applications. Inone-classimage classification, one tries to detect pixels belonging to one of the classes in the image and reject the others. When few labeled pixels of only one class are available, obtaining a reliable classifier is a difficult task. In the particular case of SVM-based classifiers, this task is even harder because the free parameters of the model need to be finely adjusted, but no clear criterion can be adopted. In order to improve the OC-SVM classifier accuracy and alleviate the problem of free-parameter selection, the information provided by unlabeled samples present in the scene can be used. In this paper, we present two state-of-the-art algorithms for semisupervised one-class classification for remote sensing classification problems. The first proposed algorithm is based on modifying the OC-SVM kernel by modeling the data marginal distribution with the graph Laplacian built with both labeled and unlabeled samples. The second one is based on a simple modification of the standard SVM cost function which penalizes more the errors made when classifying samples of the target class. The good performance of the proposed methods is illustrated in four challenging remote sensing image classification scenarios where the goal is to detect one of the classes present on the scene. In particular, we present results for multisource urban monitoring, hyperspectral crop detection, multispectral cloud screening, and change-detection problems. Experimental results show the suitability of the proposed techniques, particularly in cases with few or poorly representative labeled samples.

A framework for semisupervised remote sensing image classification based on neural networks is presented. The methodology consists of adding a flexible embedding regularizer to the loss function used for training neural networks. Training is done using stochastic gradient descent with additional balancing constraints to avoid falling into local minima. The method constitutes a generalization of both supervised and unsupervised methods and can handle millions of unlabeled samples. Therefore, the proposed approach gives rise to an operational classifier, as opposed to previously presented transductive or Laplacian support vector machines (TSVM or LapSVM, respectively). The proposed methodology constitutes a general framework for building computationally efficient semisupervised methods. The method is compared with LapSVM and TSVM in semisupervised scenarios, to SVM in supervised settings, and to online and batchk-means for unsupervised learning. Results demonstrate the improved classification accuracy and scalability of this approach on several hyperspectral image classification problems.

The increase in spatial and spectral resolution of the satellite sensors, along with the shortening of the time-revisiting periods, has provided high-quality data for remote sensing image classification. However, the high-dimensional feature space induced by using many heterogeneous information sources precludes the use of simple classifiers: thus, a proper feature selection is required for discarding irrelevant features and adapting the model to the specific problem. This paper proposes to classify the images and simultaneously to learn the relevant features in such high-dimensional scenarios. The proposed method is based on the automatic optimization of a linear combination of kernels dedicated to different meaningful sets of features. Such sets can be groups of bands, contextual or textural features, or bands acquired by different sensors. The combination of kernels is optimized through gradient descent on the support vector machine objective function. Even though the combination is linear, the ranked relevance takes into account the intrinsic nonlinearity of the data through kernels. Since a naive selection of the free parameters of the multiple-kernel method is computationally demanding, we propose an efficient model selection procedure based on the kernel alignment. The result is a weight (learned from the data) for each kernel where both relevant and meaningless image features automatically emerge after training the model. Experiments carried out in multi- and hyperspectral, contextual, and multisource remote sensing data classification confirm the capability of the method in ranking the relevant features and show the computational efficience of the proposed strategy.

The estimation of high-dimensional probability density functions (PDFs) is not an easy task for many image processing applications. The linear models assumed by widely used transforms are often quite restrictive to describe the PDF of natural images. In fact, additional non-linear processing is needed to overcome the limitations of the model. On the contrary, the class of techniques collectively known as projection pursuit, which solve the high-dimensional problem by sequential univariate solutions, may be applied to very general PDFs (e.g. iterative Gaussianization procedures). However, the associated computational cost has prevented their extensive use in image processing. In this work, we propose a fast alternative to iterative Gaussianization methods that makes it suitable for image processing while ensuring its theoretical convergence. Method performance is successfully illustrated in image synthesis and classification problems.

Remote sensing image processing is nowadays a mature research area. The techniques developed in the field allow many real-life applications with great societal value. For instance, urban monitoring, fire detection or flood prediction can have a great impact on economical and environmental issues. To attain such objectives, the remote sensing community has turned into a multidisciplinary field of science that embraces physics, signal theory, computer science, electronics, and communications. From a machine learning and signal/image processing point of view, all the applications are tackled under specific formalisms, such as classification and clustering, regression and function approximation, image coding, restoration and enhancement, source unmixing, data fusion or feature selection and extraction. This paper serves as a survey of methods and applications, and reviews the last methodological advances in remote sensing image processing.

We evaluate Gaussian Processes (GPs) for the estimation of biophysical parameters from acquired multispectral data. The standard GP formulation is used, and all hyperparameters (kernel parameters and noise variance) are optimized by maximizing the marginal likelihood. This gives rise to a fully-adaptive GP to data characteristics, both in terms of signal and noise properties. The good numerical results in the estimation of oceanic chlorophyll concentration and leaf membrane state confirm GPs as adequate, alternative non-parametric methods for biophysical parameter estimation. GPs are also analyzed by scrutinizing the predictive variance, the estimated noise variance, and the relevance of each feature after optimization.

This paper presents a cloud screening algorithm based on ensemble methods that exploits the combined information from both MERIS and AATSR instruments on board ENVISAT in order to improve current cloud masking products for both sensors. The first step is to analyze the synergistic use of MERIS and AATSR images in order to extract some physically-based features increasing the separability of clouds and surface. Then, several artificial neural networks are trained using different sets of input features and different sets of training samples depending on acquisition and surface conditions. Finally, outputs of the trained neural networks are combined at the decision level to construct a more accurate and robust ensemble of classifiers. The proposed classifier is tested on more than 80 coregistered MERIS/AATSR images providing better classification accuracy than the official cloud flags and available operational cloud screening algorithms for MERIS and AATSR. Moreover, thanks to the synergy of both sensors, it correctly classifies critical cloud-screening problems such as snow and ice covers over land and sun-glint over ocean.

This paper proposes to learn the relevant features of remote sensing images for automatic spatio-spectral classification with the automatic optimization of multiple kernels. The method consists of building dedicated kernels for different sets of bands, contextual or textural features. The optimal linear combination of kernels is optimized through gradient descent on the support vector machine (SVM) objective function. Since a naive implementation is computationally demanding, we propose an efficient model selection procedure based on kernel alignment. The result is a weight - learned from the data - for each kernel where both relevant and meaningless image features emerge after training. Excellent results are observed in both multi and hyperspectral image classification, improving standard SVM and other spatio-spectral formulations.

A semi-supervised graph-based approach to target detection is presented. The proposed method improves the Kernel Orthogonal Subspace Projection (KOSP) by deforming the kernel through the approximation of the marginal distribution using the unlabeled samples. The good performance of the proposed method is illustrated in a hyperspectral image target detection application for thermal hot spot detection. An improvement is observed with respect to the linear and the non-linear kernel-based OSP, demonstrating good generalization capabilities when low number of labeled samples are available, which is usually the case in target detection problems.

This letter presents two kernel-based methods for semisupervised regression. The methods rely on building a graph or hypergraph Laplacian with both the available labeled and unlabeled data, which is further used to deform the training kernel matrix. The deformed kernel is then used for support vector regression (SVR). Given the high computational burden involved, we present two alternative formulations based on the Nystrom method and the incomplete Cholesky factorization to achieve operational processing times. The semisupervised SVR algorithms are successfully tested in multiplatform leaf area index estimation and oceanic chlorophyll concentration prediction. Experiments are carried out with both multispectral and hyperspectral data, demonstrating good generalization capabilities when a low number of labeled samples are available, which is usually the case in biophysical parameter retrieval.

This letter presents a novel composite semisupervised support vector machine (SVM) for the spectral-spatial classification of hyperspectral images. In particular, the proposed technique exploits the following: 1) unlabeled data for increasing the reliability of the training phase when few training samples are available and 2) composite kernel functions for simultaneously taking into account spectral and spatial information included in the considered image. Experiments carried out on a hyperspectral image pointed out the effectiveness of the presented technique, which resulted in a significant increase of the classification accuracy with respect to both supervised SVMs and progressive semisupervised SVMs with single kernels, as well as supervised SVMs with composite kernels.

A semisupervised support vector machine is presented for the classification of remote sensing images. The method exploits the wealth of unlabeled samples for regularizing the training kernel representation locally by means of cluster kernels. The method learns a suitable kernel directly from the image and thus avoids assuming a priori signal relations by using a predefined kernel structure. Good results are obtained in image classification examples when few labeled samples are available. The method scales almost linearly with the number of unlabeled samples and provides out-of-sample predictions.

This paper analyzes the classification of hyperspectral remote sensing images with linear discriminant analysis (LDA) in the presence of a small ratio between the number of training samples and the number of spectral features. In these particular ill-posed problems, a reliable LDA requires one to introduce regularization for problem solving. Nonetheless, in such a challenging scenario, the resulting regularized LDA (RLDA) is highly sensitive to the tuning of the regularization parameter. In this context, we introduce in the remote sensing community an efficient version of the RLDA recently presented by Ye to cope with critical ill-posed problems. In addition, several LDA-based classifiers (i.e., penalized LDA, orthogonal LDA, and uncorrelated LDA) are compared theoretically and experimentally with the standard LDA and the RLDA. Method differences are highlighted through toy examples and are exhaustively tested on several ill-posed problems related to the classification of hyperspectral remote sensing images. Experimental results confirm the effectiveness of the presented RLDA technique and point out the main properties of other analyzed LDA techniques in critical ill-posed hyperspectral image classification problems.

The orthogonal subspace projection (OSP) algorithm is substantially a kind of matched filter that requires the evaluation of a prototype for each class to be detected. The kernel OSP (KOSP) has recently demonstrated improved results for target detection in hyperspectral images. The use of kernel methods (KMs) makes the method nonlinear, helps to combat the high-dimensionality problem, and improves robustness to noise. This paper presents a semisupervised graph-based approach to improve KOSP. The proposed algorithm deforms the kernel by approximating the marginal distribution using the unlabeled samples. Two further improvements are presented. First, a contextual selection of unlabeled samples is proposed. This strategy helps in better modeling the data manifold, and thus, improved sensitivity-specificity rates are obtained. Second, given the high computational burden involved, we present two alternative formulations based on the Nystroumlm method and the incomplete Cholesky factorization to achieve operational processing times. The good performance of the proposed method is illustrated in a toy data set and two relevant hyperspectral image target-detection applications: crop identification and thermal hot-spot detection. A clear improvement is observed with respect to the linear and the nonlinear kernel-based OSP, demonstrating good generalization capabilities when a low number of labeled samples are available, which is usually the case in target-detection problems. The relevance of unlabeled samples and the computational cost are also analyzed in detail.

Here we propose an alternative non-explicit way to take into account the relations among wavelet coefficients in natural images for denoising: we use support vector machines (SVM) to learn these relations. Since relations among the coefficients are specific to the signal, SVM regularization removes the noise, which does not share this property. Moreover, due to its non-parametric nature, the method can eventually cope with different noise sources. The results show that: (1) the proposed non-parametric method outperforms conventional methods that assume coefficient independence, and (2) its performance is similar to state-of-the-art parametric methods that do explicitly include these relations. Therefore, the proposed machine learning approach can be seen as a more flexible (model-free) alternative to the explicit description of wavelet coefficient relations in Bayesian approaches.

Two kernel-based methods for semi-supervised regression are presented. The methods rely on building a graph or hypergraph Laplacian with both the labeled and unlabeled data, which is further used to deform the training kernel matrix. The deformed kernel is then used for support vector regression (SVR). The semi-supervised SVR methods are sucessfully tested in LAI estimation and ocean chlorophyll concentration prediction from remotely sensed images.

The Orthogonal Subspace Projection (OSP) algorithm is substantially a kind of matched filter that requires the evaluation of a prototype for each class to be detected. The kernel OSP (KOSP) has recently demonstrated improved results for target detection in hyperspectral images. The use of kernel helps to combat the high dimensionality problem and makes the method robust to noise. This paper presents a semi-supervised graph-based approach to improve KOSP. The proposed algorithm deforms the kernel by approximating the marginal distribution using the unlabeled samples. The good performance of the proposed method is illustrated in a toy dataset and an hyperspectral image target detection problem.

This paper presents a semi-supervised classifier based on the combination of the expectation-maximization (EM) algorithm for Gaussian mixture models (GMM) and the mean map kernel. The proposed method uses the most reliable samples in terms of maximum likelihood to compute a kernel function that accurately reflects the similarity between clusters in the kernel space. The proposed method improves classification accuracy in situations where the available labeled information does not properly describe the classes in the test image.

This letter presents a semisupervised method based on kernel machines and graph theory for remote sensing image classification. The support vector machine (SVM) is regularized with the unnormalized graph Laplacian, thus leading to the Laplacian SVM (LapSVM). The method is tested in the challenging problems of urban monitoring and cloud screening, in which an adequate exploitation of the wealth of unlabeled samples is critical. Results obtained using different sensors, and with low number of training samples, demonstrate the potential of the proposed LapSVM for remote sensing image classification.

This paper studies the performance and applicability of a novel kernel partial least squares (KPLS) algorithm for nonlinear feature extraction in the context of remote sensing applications. The so-called kernel orthonormalized PLS algorithm with reduced complexity (rKOPLS) has the following two core parts: (1) a kernel version of OPLS (called KOPLS) and (2) a sparse approximation for large-scale data sets, which ultimately leads to the rKOPLS algorithm. The method is theoretically analyzed in terms of computational burden and memory requirements and is tested in common remote sensing applications: multi- and hyperspectral image classification and biophysical parameter estimation problems. The proposed method largely outperforms the traditional (linear) PLS algorithm and demonstrates good capabilities in terms of expressive power of the extracted nonlinear features, accuracy, and scalability as compared to the standard KPLS.

The multitemporal classification of remote sensing images is a challenging problem, in which the efficient combination of different sources of information (e.g., temporal, contextual, or multisensor) can improve the results. In this paper, we present a general framework based on kernel methods for the integration of heterogeneous sources of information. Using the theoretical principles in this framework, three main contributions are presented. First, a novel family of kernel-based methods for multitemporal classification of remote sensing images is presented. The second contribution is the development of nonlinear kernel classifiers for the well-known difference and ratioing change detection methods by formulating them in an adequate high-dimensional feature space. Finally, the presented methodology allows the integration of contextual information and multisensor images with different levels of nonlinear sophistication. The binary support vector (SV) classifier and the one-class SV domain description classifier are evaluated by using both linear and nonlinear kernel functions. Good performance on synthetic and real multitemporal classification scenarios illustrates the generalization of the framework and the capabilities of the proposed algorithms.

This paper introduces the use of Relevance Vector Machines (RVM) for remote sensing hyperspectral image classification. We also include the Mahalanobis kernel in the formulation of the RVM to take into account the covariance of the features in the classification process. Experimental results in different scenarios confirm the accuracy and robustness of the proposed method, and also the ease of free parameters tuning.

This work evaluates a new semi-supervised classification framework based on kernel methods and graph theory. In particular, the support vector machine (SVM) is further regularized with the un-normalized graph Laplacian, thus leading to the proposed Laplacian SVM. The method is tested in the challenging problem of cloud screening where the objective is to identify clouds in multispectral images acquired by space-borne sensors working in the visible and near-infrared spectral range. Preliminary results obtained using MERIS/ENVISAT data show the potential of the proposed Laplacian SVM in several scenarios.

This paper presents the study of a sparse kernel-based method for non-linear feature extraction in the context of remote sensing classification and regression problems. The so-called kernel orthonormalized PLS algorithm with reduced complexity (rKOPLS) has two core parts: (i) a kernel version of OPLS (called KOPLS), and (ii) a sparse (reduced) approximation for large scale data sets, which ultimately leads to rKOPLS. The method demonstrates good capabilities in terms of expressive power of the extracted features and scalability.

This paper presents simple but powerful combination methods of dedicated one-class classifiers (OCCs) for efficient remote sensing image classification. The mean and product combination rules are applied to the probabilistic outputs generated by OCCs, and the performance is illustrated in a urban monitoring application in which multi-sensor (optical and SAR) data and multi-source (spectral and contextual) features are available. Two OCCs are used as core parts: the classical mixture of Gaussians (MoG) and the support vector domain description (SVDD) classifier. The obtained results by combining SVDD classifier outputs show a clear improvement in the accuracy, and more robustness to high dimensional samples compared to both MoG and stacked approaches.

Nonlinear system identification based on relevance vector machines (RVMs) has been traditionally addressed by stacking the input and/or output regressors and then performing standard RVM regression. This letter introduces a full family of composite kernels in order to integrate the input and output information in the mapping function efficiently and hence generalize the standard approach. An improved trade-off between accuracy and sparsity is obtained in several benchmark problems. Also, the RVM yields confidence intervals for the predictions, and it is less sensitive to free parameter selection

This paper presents a semi-supervised graph-based method for the classification of hyperspectral images. The method is designed to handle the special characteristics of hyperspectral images, namely, high-input dimension of pixels, low number of labeled samples, and spatial variability of the spectral signature. To alleviate these problems, the method incorporates three ingredients, respectively. First, being a kernel-based method, it combats the curse of dimensionality efficiently. Second, following a semi-supervised approach, it exploits the wealth of unlabeled samples in the image, and naturally gives relative importance to the labeled ones through a graph-based methodology. Finally, it incorporates contextual information through a full family of composite kernels. Noting that the graph method relies on inverting a huge kernel matrix formed by both labeled and unlabeled samples, we originally introduce the NystrÖm method in the formulation to speed up the classification process. The presented semi-supervised-graph-based method is compared to state-of-the-art support vector machines in the classification of hyperspectral data. The proposed method produces better classification maps, which capture the intrinsic structure collectively revealed by labeled and unlabeled points. Good and stable accuracy is produced in ill-posed classification problems (high dimensional spaces and low number of labeled samples). In addition, the introduction of the composite-kernel framework drastically improves results, and the new fast formulation ranks almost linearly in the computational cost, rather than cubic as in the original method, thus allowing the use of this method in remote-sensing applications.

This paper presents a methodology for cloud screening of multispectral images acquired with the Medium Resolution Imaging Spectrometer (MERIS) instrument on-board the Environmental Satellite (ENVISAT). The method yields both a discrete cloud mask and a cloud-abundance product from MERIS level-1b data on a per-pixel basis. The cloud-screening method relies on the extraction of meaningful physical features (e.g., brightness and whiteness), which are combined with atmospheric-absorption features at specific MERIS-band locations (oxygen and water-vapor absorptions) to increase the cloud-detection accuracy. All these features are inputs to an unsupervised classification algorithm; the cloud-probability output is then combined with a spectral unmixing procedure to provide a cloud-abundance product instead of binary flags. The method is conceived to be robust and applicable to a broad range of actual situations with high variability of cloud types, presence of ground covers with bright and white spectra, and changing illumination conditions or observation geometry. The presented method has been shown to outperform the MERIS level-2 cloud flag in critical cloud-screening situations, such as over ice/snow covers and around cloud borders. The proposed modular methodology constitutes a general framework that can be applied to multispectral images acquired by spaceborne sensors working in the visible and near-infrared spectral range with proper spectral information to characterize atmospheric-oxygen and water-vapor absorptions.

This paper addresses the problem of supervised classification of remote sensing images in the presence of incomplete (nonexhaustive) training sets. The problem is analyzed according to two different perspectives: 1) description and recognition of a specific land-cover class by using single-class classifiers and 2) solution of multiclass problems with single-class classification techniques. In this framework, we analyze different one-class classifiers and introduce in the remote sensing community the support vector domain description method (SVDD). The SVDD is a kernel-based method that exhibits intrinsic regularization ability and robustness versus low numbers of high-dimensional samples. The SVDD technique is compared with other standard single-class methods both in problems focused on the recognition of a single specific land-cover class and in multiclass problems. For the latter, we properly define an easily scalable multiclass architecture capable to deal with incomplete training data. Experimental results, obtained on different kinds of data (synthetic, hyperspectral, and multisensor images), point out the effectiveness of the SVDD technique and provide important indications for driving the choice of the classification technique and architecture in the presence of incomplete training data.

This paper proposes a twofold approach for therapeutic drug monitoring (TDM) of kidney recipients using support vector machines (SVMs), for both predicting and detecting Cyclosporine A (CyA) blood concentrations. The final goal is to build useful, robust, and ultimately understandable models for individualizing the dosage of CyA. We compare SVMs with several neural network models, such as the multilayer perceptron (MLP), the Elman recurrent network, finite/infinite impulse response networks, and neural network ARMAX approaches. In addition, we present a profile-dependent SVM (PD-SVM), which incorporates a priori knowledge in both tasks. Models are compared numerically, statistically, and in the presence of additive noise. Data from 57 renal allograft recipients were used to develop the models. Patients followed a standard triple therapy, and CyA trough concentration was the dependent variable. The best results for the CyA blood concentration prediction were obtained using the PD-SVM (mean error of 0.36 ng/mL and root-mean-square error of 52.01 ng/mL in the validation set) and appeared to be more robust in the presence of additive noise. The proposed PD-SVM improved results from the standard SVM and MLP, specially significant (both numerical and statistically) in the one-against-all scheme. Finally, some clinical conclusions were obtained from sensitivity rankings of the models and distribution of support vectors. We conclude that the PD-SVM approach produces more accurate and robust models than do neural networks. Finally, a software tool for aiding medical decision-making including the prediction models is presented

Hyperspectral imaging offers the possibility of characterizing materials and objects in the air, land and water on the basis of the unique reflectance patterns that result from the interaction of solar energy with the molecular structure of the material. In this paper, we provide a seminal view on recent advances in techniques for hyperspectral data processing. Our main focus is on the development of approaches able to naturally integrate the spatial and spectral information available from the data. Special attention is paid to techniques that circumvent the curse of dimensionality introduced by high-dimensional data spaces. Experimental results, focused in this work on a specific case-study of urban data analysis, demonstrate the success of the considered techniques. This paper represents a first step towards the development of a quantitative and comparative assessment of advances in hyperspectral data processing techniques.

This letter introduces the epsiv-Huber loss function in the support vector regression (SVR) formulation for the estimation of biophysical parameters extracted from remotely sensed data. This cost function can handle the different types of noise contained in the dataset. The method is successfully compared to other cost functions in the SVR framework, neural networks and classical bio-optical models for the particular case of the estimation of ocean chlorophyll concentration from satellite remote sensing data. The proposed model provides more accurate, less biased, and improved robust estimation results on the considered case study, especially significant when few in situ measurements are available

This letter presents a framework of composite kernel machines for enhanced classification of hyperspectral images. This novel method exploits the properties of Mercer's kernels to construct a family of composite kernels that easily combine spatial and spectral information. This framework of composite kernels demonstrates: 1) enhanced classification accuracy as compared to traditional approaches that take into account the spectral information only: 2) flexibility to balance between the spatial and spectral information in the classifier; and 3) computational efficiency. In addition, the proposed family of kernel classifiers opens a wide field for future developments in which spatial and spectral information can be easily integrated.

Nonlinear system identification based on support vector machines (SVM) has been usually addressed by means of the standard SVM regression (SVR), which can be seen as an implicit nonlinear autoregressive and moving average (ARMA) model in some reproducing kernel Hilbert space (RKHS). The proposal of this letter is twofold. First, the explicit consideration of an ARMA model in an RKHS (SVM-ARMA2K) is proposed. We show that stating the ARMA equations in an RKHS leads to solving the regularized normal equations in that RKHS, in terms of the autocorrelation and cross correlation of the (nonlinearly) transformed input and output discrete time processes. Second, a general class of SVM-based system identification nonlinear models is presented, based on the use of composite Mercer's kernels. This general class can improve model flexibility by emphasizing the input-output cross information (SVM-ARMA4K), which leads to straightforward and natural combinations of implicit and explicit ARMA models (SVR-ARMA2K and SVR-ARMA4K). Capabilities of these different SVM-based system identification schemes are illustrated with two benchmark problems.

This paper presents the framework of kernel-based methods in the context of hyperspectral image classification, illustrating from a general viewpoint the main characteristics of different kernel-based approaches and analyzing their properties in the hyperspectral domain. In particular, we assess performance of regularized radial basis function neural networks (Reg-RBFNN), standard support vector machines (SVMs), kernel Fisher discriminant (KFD) analysis, and regularized AdaBoost (Reg-AB). The novelty of this work consists in: 1) introducing Reg-RBFNN and Reg-AB for hyperspectral image classification; 2) comparing kernel-based methods by taking into account the peculiarities of hyperspectral images; and 3) clarifying their theoretical relationships. To these purposes, we focus on the accuracy of methods when working in noisy environments, high input dimension, and limited training sets. In addition, some other important issues are discussed, such as the sparsity of the solutions, the computational burden, and the capability of the methods to provide outputs that can be directly interpreted as probabilities.

Support vector machine (SVM) learning has been recently proposed for image compression in the frequency domain using a constant epsilon-insensitivity zone by Robinson and Kecman. However, according to the statistical properties of natural images and the properties of human perception, a constant insensitivity makes sense in the spatial domain but it is certainly not a good option in a frequency domain. In fact, in their approach, they made a fixed low-pass assumption as the number of discrete cosine transform (DCT) coefficients to be used in the training was limited. This paper extends the work of Robinson and Kecman by proposing the use of adaptive insensitivity SVMs [2] for image coding using an appropriate distortion criterion [3], [4] based on a simple visual cortex model. Training the SVM by using an accurate perception model avoids any a priori assumption and improves the rate-distortion performance of the original approach.

BACKGROUND: This paper presents the use of Support Vector Machines (SVMs) for prediction and analysis of antisense oligonucleotide (AO) efficacy. The collected database comprises 315 AO molecules including 68 features each, inducing a problem well-suited to SVMs. The task of feature selection is crucial given the presence of noisy or redundant features, and the well-known problem of the curse of dimensionality. We propose a two-stage strategy to develop an optimal model: (1) feature selection using correlation analysis, mutual information, and SVM-based recursive feature elimination (SVM-RFE), and (2) AO prediction using standard and profiled SVM formulations. A profiled SVM gives different weights to different parts of the training data to focus the training on the most important regions. RESULTS: In the first stage, the SVM-RFE technique was most efficient and robust in the presence of low number of samples and high input space dimension. This method yielded an optimal subset of 14 representative features, which were all related to energy and sequence motifs. The second stage evaluated the performance of the predictors (overall correlation coefficient between observed and predicted efficacy, r; mean error, ME; and root-mean-square-error, RMSE) using 8-fold and minus-one-RNA cross-validation methods. The profiled SVM produced the best results (r = 0.44, ME = 0.022, and RMSE= 0.278) and predicted high (>75% inhibition of gene expression) and low efficacy (<25%) AOs with a success rate of 83.3% and 82.9%, respectively, which is better than by previous approaches. A web server for AO prediction is available online at http://aosvm.cgb.ki.se/. CONCLUSIONS: The SVM approach is well suited to the AO prediction problem, and yields a prediction accuracy superior to previous methods. The profiled SVM was found to perform better than the standard SVM, suggesting that it could lead to improvements in other prediction problems as well.

We propose the use of support vector machines (SVMs) for automatic hyperspectral data classification and knowledge discovery. In the first stage of the study, we use SVMs for crop classification and analyze their performance in terms of efficiency and robustness, as compared to extensively used neural and fuzzy methods. Efficiency is assessed by evaluating accuracy and statistical differences in several scenes. Robustness is analyzed in terms of: (1) suitability to working conditions when a feature selection stage is not possible and (2) performance when different levels of Gaussian noise are introduced at their inputs. In the second stage of this work, we analyze the distribution of the support vectors (SVs) and perform sensitivity analysis on the best classifier in order to analyze the significance of the input spectral bands. For classification purposes, six hyperspectral images acquired with the 128-band HyMAP spectrometer during the DAISEX-1999 campaign are used. Six crop classes were labeled for each image. A reduced set of labeled samples is used to train the models, and the entire images are used to assess their performance. Several conclusions are drawn: (1) SVMs yield better outcomes than neural networks regarding accuracy, simplicity, and robustness; (2) training neural and neurofuzzy models is unfeasible when working with high-dimensional input spaces and great amounts of training data; (3) SVMs perform similarly for different training subsets with varying input dimension, which indicates that noisy bands are successfully detected; and (4) a valuable ranking of bands through sensitivity analysis is achieved.

In this paper, a twofold approach to improve the performance of genetic algorithms (GAs) in the feature selection problem (FSP) is presented. First, a novel genetic operator is introduced to solve the FSP. This operator fixes in each iteration the number of features to be selected among the available ones and consequently reduces the size of the search space. This approach yields two main advantages: a) training the learning machine becomes faster and b) a higher performance is achieved by using the selected subset. Second, we propose using the Walsh expansion of the FSP fitness function in order to perform ranking on the problem features. Ranking features have been traditionally considered to be a challenging problem, especially significant in health sciences where the number of available and potentially noisy signals is high. Three real biological datasets are used to test the behavior of the two approaches proposed.

In this paper, we propose a procedure to reduce data dimensionality while preserving relevant information for posterior crop cover classification. The huge amount of data involved in hyperspectral image processing is one of the main problems in order to apply pattern recognition techniques. We propose a dimensionality reduction strategy that eliminates redundant information and a subsequent selection of the most discriminative features based on classification and regression trees (CART). CART allow feature selection based on the classification success, it is a non-linear method and specially allows knowledge discovery. The main advantage of our proposal relies on model interpretability, since we can get qualitative information by analyzing the surrogate and main splits of the tree. This method is tested with a crop cover recognition application of six hyperspectral images from the same area acquired with the 128-bands HyMap spectrometer. Even though CART do not provide the best results in classification it is useful for a previous pre-processing step of feature selection. Finally, we analyze the selected bands of the input space in order to gain knowledge on the problem and to give a physical interpretation of results.

In this paper, we propose a procedure to reduce dimensionality of hyperspectral data while preserving relevant information for posterior crop cover classification. One of the main problems with hyperspectral image processing is the huge amount of data involved. In addition, pattern recognition methods are sensitive to problems associated to high dimensionality feature spaces (referred to as Hughes phenomenon of curse of dimensionality). We propose a dimensionality reduction strategy that eliminates redundant information by means of local correlation criterion between contiguous spectral bands; and a subsequent selection of the most discriminative features based on a Sequential Float Feature Selection algorithm. This method is tested with a crop cover recognition application of six hyperspectral images from the same area acquired with the 128-bands HyMap spectrometer during the DAISEX99 campaign. In the experiments, we analyze the dependence on the dimension and employed metrics. The results obtained using the Gaussian Maximum Likelihood improve the classification accuracy and confirm the validity of the proposed approach. Finally, we analyze the selected bands of the input space on order to gain knowledge on the problem and to give a physical interpretation of the results.

A new approach to the classification of hyperspectral images is proposed. The main problem with supervised methods is that the learning process heavily depends on the quality of the training data set. In remote sensing, the training set is useful only for simultaneous images or for images with the same classes taken under the same conditions; and, even worse, the training set is frequently not available. On the other hand, unsupervised methods are not sensitive to the number of labelled samples since they work on the whole image. Nevertheless, relationship between clusters and classes is not ensured. In this context, we propose a combined strategy of supervised and unsupervised learning methods that avoids these drawbacks and automates the classification process. The method is based on the general formulation of the expectation-maximization (EM) algorithm. This method is applied to crop cover recognition of six hyperspectral images from the same area acquired with the HyMap spectrometer during the DAISEX-99 campaign. For classification purposes, six different classes are considered. Classification accuracy results are compared to common methods: ISODATA, Learning Vector Quantization, Gaussian Maximum Likelihood, Expectation-Maximization, and Neural Networks. The good performance confirms the validity of the proposed approach in terms of accuracy and robustness.

A novel fuzzy-based activation function for artificial neural networks is proposed. This approach provides easy hardware implementation and straightforward interpretability in the basis of IF-THEN rules. Backpropagation learning with the new activation function also has low computational complexity. Several application examples ( XOR gate, chaotic time-series prediction, channel equalization, and independent component analysis) support the potential of the proposed scheme.