Vegetation indices allow to efficiently monitor vegetation growth and agricultural activities. Previous generations of satellites were capturing a limited number of spectral bands, and a few expert-designed vegetation indices were sufficient to harness their potential. New generations of multi- and hyperspectral satellites can however capture additional bands, but are not yet efficiently exploited. In this work, we propose an explainable-AI-based method to select and design suitable vegetation indices. We first train a deep neural network using multispectral satellite data, then extract feature importance to identify the most influential bands. We subsequently select suitable existing vegetation indices or modify them to incorporate the identified bands and retrain our model. We validate our approach on a crop classification task. Our results indicate that models trained on individual indices achieve comparable results to the baseline model trained on all bands, while the combination of two indices surpasses the baseline in certain cases.

Earth observation applications effectively leverage deep learning models to harness the abundantly available remote sensing data. In order to use all the different modalities relevant to a specific task, the fusion of these data sources can be achieved using multi-modal learning techniques. This is especially helpful when the input dataset contains both images and tabular data, or when the temporal and spatial resolutions vary across the modalities of interest. Nevertheless, these fusion techniques typically increase in complexity as the disparities in the nature of the fused modalities increase. The resulting complex deep learning models suffer from a lack of explainability and transparency, which is crucial in many sensitive human-related applications. In this letter, we describe how the research community in remote sensing addresses the issue of model explainability in the context of multi-modal learning. We additionally review the practices used in other application fields and identify some of the most promising explainability methods tailored for multi-modal deep networks to be exploited in remote sensing applications.

Accurate and reliable crop yield prediction is a complex task. The yield of a crop depends on a variety of factors whose accurate measurement and modeling is challenging. At the same time, reliable yield prediction is highly desirable for farmers to optimize crop production. In this paper, we introduce a modeling based on remote sensing data and Machine Learning models evaluated on a large-scale dataset to address the challenge of an operational crop yield estimation and forecasting on field and subfield level. With our approach, we aim towards a global yield modeling based on Machine Learning models which operates across crop types without the need for crop-specific modeling. We demonstrate that our approach learns to map in-field variability for all studied crop types. Overall, the predictions have an error (RRMSE) of around 15% and an R2value of 0.77 at field level.

Numerous remote sensing applications rely on temporal satellite data, and Deep learning models are increasingly being used for such tasks. Nevertheless, these models operate as black boxes, lacking transparency and understandability. We address this gap by using explainable AI on an agricultural task. Specifically, we trained a recurrent neural network on individual pixels from multispectral time-series of Sentinel-2 satellite images to predict crop yield. We then applied nine feature attribution methods on a sample of the dataset and computed the spectral and temporal contributions to the final individual predictions. The aggregated results were evaluated qualitatively and quantitatively. Results suggest that LIME and Shapley sampling value methods performed best on the quantitative scores, followed by GradientShap. Most backpropagation-based techniques had highly inconsistent scores across the explained data points. Finally, to guide remote sensing practitioners in using Explainable AI on similar datasets, we further discuss some selection criteria to be considered.

We introduce a simple yet effective early fusion method for crop yield prediction that handles multiple input modalities with different temporal and spatial resolutions. We use high-resolution crop yield maps as ground truth data to train crop and machine learning model agnostic methods at the sub-field level. We use Sentinel-2 satellite imagery as the primary modality for input data with other complementary modalities, including weather, soil, and DEM data. The proposed method uses input modalities available with global coverage, making the framework globally scalable. We explicitly highlight the importance of input modalities for crop yield prediction and emphasize that the best-performing combination of input modalities depends on region, crop, and chosen model.

Modern combine harvesters can collect geo-located real-time yield measurement while harvesting. This data can be used to train Machine Learning models that predict the yield at sub-field level based on remote sensing input data. The performance of these models is, however, highly dependent on the quality of the yield data. It is therefore important to develop automatic cleaning techniques to correct for common errors in combine harvester yield maps. In this work, we compare different combinations of data cleaning techniques by evaluating their impact on the yield-prediction model performance at field and sub-field level. Our findings indicate that basic cleaning techniques such as absolute thresholds are sufficient at the field level, whereas the performance at the sub-field level is enhanced through the utilization of more intricate statistical cleaning methods.

Many biological processes are mediated by protein-protein interactions (PPIs). Because protein domains are the building blocks of proteins, PPIs likely rely on domain-domain interactions (DDIs). Several attempts exist to infer DDIs from PPI networks but the produced datasets are heterogeneous and sometimes not accessible, while the PPI interactome data keeps growing. We describe a new computational approach called "PPIDM" (Protein-Protein Interactions Domain Miner) for inferring DDIs using multiple sources of PPIs. The approach is an extension of our previously described "CODAC" (Computational Discovery of Direct Associations using Common neighbors) method for inferring new edges in a tripartite graph. The PPIDM method has been applied to seven widely used PPI resources, using as "Gold-Standard" a set of DDIs extracted from 3D structural databases. Overall, PPIDM has produced a dataset of 84,552 non-redundant DDIs. Statistical significance (p-value) is calculated for each source of PPI and used to classify the PPIDM DDIs in Gold (9,175 DDIs), Silver (24,934 DDIs) and Bronze (50,443 DDIs) categories. Dataset comparison reveals that PPIDM has inferred from the 2017 releases of PPI sources about 46% of the DDIs present in the 2020 release of the 3did database, not counting the DDIs present in the Gold-Standard. The PPIDM dataset contains 10,229 DDIs that are consistent with more than 13,300 PPIs extracted from the IMEx database, and nearly 23,300 DDIs (27.5%) that are consistent with more than 214,000 human PPIs extracted from the STRING database. Examples of newly inferred DDIs covering more than 10 PPIs in the IMEx database are provided. Further exploitation of the PPIDM DDI reservoir includes the inventory of possible partners of a protein of interest and characterization of protein interactions at the domain level in combination with other methods. The result is publicly available at http://ppidm.loria.fr/.