Semi-supervised domain adaptation aims to transfer knowledge from a labeled source domain to a scarcely labeled target domain, despite distribution shifts. The challenge becomes greater when source and target data differ in acquisition modality, as in remote sensing where variations in sensor type (e.g., optical vs. radar), spectral properties (e.g., RGB vs. multispectral), or spatial resolution are common. This challenging scenario, known as Semi-Supervised Heterogeneous Domain Adaptation (SSHDA), requires learning across modalities with limited target labels. In this work, we propose SAHARA (Semi-supervised Adaptation in Heterogeneous domains via conditional Adversarial Representation disentanglement and Adaptive pseudo-labeling), a new method for SSHDA that combines conditional adversarial feature adaptation with dynamic pseudo-labeling to learn domain-invariant features and handle extremely scarce target annotations. Experiments on two heterogeneous remote sensing benchmarks for scene classification, conducted with both convolutional and transformer-based backbones, demonstrate that SAHARA consistently outperforms existing SSHDA and semi-supervised methods. The code is available at https: //TO-BE-DISCLOSED-UPON-ACCEPTANCE.

Pre-trained vision-language models (VLMs), such as CLIP, demonstrate impressive zero-shot classification capabilities with free-form prompts and even show some generalization in specialized domains. However, their performance on satellite imagery is limited due to the underrepresentation of such data in their training sets, which predominantly consist of ground-level images. Existing prompting techniques for satellite imagery are often restricted to generic phrases like “a satellite image of …”, limiting their effectiveness for zero-shot land-use/land-cover (LULC) mapping. To address these challenges, we introduce SenCLIP, which transfers CLIP's representation to Sentinel-2 imagery by leveraging a large dataset of Sentinel-2 images paired with geotagged ground-level photos from across Europe. We evaluate SenCLIP alongside other state-of-the-art remote sensing VLMs on zero-shot LULC mapping tasks using the EuroSAT and BigEarthNet datasets with both aerial and ground-level prompting styles. Our approach, which aligns ground-level representations with satellite imagery, demonstrates significant improvements in classification accuracy across both prompt styles, opening new possibilities for applying free-form textual descriptions in zero-shot LULC mapping. Code, dataset and pretrained models are available at https://github.com/pallavijain-pj/SenCLIP

In many scientific fields, complex systems are characterized by a multitude of heterogeneous interactions/relationships that are challenging to model. Multilayer graphs constitute valuable tools that can represent such complex systems, thus making possible their analysis for downstream decision-making processes. Nevertheless, modeling such complex information still remains challenging in real-world scenarios. On the one hand, holistically including all relationships may lead to noisy or computationally intensive graphs. On the other hand, limiting the amount of information to model through the selection of a portion of the available relationships can introduce boundary specification biases. However, the current research studies are demonstrating that it is more beneficial to retain as much information as possible and at a later stage perform graph simplification i.e., removing uninformative or redundant parts of the graph to facilitate the final analysis. While simplification strategies, based on deep learning methods, have been already extensively explored in the context of single-layer graphs, only a limited amount of efforts have been devoted to simplification strategies for multilayer graphs. In this work, we propose the MultilAyer gRaph simplificAtion ( MARA ) framework, a GNN-based approach designed to simplify multilayer graphs based on the downstream task. MARA generates node embeddings for a specific task by training jointly two main components: (i) an edge simplification module and (ii) a (multilayer) graph neural network. We tested MARA on different real-world multilayer graphs for node classification tasks. Experimental results show the effectiveness of the proposed approach: MARA reduces the dimension of the input graph while keeping and even improving the performance of node classification tasks in different domains and across graphs characterized by different structures. Moreover, deep learning-based simplification allows MARA to preserve and enhance important graph properties for the downstream task. To our knowledge, MARA represents the first simplification framework especially tailored for multilayer graphs analysis.

Effectively characterize the current land cover status is crucial for assessing agricultural production, monitoring natural resources, and making informed land management decisions. Satellite Image Time Series (SITS) data, capturing spatio-temporal information, are the primary source of information to support the general task of Land Use Land Cover (LULC) mapping. With the aim to effectively exploit the spatio-temporal information carried out by SITS data for the underlying task of land cover mapping, here we introduce a multi-scale classification framework that combines together pixel-level and object-level multivariate SITS information with the objective to ameliorate the pixel-level time series analysis. To assess the behaviour of the proposed framework, we provide a comparative analysis with several SITS-based land cover mapping strategies on a challenging study area, namely Koumbia, located in the Burkina Faso. The obtained results reveal that the joint use of pixel-level and object-level information clearly ameliorates the classification performances.

Forest tree dieback inventory plays a crucial role to improve forest management strategies. In this study, we explore the performance of a spectral-spatial machine learning approach used to analyse Sentinel-2 images to detect forest tree dieback events due to bark beetle infestation. We analyse the performance of classification models trained with Random Forest, XGBoost and Multi-Layer Perceptron, as well as semantic segmentation models trained with U-Net by accounting for both spectral and spatial information contained in the remote sensing data. We consider a set of Sentinel-2 images acquired in non-overlapping forest scenes from a region located in the Northeast of France. The selected scenes host bark beetle infestation hotspots originated from the mass reproduction of the bark beetle in the 2018 infestation. Results show that the U-Net model, trained accounting for spectral and spectral-spatial data, achieves the best performance. However, the simpler Random Forest model achieves competitive results with respect to the more complex one, namely U-Net.

This paper analyzes the accuracy on the detection of rapeseed fields using Sentinel-1 (S1) time series. Random Forest (RF) and three deep learning (DL) algorithms namely Long Short-Term Memory Fully Convolutional Network (LSTM-FCN), InceptionTime, and Multi-layer Perceptron (MLP) were tested in this study. All four algorithms were used to classify the S1 time series with a large number of ground samples. To test the transferability of classification models, the algorithms were trained on a given year, and then tested on different years. The results demonstrated the high performance of all four algorithms in mapping rapeseed fields when using different years in training and testing phases (F1 between 85.5% and 92.7%, kappa between 0.85 and 0.93).

Highly accurate estimates of canopy height (CH) and above ground biomass (AGB) are key parameters for forest disturbance analysis, resource monitoring, and carbon flux analyses. In this work we present a deep learning-based approach for mapping CH and AGB on country-scales from single-baseline, single-polarization, single-pass TanDEM-X InSAR data. The proposed approach consists in a convolutional neural network (CNN), trained and validated on the five test-sites covered by the 2016 AfriSAR campaign. The resulting performance is in line or better than those of current state-of-the-art approaches. The framework is subsequently deployed on a large-scale to map the entire country of Gabon (in West Central Africa), showcasing the flexibility, scalability, and accuracy of our proposed approach for forest parameter estimation.

A major difference between ground-level and satellite imagery of landscapes lies in their semantic granularity: ground-level images tend to offer details on objects and human activities, while satellite images provide broader geographic context but, typically, with coarser semantics. This study aims to leverage this complementary information by integrating fine-grained insights from a ground-level view into the analysis of satellite image data. To achieve this integration, we propose to align a satellite image representation with co-located geo-tagged ground-level image CLIP representations. This method focuses on enriching satellite image visual features by leveraging the inherent visual characteristics found in ground-level images as a reference in a contrastive manner, without relying on additional textual information to guide the learning process. We evaluate the quality of the learned representations on the EuroSAT benchmark in various few-shot settings.

With the advent of the Sentinel-2 mission and its high revisit frequency, high-resolution time series of optical images, the use of satellite image time series for automatic land cover mapping has fostered. However, one of the main limitations related to this kind of imagery is the presence of clouds, which often hinders its descriptive potential by reducing the actual temporal resolution. Although some common practices exist to enable their use in land cover processing chains, the majority of them aims at reconstructing the time series upstream to the classification task, hence introducing a heavy, error-prone pre-processing step. With the aim of exploiting the capacity of deep learning networks to adaptively combine tasks, in this preliminary study we propose an end-to-end framework that simultaneously perform cloud removal and classification of a Sentinel-2 image time series for the downstream task of land cover mapping. The proposed framework is evaluated over an agricultural area in Northern Benin. Our first results show comparable performances with respect to using state-of-the-art gap filling pre-processing on Sentinel-2 time series, hence motivating further exploration.

Deep learning algorithms have recently gained significant attention due to their impressive performance. However, their high complexity and un-interpretable mode of operation hinders their confident deployment in real-world safety-critical tasks. This work targets ante hoc interpretability, and specifically Concept Bottleneck Models (CBMs). Our goal is to design a framework that admits a highly interpretable decision making process with respect to human understandable concepts, on two levels of granularity. To this end, we propose a novel two-level concept discovery formulation leveraging: (i) recent advances in vision-language models, and (ii) an innovative formulation for coarse-to-fine concept selection via data-driven and sparsity inducing Bayesian arguments. Within this framework, concept information does not solely rely on the similarity between the whole image and general unstructured concepts; instead, we introduce the notion of concept hierarchy to uncover and exploit more granular concept information residing in patch-specific regions of the image scene. As we experimentally show, the proposed construction not only outperforms recent CBM approaches, but also yields a principled framework towards interpetability.

Abstract Forest tree dieback inventory has a crucial role in improving forest management strategies. This inventory is traditionally performed by forests through laborious and time-consuming human assessment of individual trees. On the other hand, the large amount of Earth satellite data that are publicly available with the Copernicus program and can be processed through advanced deep learning techniques has recently been established as an alternative to field surveys for forest tree dieback tasks. However, to realize its full potential, deep learning requires a deep understanding of satellite data since the data collection and preparation steps are essential as the model development step. In this study, we explore the performance of a data-centric semantic segmentation approach to detect forest tree dieback events due to bark beetle infestation in satellite images. The proposed approach prepares a multisensor data set collected using both the SAR Sentinel-1 sensor and the optical Sentinel-2 sensor and uses this dataset to train a multisensor semantic segmentation model. The evaluation shows the effectiveness of the proposed approach in a real inventory case study that regards non-overlapping forest scenes from the Northeast of France acquired in October 2018. The selected scenes host bark beetle infestation hotspots of different sizes, which originate from the mass reproduction of the bark beetle in the 2018 infestation.

In this paper, we explore the usage of hierarchical priors to improve learning in contexts where the number of available examples is extremely low.Specifically, we consider a Prototype Learning setting where deep neural networks are used to embed data in hyperspherical geometries.In this scenario, we propose an innovative way to learn the prototypes by combining class separation and hierarchical information.In addition, we introduce a contrastive loss function capable of balancing the exploitation of prototypes through a prototype pruning mechanism.We compare the proposed method with state-of-the-art approaches on two public datasets.This work has been partially supported by the Spoke 1 "FutureHPC & BigData" of ICSC -Centro Nazionale di Ricerca in High-Performance-Computing, Big Data and Quantum Computing, funded by European Union -NextGenerationEU.

Fine-grained classification often requires recognizing specific object parts, such as beak shape and wing patterns for birds. Encouraging a fine-grained classification model to first detect such parts and then using them to infer the class could help us gauge whether the model is indeed looking at the right details better than with interpretability methods that provide a single attribution map. We propose PDiscoNet to discover object parts by using only image-level class labels along with priors encouraging the parts to be: discriminative, compact, distinct from each other, equivariant to rigid transforms, and active in at least some of the images. In addition to using the appropriate losses to encode these priors, we propose to use part-dropout, where full part feature vectors are dropped at once to prevent a single part from dominating in the classification, and part feature vector modulation, which makes the information coming from each part distinct from the perspective of the classifier. Our results on CUB, CelebA, and PartImageNet show that the proposed method provides substantially better part discovery performance than previous methods while not requiring any additional hyper-parameter tuning and without penalizing the classification performance. The code is available at https://github.com/robertdvdk/part_detection

Large-scale and up-to-date canopy height model (CHM) estimates are key to forest resources assessment and disturbance analysis. In this work we present an investigation of the potential of Deep Learning (DL) for the regression of forest height from TanDEM-X bistatic InSAR data. We propose a novel fully convolutional neural network (CNN) framework, trained and tested on four tropical sites in Gabon, Africa, together with a series of experiments for assessing the impact of different input features with specific focus on bistatic InSAR. The obtained results are extremely promising and already in line with state-of-the-art methods based on theoretical modelling, with the remarkable advantage of requiring only one single TanDEM-X acquisition at inference time.

Huge amount of data are nowadays produced by a large and disparate family of sensors, which typically measure multiple variables over time. Such rich information can be profitably organized as multivariate time-series. Collect enough labelled samples to set up supervised analysis for such kind of data is challenging while a reasonable assumption is to dispose of a limited background knowledge that can be injected in the analysis process. In this context, semi-supervised clustering methods represent a well suited tool to get the most out of such reduced amount of knowledge. With the aim to deal with multivariate time-series analysis under a limited background knowledge setting, we propose a semi-supervised (constrained) deep embedding time-series clustering framework that exploits knowledge supervision modeled as Must- and Cannot-link constraints. More in detail, our proposal, named conDetSEC (constrained Deep embedding time SEries Clustering), is based on Gated Recurrent Units (GRUs) with the aim to explicitly manage the temporal dimension associated to multi-variate time series data. conDetSEC implements a procedure in which an embedding generation step is combined with a clustering refinement step. Both steps exploit the small amount of available knowledge provided by Must- and Cannot-link constraints. More specifically, during the data embedding generation the constraints are used by jointly optimizing the network parameters via both unsupervised and semi-supervised tasks, while at the refinement step they are used in conjunction with the goal to stretch the embedding manifold towards the clustering centroids to recover a more clear cluster structure. Experimental evaluation on real-world benchmarks coming from diverse domains has highlighted the effectiveness of our proposal in comparison with state-of-the-art unsupervised and semi-supervised time-series clustering methods.

Abstract Semi-supervised learning has shown its potential in many real-world applications where only few labeled examples are available. However, when some fairness constraints need to be satisfied, semi-supervised classification models often struggle as they are required to cope with the lack of sufficient information for predicting the target variable while forgetting its relationships with any sensitive and potentially discriminatory attribute. To address this issue, we propose a fair semi-supervised representation learning architecture that leads to fair and accurate classification results even in very challenging scenarios with few labeled (but biased) instances. We show experimentally that our model can be easily adopted in very general settings, as the learned representations may be employed to train any supervised classifier. Moreover, when applied to several synthetic and real-world datasets, our method is competitive with state-of-the-art fair semi-supervised approaches.

With the huge variety of earth observation satellite missions available nowadays, the collection of multi-sensor remote sensing information depicting the same geographical area has become systematic in practice, paving the way to the further breakthroughs in automatic land cover mapping with the aim to support decision makers in a variety of land management applications. In this context, along with the increase in the volume of data available, the availability of ground truth data to train supervised models, which is usually time-consuming and costly, may even be more critical. In this scenario, the possibility to transfer a model learnt on a particular time span (source domain) to a different period of time (target domain), over the same geographical area, can be advantageous in terms of both cost and time efforts. However, such model transfer is challenging due to different climate, weather or environmental conditions affecting remote sensing data collected at different time periods, resulting in possible distribution shifts between thesourceandtargetdomains. With the aim to cope with the multi-sensor temporal transfer scenario in the context of land cover mapping, where multi-temporal and multi-scale information are used jointly, we proposeM3SPADA(Multi-sensor, Multi-temporal and Multi-scale SPatially-Aware Domain Adaptation framework), a deep learning methodology that jointly exploits self-training and adversarial learning to transfer a multi-sensor land cover classifier from a time period (year) to a different one on the same geographical area. Here, we consider the case in which each domain (source and target) is described by a pair of remote sensing data sets: a satellite image time series (SITS) of optical images and a single Very High spatial Resolution (VHR) scene. Experimental evaluation on a real-world study case located in Burkina Faso and characterized by operational constraints shows the quality of our proposal to deal with the temporal multi-sensor transfer in the context of land cover mapping.

Nowadays, Satellite Image Time Series (SITS) are employed as input to derive land cover maps (LCM) to support decision makers in several application domains like agriculture and biodiversity. The generation of LCM largely relies on available ground truth (GT) data to calibrate supervised ma-chine learning models. Unfortunately, this data are not always accessible. In this scenario, the possibility to transfer a model learnt on a particular year (source domain) to another period of time (target domain) could be a valuable tool to deal with the previously mentioned restrictions. In this paper, we provide an experimental evaluation of recent Unsupervised Domain Adaptation (UDA) methods in the specific context of temporal transfer learning for SITS-based LCM. The objective is to learn a classification model at a certain year (exploiting available GT data) and, successively, transfer such a model on a subsequent year where no labelled samples are accessible. The obtained findings reveal that UDA methods represent a promising research direction to cope with the problem of temporal transfer learning for LCM. While a model learnt on the source data and directly applied on target data achieves an weighted F1-score of 67.1, the best UDA method obtains an F1-score of 83.7 with more than 15 points of positive gap. Nevertheless, there is still room for improvement that should be explored in future works.

The estimation of forest parameters, such as canopy height model (CHM) and above ground biomass (AGB), is of ut-most importance for forest monitoring, carbon-cycle modelling, disturbance analysis, resource inventorying and natural disaster prevention. In this work, we profit from the most recent advancements in deep learning research to propose a convolutional neural network (CNN) architecture for frequent forest parameter estimation at large scale. Our technique consists of a fully convolutional, multi-modal framework, which works on a single set of complementary multi-spectral and interferometric SAR data, acquired by ESA's Sentinel-2 and DLR's TanDEM-X missions, respectively. The regression performance of our framework has been tested over four tropical forest test sites in Gabon, Africa. The estimation of CHM shows promising early results when compared to state-of-the-art methods and has the advantage of requiring only a single input image pair instead of a longer time-series, as commonly done for state-of-the-art model-based techniques.

The Global Ecosystem Dynamics Investigation (GEDI) instrument, as all FW systems, relies on very sophisticated pre-processing steps to generate a priori metrics in order to accurately estimate forest characteristics, such as forest heights and wood volume. The ever-expanding volume of acquired GEDI data, which to September 2020 comprised more than 25 billion shots, and requiring more than 90 TB of storage space, raises new challenges in terms of adapted preprocessing methods for the suitable exploitation of such a huge and complex amount of LiDAR data. Therefore, to avoid metric computation, we leveraged deep learning techniques in order to estimate canopy dominant heights (Hdom) and wood volume (V) of Eucalyptus plantations over five different regions in Brazil. Performance comparisons were conducted between a convolutional neural network based model that uses GEDI waveform data and a previously used, metric based, Random Forest regressor (RF). Cross-validated results showed that the CNN based model compared well against the RF counterpart for both Hdomand V. Indeed, the RMSE on the estimation of Hdomfrom the CNN based model was 1.61 m with a coefficient of determination R2of 0.90, while the RF model produced an accuracy on Hdomestimates of 1.45 m(R2=0.92). For V, CNN based estimates was 27.35 m3.ha-1(R2of 0.88), while for RF, the RMSE was 27.60 m3.ha-1 (R2=0.88).

Very High spatial Resolution (VHR) imagery is a standard in-put to derive fine grain land cover maps (LCM) to support pol-icy makers in many application domains like urban planning and biodiversity. The generation of LCM mainly relies on available ground truth (GT) data to calibrate machine learning methods. Unfortunately, this data is not always accessi-ble. In this scenario, the possibility to transfer a model learnt on a certain period (source domain), where GT data is avail-able, to another period (target domain) without the necessity to collect new GT data would be a cost-effective strategy. To cope with this issue, in this paper, we present a re-search study on temporal transfer learning for the semantic segmentation of VHR imagery. To this end, we propose a case study in which a lightweight procedure such as histogram matching and a recent domain adaptation technique are com-bined together to cope with possible distribution shifts affecting VHR imagery acquired on the same area but at different period of time.

Many real world data can be modeled by a graph with a set of nodes interconnected to each other by multiple relationships. Such a rich graph is called multilayer graph or network. Providing useful visualization tools to support the query process for such graphs is challenging. Although many approaches have addressed the visual query construction, few efforts have been done to provide a contextualized exploration of query results and suggestion strategies to refine the original query. This is due to several issues such as i) the size of the graphs ii) the large number of retrieved results and iii) the way they can be organized to facilitate their exploration. In this article, we present VERTIGo, a novel visual platform to query, explore and support the analysis of large multilayer graphs. VERTIGo provides coordinated views to navigate and explore the large set of retrieved results at different granularity levels. In addition, the proposed system supports the refinement of the query by visual suggestions to guide the user through the exploration process. Two examples and a user study demonstrate how VERTIGo can be used to perform visual analysis (query, exploration, and suggestion) on real world multilayer networks.

The need for reliable and exhaustive data on land use is a major issue in planning policies and monitoring of land take. In this work, we evaluate urban building classification resulting from deep learning based approaches using as input SPOT 6/7 satellite images (at a spatial resolution of 1.5m) and national databases. In addition to assessing the classifier behaviour on urban land cover, the objective here is to compare the deep learning results at both pixel-based and object-based scale to qualify the overall results. Standard evaluation metrics (such as F-score) have shown better scores at the object-based assessment level (median F-score = 0.78) than the pixel ones (median F-score = 0.68). Identifying the impact of object characteristics using the object-based level of analysis has also revealed that beyond a surface area of 100 m2, objects are much better detected (median F-score > 0.91). It is the same with a high urban density (median F-score > 0.97). The accuracy of the intersections evaluated from the Intersection over Union also follows these trends as the entities area increases.

Deep Neural Networks (DNNs) are getting increasing attention to deal with land cover classification relying on Satellite Image Time Series (SITS). Though high performances can be achieved, the rationale of a prediction yielded by a DNN often remains unclear. An architecture expressing predictions with respect to input channels is thus proposed in this paper. It relies on convolutional layers and an attention mechanism weighting the importance of each channel in the final classification decision. The correlation between channels is taken into account to set up shared kernels and lower model complexity. Experiments based on Sentinel-2 SITS show promising results.

The obj ective of this paper is to present an approach for mapping irrigated areas at plot scale using the Sentinel-1 radar time series. Over a study site located in Catalonia region of north Spain, a dense temporal series of S1 backscattering coefficients were first obtained at plot scale and grid scale (10km x 10km). The S1 time series at plot and grid scales were conjointly used to remove the ambiguity between rainfall events and irrigation events. The principal component analysis (PCA) and the wavelet transformation were applied to the SAR temporal series. Then, to classify irrigated/non-irrigated plots the random forest (RF) classifier was employed using the obtained principal components (PC) and the wavelet coefficients (WT). A convolutional neural network was also tested using the prepared S1 temporal series. The result of the classification reaches 90.7% and 89.1% using the PC and the WT in a random forest classifier respectively. The accuracy of the classification reaches 94.1% using the CNN.

This letter proposes a deep learning model to deal with the spatial transfer challenge for the mapping of irrigated areas through the analysis of Sentinel-1 data. First, a convolutional neural network (CNN) model called “Teacher Model” is trained on a source geographical area characterized by a huge volume of samples. Then, this model is transferred from the source area to the target area characterized by a limited number of samples. The transfer learning framework is based on a distill and refine strategy, in which the teacher model is first distilled into a student model and, successively, refined by data samples coming from the target geographical area. The proposed strategy is compared with different approaches including a random forest (RF) classifier trained on the target data set and a CNN trained on the source data set and directly applied on the target area as well as several CNN classifiers trained on the target data set. The evaluation of the performed transfer strategy shows that the “distill and refine” framework obtains the best performance compared with other competing approaches. The obtained findings represent a first step toward the understanding of the spatial transferability of deep learning models in the Earth observation domain.

Modern earth observation (EO) systems produce huge volumes of images with the objective to monitor the earth surface. Due to the high revisit time of EO systems, such as Sentinel-2 constellation, satellite image time series (SITS) is continuously produced allowing to improve the monitoring of spatiotemporal phenomena. How to efficiently analyze SITS considering both spectral and spatial information is still an open question in the remote sensing field. To deal with SITS classification, in this letter, we propose a spatio-spectral classification framework that leverages the mathematical morphology to extract spatial characteristics from SITS data and combines them with the already available spectral and temporal information. Experiments carried out on two study sites characterized by different heterogeneous land covers have demonstrated the significance of our proposal and the value to combine spatial as well as spectral information in the context of SITS land cover classification.

Semisupervised learning (SSL) is a family of classification methods conceived to reduce the amount of required labeled information in the training phase. Graph-based methods are among the most popular semisupervised strategies: the nearest neighbor graph is built in such a way that the manifold of the data is captured and the labeled information is propagated to target samples along the structure of the manifold. Research in graph-based SSL has mainly focused on two aspects: 1) the construction of the k -nearest neighbors graph and/or 2) the propagation algorithm providing the classification. Differently from the previous literature, in this article, we focus on the data representation with the aim of incorporating semisupervision earlier in the process. To this end, we propose an algorithm that learns a new knowledge-aware data embedding via an ensemble of semisupervised autoencoders to enhance a graph-based semisupervised classification. The experiments carried out on different classification tasks demonstrate the benefit of our approach.

A major issue affecting optical imagery is the presence of clouds. The need of cloud-free scenes at specific date is crucial in a number of operational monitoring applications. On the other hand, the cloud-insensitive SAR sensors are a solid asset and they provide orthogonal information with respect to optical satellite, that enable the retrieval of information lost in optical images due to cloud cover. In the context of an increasing availability of both optical and SAR images, thank to the Sentinel constellation, we propose a deep learning method to reconstruct (gap-fill) optical data, polluted by cloud phenomena, exploiting multi-temporal SAR and optical images.

Radar and Optical Satellite Image Time Series (SITS) are sources of information that are commonly employed to monitor earth surfaces for tasks related to ecology, agriculture, mobility, land management planning and land cover monitoring. Many studies have been conducted using one of the two sources, but how to smartly combine the complementary information provided by radar and optical SITS is still an open challenge. In this context, we propose a new neural architecture for the combination of Sentinel-1 (S1) and Sentinel-2 (S2) imagery at object level, applied to a real-world land cover classification task. Experiments carried out on the Reunion Island, a overseas department of France in the Indian Ocean, demonstrate the significance of our proposal.

In most real world clustering scenarios, experts generally dispose of limited background information, but such knowledge is valuable and may guide the analysis process. Semi-supervised clustering can be used to drive the algorithmic process with prior knowledge and to enable the discovery of clusters that meet the analyst's expectations. Usually, in the semi-supervised clustering setting, the background knowledge is converted to some kind of constraint and, successively, metric learning or constrained clustering are adopted to obtain the final data partition. Conversely, we propose a new semi-supervised clustering algorithm that directly exploits prior knowledge, under the form of labeled examples, avoiding the necessity to derive constraints. Our algorithm employs a multiresolution strategy to generate an ensemble of semi-supervised autoencoders that fit the data together with the background knowledge. Successively, the network models are employed to supply a new embedding representation on which clustering is performed. The proposed strategy is evaluated on a set of real-world benchmarks also in comparison with well-known state-of-the-art semi-supervised clustering methods. The experimental results highlight the benefit of directly leveraging the prior knowledge and show the quality of the representation learnt by the multiresolution schema.

Many real world data can be represented by a network with a set of nodes linked each other by multiple relations. Such a rich graph is called multilayer graph. In this demo, we present a tool for Visual Querying of Large Multilayer Graphs that allows to visually draw the query, retrieve result patterns and finally navigate and browse the results considering the original multilayer graph database. Our approach does not only provide a graphical user interface for the graph engine but the query processing is fully integrated.

Mapping winter vegetation quality is a challenging problem in remote sensing. This is due to cloud coverage in winter periods, leading to a more intensive use of radar rather than optical images. The aim of this letter is to provide a better understanding of the capabilities of Sentinel-1 radar images for winter vegetation quality mapping through the use of deep learning techniques. Analysis is carried out on a multitemporal Sentinel-1 data over an area around Charentes-Maritimes, France. This data set was processed in order to produce an intensity radar data stack from October 2016 to February 2017. Two deep recurrent neural network (RNN)-based classifiers were employed. Our work revealed that the results of the proposed RNN models clearly outperformed classical machine learning approaches (support vector machine and random forest).

The problem of local community detection refers to the identification of a community starting from a query node and using limited information about the network structure. Existing methods for solving this problem however are not designed to deal with multilayer network models, which are becoming pervasive in many fields of science. In this work, we present the first method for local community detection in multilayer networks. Our method exploits both internal and external connectivity of the nodes in the community being constructed for a given seed, while accounting for different layer-specific topological information. Evaluation of the proposed method has been conducted on real-world multilayer networks.

Nowadays, modern earth observation programs produce huge volumes of satellite images time series that can be useful to monitor geographical areas through time. How to efficiently analyze such a kind of information is still an open question in the remote sensing field. Recently, deep learning methods proved suitable to deal with remote sensing data mainly for scene classification(i.e., convolutional neural networks on single images) while only very few studies exist involving temporal deep learning approaches [i.e., recurrent neural networks (RNNs)] to deal with remote sensing time series. In this letter, we evaluate the ability of RNNs, in particular, the long short-term memory (LSTM) model, to perform land cover classification considering multitemporal spatial data derived from a time series of satellite images. We carried out experiments on two different data sets considering both pixel-based and object-based classifications. The obtained results show that RNNs are competitive compared with the state-of-the-art classifiers, and may outperform classical approaches in the presence of low represented and/or highly mixed classes. We also show that the alternative feature representation generated by LSTM can improve the performances of standard classifiers.

In this paper, we introduce a new approach of semisupervised anomaly detection that deals with categorical data. Given a training set of instances (all belonging to the normal class), we analyze the relationship among features for the extraction of a discriminative characterization of the anomalous instances. Our key idea is to build a model that characterizes the features of the normal instances and then use a set of distance-based techniques for the discrimination between the normal and the anomalous instances. We compare our approach with the state-of-the-art methods for semisupervised anomaly detection. We empirically show that a specifically designed technique for the management of the categorical data outperforms the general-purpose approaches. We also show that, in contrast with other approaches that are opaque because their decision cannot be easily understood, our proposed approach produces a discriminative model that can be easily interpreted and used for the exploration of the data.

Many real world information can be represented by a graph with a set of nodes interconnected with each other by multiple type of relations called edge layers (e.g., social network, biological data). Edge bundling techniques have been proposed to solve cluttering issue for standard graphs while few efforts were done to deal with the similar issue for multilayer graphs. In multilayer graphs scenario, not only the clutter induced by large amount of edges is a problem but also the fact that different type of edges can overlap each other making useless the final visualization. In this paper we introduce a new multilayer graph edge bundling technique that firstly produces a preliminary edge bundling independently of the different edge layers and then deals with the specificity of multilayer graphs where more than one type of edges can be routed on the same bundle. The proposed visualization is tested on a real world case study and the outcomes point out the ability of our proposal to discover patterns present in the data.

RDF is a standard for the conceptual description of knowledge, and SPARQL is the query language conceived to query RDF data. The RDF data is cherished and exploited by various domains such as life sciences, Semantic Web, social network, etc. Further, its integration at Web-scale compels RDF management engines to deal with complex queries in terms of both size and structure. In this paper, we propose AMbER (Attributed Multigraph Based Engine for RDF querying), a novel RDF query engine specifically designed to optimize the computation of complex queries. AMbER leverages subgraph matching techniques and extends them to tackle the SPARQL query problem. First of all RDF data is represented as a multigraph, and then novel index- ing structures are established to efficiently access the in- formation from the multigraph. Finally a SPARQL query is represented as a multigraph, and the SPARQL querying problem is reduced to the subgraph homomorphism prob- lem. AMbER exploits structural properties of the query multigraph as well as the proposed indexes, in order to tackle the problem of subgraph homomorphism. The performance of AMbER, in comparison with state-of-the-art systems, has been extensively evaluated over several RDF benchmarks. The advantages of employing AMbER for complex SPARQL queries have been experimentally validated.

Multilingual document classification is often addressed by approaches that rely on language-specific resources (e.g., bilingual dictionaries and machine translation tools) to evaluate cross-lingual document similarities. However, the required transformations may alter the original document semantics, raising additional issues to the known difficulty of obtaining high-quality labeled datasets. To overcome such issues we propose a new framework for multilingual document classification under a transductive learning setting. We exploit a large-scale multilingual knowledge base, BabelNet, to support the modeling of different language-written documents into a common conceptual space, without requiring any language translation process. We resort to a state-of-the-art transductive learner to produce the document classification. Results on two real-world multilingual corpora have highlighted the effectiveness of the proposed document model w.r.t. document representations usually involved in multilingual and cross-lingual analysis, and the robustness of the transductive setting for multilingual document classification. These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Recent advances in network science allows the modeling and analysis of complex inter-related entities. These entities often interact with each other in a number of different ways. Simple graphs fail to capture these multiple types of relationships requiring more sophisticated mathematical structures. One such structure is multigraph, where entities (or nodes) can be linked to each other through multiple edges. In this paper we describe a new method to manage multiple types of relationships existing in multigraphs. Our approach is based on the concept of pair of nodes (edges) and, in particular, we study how nodes on different layers interact which each other considering the edges they share. We propose a two level strategy that summarizes global/local multigraph features. The global view helps us to gain knowledge related to the characteristics of layers and how they interact while the local view provides an analysis of individual layers highlighting edge properties such as cluster structure. Our proposal is complementary to standard node-link diagram and it can be coupled with such techniques in order to intelligently explore multigraphs. The proposed visualization is tested on a real world case study and the outcomes point out the ability of our proposal to discover patterns present in the data.

Gradual patterns highlight covariations of attributes of the form “The more/less X, the more/less Y”. Their usefulness in several applications has recently stimulated the synthesis of several algorithms for their automated discovery from large datasets. However, existing techniques require all the interesting data to be in a single database relation or table. This paper extends the notion of gradual pattern to the case in which the co-variations are possibly expressed between attributes of different database relations. The interestingness measure for this class of “relational gradual patterns” is defined on the basis of both Kendall's τ and gradual supports. Moreover, this paper proposes two algorithms, named τRGP Miner and gRGP Miner, for the discovery of relational gradual rules. Three pruning strategies to reduce the search space are proposed. The efficiency of the algorithms is empirically validated, and the usefulness of relational gradual patterns is proved on some real-world databases.

A major challenge in document clustering research arises from the growing amount of text data written in different languages. Previous approaches depend on language-specific solutions (e.g., bilingual dictionaries, sequential machine translation) to evaluate document similarities, and the required transformations may alter the original document semantics. To cope with this issue we propose a new document clustering approach for multilingual corpora that (i) exploits a large-scale multilingual knowledge base, (ii) takes advantage of the multi-topic nature of the text documents, and (iii) employs a tensor-based model to deal with high dimensionality and sparseness. Results have shown the significance of our approach and its better performance w.r.t. classic document clustering approaches, in both a balanced and an unbalanced corpus evaluation.

Due to their high degree of vegetation heterogeneity, fragmentation and biodiversity, Mediterranean natural habitats are difficult to assess and monitor with in-situ observations solely. Together with standardized ground plots and regular in-situ measurements, remote sensing is a powerful device that can contribute to a better understanding of the diversity of natural and semi-natural habitats and to monitor their phenology. In this paper, we implemented a systematic test of the suitability of multiseasonal remote sensing data for monitoring the phenological variations of natural habitats in a Mediterranean landscape. Six multispectral sensor signals were simulated for comparison based on their spectral response curves and in-situ averaged spectra collected at monthly intervals between February and October 2013 (IKONOS, Landsat 5 TM, Landsat 8, Pléiades, Sentinel-2, and Worldview-2). The simulations and comparisons performed in this study showed that Sentinel-2 sensor has the higher sensitivity to the variations in the coverage of photosynthetic vegetation, thus offering interesting perspectives for operational monitoring of natural habitats.

High repetitivity remote sensing could substantially improve natural habitats monitoring and mapping in the next years. However, dense time series of satellite images require new processing methodologies. In this paper we proposed an approach which combines Object Based Image Analysis (OBIA) and k-partite graphs for detecting spatiotemporal evolutions in a Mediterranean protected site composed of several types of natural and semi-natural habitats. The method was applied over a recent dataset (SPOT4 Take-5) specially conceived to simulate the acquisition frequency of the future Sentinel-2 satellites. The results indicate our method is capable to synthesize complex spatiotemporal evolutions in a semi-automatic way, therefore offering a new tool to analyze high repetitivity satellite time series.

One of the objectives of spatio-temporal data mining is to analyze moving object datasets to exploit interesting patterns. Traditionally, existing methods only focus on an unchanged group of moving objects during a time period. Thus, they cannot capture object moving trends which can be very useful for better understanding the natural moving behavior in various real world applications. In this paper, we present a novel concept of "time relaxed gradual trajectory pattern", denoted real-Gpattern, which captures the object movement tendency. Additionally, we also propose an efficient algorithm, called ClusterGrowth, designed to extract the complete set of all interesting maximal real-Gpatterns. Conducted experiments on real and large synthetic datasets demonstrate the effectiveness, parameter sensitiveness and efficiency of our methods.

Clustering data described by categorical attributes is a challenging task in data mining applications. Unlike numerical attributes, it is difficult to define a distance between pairs of values of a categorical attribute, since the values are not ordered. In this article, we propose a framework to learn a context-based distance for categorical attributes. The key intuition of this work is that the distance between two values of a categorical attribute A i can be determined by the way in which the values of the other attributes A j are distributed in the dataset objects: if they are similarly distributed in the groups of objects in correspondence of the distinct values of A i a low value of distance is obtained. We propose also a solution to the critical point of the choice of the attributes A j . We validate our approach by embedding our distance learning framework in a hierarchical clustering algorithm. We applied it on various real world and synthetic datasets, both low and high-dimensional. Experimental results show that our method is competitive with respect to the state of the art of categorical data clustering approaches. We also show that our approach is scalable and has a low impact on the overall computational time of a clustering task.

In this paper we propose and test the use of hierarchical clustering for feature selection. The clustering method is Ward's with a distance measure based on Goodman-Kruskal tau. We motivate the choice of this measure and compare it with other ones. Our hierarchical clustering is applied to over 40 data-sets from UCI archive. The proposed approach is interesting from many viewpoints. First, it produces the feature subsets dendrogram which serves as a valuable tool to study relevance relationships among features. Secondarily, the dendrogram is used in a feature selection algorithm to select the best features by a wrapper method. Experiments were run with three different families of classifiers: Naive Bayes, decision trees and k nearest neighbours. Our method allows all the three classifiers to generally outperform their corresponding ones without feature selection. We compare our feature selection with other state-of-the-art methods, obtaining on average a better classification accuracy, though obtaining a lower reduction in the number of features. Moreover, differently from other approaches for feature selection, our method does not require any parameter tuning.