We present TerraMind, the first any-to-any generative, multimodal foundation model for Earth observation (EO). Unlike other multimodal models, TerraMind is pretrained on dual-scale representations combining both token-level and pixel-level data across modalities. On a token level, TerraMind encodes high-level contextual information to learn cross-modal relationships, while on a pixel level, TerraMind leverages fine-grained representations to capture critical spatial nuances. We pretrained TerraMind on nine geospatial modalities of a global, large-scale dataset. In this paper, we demonstrate that (i) TerraMind's dual-scale early fusion approach unlocks a range of zero-shot and few-shot applications for Earth observation, (ii) TerraMind introduces "Thinking-in-Modalities" (TiM) -- the capability of generating additional artificial data during finetuning and inference to improve the model output -- and (iii) TerraMind achieves beyond state-of-the-art performance in community-standard benchmarks for EO like PANGAEA. The pretraining dataset, the model weights, and our code are open-sourced under a permissive license.

The increasing availability of high-resolution, open-access satellite data facilitates the production of global Land Cover (LC) maps, an essential source of information for managing and monitoring natural and human-induced processes. However, the accuracy of the obtained LC maps can be affected by the discrepancy between the spatial resolution of the satellite images and the extent of the LC present in the scene. Indeed, several pixels may be misclassified because of their mixed spectral signatures, i.e., more than two LC classes are present in the pixel. To solve this problem, this paper proposes an approach that explores the possibility of using simple but effective unmixing approaches to enhance the classification accuracy of the mixed spectral pixels. The results showed that several pixels, including buildings and grassland LC, are typically classified as cropland. By unmixing their spectral content, it is possible to extract the most prevalent class within the area of each pixel to update the classification map, thus sharply increasing the map accuracy. These promising preliminary results indicate the potential for broader applicability and efficiency in global LC mapping.

Seismic Imaging (SI) survey design for onshore applications faces challenges such as accessibility and poor data quality due to unexpected (near-)surface conditions. In this paper, we explore the correlation between the surface conditions provided by Land-Cover (LC) maps generated using Remote Sensing (RS) data and different settings of Seismic Processing (SP) parameters. The study involves a 2D seismic line related to geothermal exploration from the Netherlands.

The continuous stream of high spatial resolution satellite data offers the opportunity to regularly produce land cover (LC) maps. To this end, Transformer deep learning (DL) models have recently proven their effectiveness in accurately classifying long time series (TS) of satellite images. The continual generation of regularly updated LC maps can be used to analyze dynamic phenomena and extract multi-temporal information. However, several challenges need to be addressed. Our paper aims to study how the performance of a Transformer model changes when classifying TS of satellite images acquired in years later than those in the training set. In particular, the behavior of the attention in the Transformer model is analyzed to determine when the information provided by the initial training set needs to be updated to keep generating accurate LC products. Preliminary results show that: (i) the selection of the positional encoding strategy used in the Transformer has a significant impact on the classification accuracy obtained with multi-year TS, and (ii) the most affected classes are the seasonal ones.

Earth Observation (EO) data processing faces challenges due to large volumes, multiple sources, and diverse formats. To address this issue, this paper presents a scalable and parallelizable workflow using Apache Airflow, capable of integrating Machine Learning (ML) and Deep Learning (DL) models with Modular Supercomputing Architecture (MSA) systems. To test the workflow, we considered the production of large-scale Land-Cover (LC) maps as a case study. The workflow manager, Airflow, offers scalability, extensibility, and programmable task definition in Python. It allows us to execute different steps of the workflow in different High-Performance Computing (HPC) systems. The workflow is demonstrated on the Dynamical Exascale Entry Platform (DEEP) and Jülich Research on Exascale Cluster Architectures (JURECA) hosted at the Jülich Supercomputing Centre (JSC), a platform that incorporates heterogeneous JSC systems.

Land cover maps generated by the classification of remote sensing data allow for monitoring Earth processes and the dynamics of objects and phenomena. For accurate land cover variability quantification in environmental monitoring, maps need to be spatiotemporally consistent, continually updated, and indicate permanent changes. However, producing frequent and spatiotemporally consistent land cover maps is challenging because it involves balancing the need for temporal consistency with the risk of missing real changes. In this work, we propose a scalable and semi-automatic method for generating annual land cover maps with labels that are consistently applied from one year to the next. It uses a Transformer deep learning model as a classifier, which is trained on satellite time series of images using High Performance Computing (HPC). The trained model can generate stable maps by shifting the prediction window along the temporal direction. The effectiveness of the proposed approach is tested qualitatively and quantitatively on a multi-annual Sentinel-2 dataset acquired over a three-year period in a study area located in the southern Italian Alps.

Deep Learning models have proven necessary in dealing with the challenges posed by the continuous growth of data volume acquired from satellites and the increasing complexity of new Remote Sensing applications. To obtain the best performance from such models, it is necessary to fine-tune their hyperparameters. Since the models might have massive amounts of parameters that need to be tuned, this process requires many computational resources. In this work, a method to accelerate hyperparameter optimization on a High-Performance Computing system is proposed. The data batch size is increased during the training, leading to a more efficient execution on Graphics Processing Units. The experimental results confirm that this method reduces the runtime of the hyperparameter optimization step by a factor of 3 while achieving the same validation accuracy as a standard training procedure with a fixed batch size.

This paper presents an approach that aims to produce a Time-Series (TS) of consistent Land-Cover (LC) maps, typically needed to perform environmental monitoring. First, it creates an annual training set for each TS to be classified, leveraging on publicly available thematic products. These annual training sets are then used to generate a set of preliminary LC maps that allow for the identification of the unchanged areas, i.e., the stable temporal component. Such areas can be used to define an informative and reliable multi-year training set, by selecting samples belonging to the different years for all the classes. The multi-year training set is finally employed to train a unique multi-year Long Short Term Mem-ory (LSTM) model, which enhances the consistency of the annual LC maps. The preliminary results carried out on three TSs of Sentinel 2 images acquired in Italy in 2018,2019 and 2020 demonstrates the capability of the method to improve the consistency of the annual LC maps. The agreement of the obtained maps is$\approx 78{\%}$, compared to the$\approx 74{\%}$achieved by the LSTM models trained separately.

BigEarthNet is one of the standard large remote sensing datasets. It has been shown previously that neural networks are effective tools to classify the image patches in this data. However, finding the optimum network hyperparameters and architecture to accurately classify the image patches in BigEarthNet remains a challenge. Searching for more accurate models manually is extremely time consuming and labour intensive. Hence, a systematic approach is advisable. One possibility is automated evolutionary Neural Architecture Search (NAS). With this NAS many of the commonly used network hyperparameters, such as loss functions, are eliminated and a more accurate network is determined.

A wide variety of Remote Sensing (RS) missions are continuously acquiring a large volume of data every day. The availability of large datasets has propelled Deep Learning (DL) methods also in the RS domain. Convolutional Neural Networks (CNNs) have become the state of the art when tackling the classification of images, however the process of training is time consuming. In this work we exploit the Layer-wise Adaptive Moments optimizer for Batch training (LAMB) optimizer to use large batch size training on High-Performance Computing (HPC) systems. With the use of LAMB combined with learning rate scheduling and warm-up strategies, the experimental results on RS data classification demonstrate that a ResNet50 can be trained faster with batch sizes up to 32K.

Similarly to other scientific domains, Deep Learning (DL) holds great promises to fulfil the challenging needs of Remote Sensing (RS) applications. However, the increase in volume, variety and complexity of acquisitions that are carried out on a daily basis by Earth Observation (EO) missions generates new processing and storage challenges within operational processing pipelines. The aim of this work is to show that High-Performance Computing (HPC) systems can speed up the training time of Convolutional Neural Networks (CNNs). Particular attention is put on the monitoring of the classification accuracy that usually degrades when using large batch sizes. The experimental results of this work show that the training of the model scales up to a batch size of 8,000, obtaining classification performances in terms of accuracy in line with those using smaller batch sizes.