Automated analysis of vast Earth observation data via interactive Vision-Language Models (VLMs) can unlock new opportunities for environmental monitoring, disaster response, and resource management. Existing generic VLMs do not perform well on Remote Sensing data, while the recent Geo-spatial VLMs remain restricted to a fixed resolution and few sensor modalities. In this paper, we introduce EarthDial, a conversational assistant specifically designed for Earth Observation (EO) data, transforming complex, multi-sensory Earth observations into interactive, natural language dialogues. EarthDial supports multi- spectral, multi-temporal, and multi-resolution imagery, enabling a wide range of remote sensing tasks, including classification, detection, captioning, question answering, visual reasoning, and visual grounding. To achieve this, we introduce an extensive instruction tuning dataset comprising over 11.11M instruction pairs covering RGB, Synthetic Aperture Radar (SAR), and multispectral modalities such as Near-Infrared (NIR) and infrared. Furthermore, EarthDial handles bi-temporal and multi-temporal sequence analysis for applications like change detection. Our extensive experimental results on 44 downstream datasets demonstrate that EarthDial outperforms existing generic and domain-specific models, achieving better generalization across various EO tasks. Our source codes and pre-trained models are at https://github.com/hiyamdebary/EarthDial.

While numerous recent benchmarks focus on evaluating generic Vision-Language Models (VLMs), they do not effectively address the specific challenges of geospatial applications. Generic VLM benchmarks are not designed to handle the complexities of geospatial data, an essential component for applications such as environmental monitoring, urban planning, and disaster management. Key challenges in the geospatial domain include temporal change detection, large-scale object counting, tiny object detection, and understanding relationships between entities in remote sensing imagery. To bridge this gap, we present GEOBench-VLM, a comprehensive benchmark specifically designed to evaluate VLMs on geospatial tasks, including scene understanding, object counting, localization, fine-grained categorization, segmentation, and temporal analysis. Our benchmark features over 10,000 manually verified instructions and spanning diverse visual conditions, object types, and scales. We evaluate several state-of-the-art VLMs to assess performance on geospatial-specific challenges. The results indicate that although existing VLMs demonstrate potential, they face challenges when dealing with geospatial-specific tasks, highlighting the room for further improvements. Notably, the best-performing LLaVa-OneVision achieves only 41.7% accuracy on MCQs, slightly more than GPT-4o, which is approximately double the random guess performance. Our benchmark is publicly available at https://github.com/The-AI-Alliance/GEO-Bench-VLM .

We present TerraMind, the first any-to-any generative, multimodal foundation model for Earth observation (EO). Unlike other multimodal models, TerraMind is pretrained on dual-scale representations combining both token-level and pixel-level data across modalities. On a token level, TerraMind encodes high-level contextual information to learn cross-modal relationships, while on a pixel level, TerraMind leverages fine-grained representations to capture critical spatial nuances. We pretrained TerraMind on nine geospatial modalities of a global, large-scale dataset. In this paper, we demonstrate that (i) TerraMind's dual-scale early fusion approach unlocks a range of zero-shot and few-shot applications for Earth observation, (ii) TerraMind introduces "Thinking-in-Modalities" (TiM) -- the capability of generating additional artificial data during finetuning and inference to improve the model output -- and (iii) TerraMind achieves beyond state-of-the-art performance in community-standard benchmarks for EO like PANGAEA. The pretraining dataset, the model weights, and our code are open-sourced under a permissive license.

Floods cause loss of lives and multi-billion dollar damages every year. When these events strike, automated tools using remote sensing for quick mapping of the affected areas are critical for planning rescue activities and assessing impact. The most recent techniques to map floods are based on semantic segmentation and deep learning, which require large datasets and ground truth for training, that are difficult to get. To overcome this challenge, we propose an effective method for synthesizing patches of synthetic aperture radar images containing open-land flooded areas. With bi-temporal image acquisitions, we replace portions of land areas in the second acquisition with water pixels borrowed from permanent water bodies. Spatial patterns derived from elevation data help guide the process. With this approach, we build a large dataset with pre and post-event Sentinel-1-based VH-polarized radar intensity images to train deep neural networks to map real-life floods. In a case study, we employ an established U-Net architecture and show that a model version with less than 2% of the number of original parameters achieves almost identical flood detection accuracy. This allows for faster processing and is a clear advantage from an operational perspective. For comparison, we provide empirical segmentation results of four flood cases. F1 scores agree between 0.80 and 0.90 when compared to reference flood maps from the Copernicus service. This confirms that the approach is valuable for mapping areas affected by floods, e.g. during or immediately after catastrophic events, even in areas that were not included during the model training.

As climate change is increasing the frequency and intensity of climate and weather hazards, improving detection and monitoring of flood events is a priority. Being weather independent and high resolution, Sentinel 1 (S1) radar satellite imagery data has become the go to data source to detect flood events accurately. However, current methods are either based on fixed thresholds to differentiate water from land or train Artificial Intelligence (AI) models based on only S1 data, despite the availability of many other relevant data sources publicly. These models also lack comprehensive validations on out-of-sample data and deployment at scale. In this study, we investigated whether adding extra input layers could increase the performance of AI models in detecting floods from S1 data. We also provide performance across a range of 11 historical events, with results ranging between 0.93 and 0.97 accuracy, 0.53 and 0.81 IoU, and 0.68 and 0.89 F1 scores. Finally, we show the infrastructure we developed to deploy our AI models at scale to satisfy a range of use cases and user requests.

Floods are among the most costly weather hazards for societies and businesses globally. With increasing global warming, these events have become even more frequent and more devastating. Thus, accurate flood mapping has become critical for disaster relief, risk management and mitigation. Current flood segmentation methods use either threshold-based approaches or deep-learning schemes, e.g. using the U-Net architecture, to differentiate between water-covered bodies or dry land on Earth observation images. Many schemes are exploiting imagery from synthetic aperture radar (e.g. Sentinel 1 satellites) or visual bands of satellites such as the Sentinel 2, but often restrict themselves to using one or very few modalities, i.e. spectral wavelengths, despite the availability of many more wavelengths or pre-processed indices with potential value to the challenge. In support of operationalizing flood segmentation on a global scale using deep learning, we propose semantic flood segmentation exploiting optionally many different modalities (i.e. multimodal flood segmentation), making the approach largely immune to geographic differences across the globe. Using U-Net at the core of our work, we observe very good generalisation of our segmentation model to unseen flood events in our holdout set at the level of 0.95 F1 Score (0.92 IoU) for both no water and water class, and 0.53 F1 Score (0.43 IoU) for water class, respectively.

When providing the boundary conditions for hydrological flood models and estimating the associated risk, interpolating precipitation at very high temporal resolutions (e.g. 5 minutes) is essential not to miss the cause of flooding in local regions. In this paper, we study optical flow-based interpolation of globally available weather radar images from satellites. The proposed approach uses deep neural networks for the interpolation of multiple video frames, while terrain information is combined with temporarily coarse-grained precipitation radar observation as inputs for self-supervised training. An experiment with the Meteonet radar precipitation dataset for the flood risk simulation in Aude, a department in Southern France (2018), demonstrated the advantage of the proposed method over a linear interpolation baseline, with up to 20% error reduction.

Surface water mapping in satellite images enables flood monitoring, a task with increasing importance under changing climate conditions. Current segmentation methods based on Deep Learning require large, curated datasets for training’ which are difficult to obtain. In this paper, we present a probabilistic approach to water segmentation based on established computer vision methods that requires little training and is easy to interpret. We use prior knowledge of typical backscatter intensity to locate seed pixels likely to be in water and land. A preliminary rough segmentation using thresholding guides the selection of two image patches that will be fully labeled using seeded region growing segmentation. Then, we sample small patches within the automatically labeled regions to train a fully connected neural network that, running in sliding windows, scores for the presence of water in the entire image. The approach is tested for mapping surface water during a large flood event in Aude, France. Outputs of the proposed approach are compared to the reference flood delineation map provided online by the Copernicus service. A F1 score of 0.67 suggests that performance of the proposed approach is similar or better than classic thresholding methods used as benchmark.

OBJECTIVE: The study sought to explore to what extent geolocation data has been used to study serious mental illness (SMI). SMIs such as bipolar disorder and schizophrenia are characterized by fluctuating symptoms and sudden relapse. Currently, monitoring of people with an SMI is largely done through face-to-face visits. Smartphone-based geolocation sensors create opportunities for continuous monitoring and early intervention. MATERIALS AND METHODS: We searched MEDLINE, PsycINFO, and Scopus by combining terms related to geolocation and smartphones with SMI concepts. Study selection and data extraction were done in duplicate. RESULTS: Eighteen publications describing 16 studies were included in our review. Eleven studies focused on bipolar disorder. Common geolocation-derived digital biomarkers were number of locations visited (n = 8), distance traveled (n = 8), time spent at prespecified locations (n = 7), and number of changes in GSM (Global System for Mobile communications) cell (n = 4). Twelve of 14 publications evaluating clinical aspects found an association between geolocation-derived digital biomarker and SMI concepts, especially mood. Geolocation-derived digital biomarkers were more strongly associated with SMI concepts than other information (eg, accelerometer data, smartphone activity, self-reported symptoms). However, small sample sizes and short follow-up warrant cautious interpretation of these findings: of all included studies, 7 had a sample of fewer than 10 patients and 11 had a duration shorter than 12 weeks. CONCLUSIONS: The growing body of evidence for the association between SMI concepts and geolocation-derived digital biomarkers shows potential for this instrument to be used for continuous monitoring of patients in their everyday lives, but there is a need for larger studies with longer follow-up times.

Risk of psychotic relapse in schizophrenic patients is commonly measured by social functioning (SF), which focuses on patients' daily activities. Monitoring of SF usually relies on infrequent clinic visits, limiting the capacity to detect sudden changes. GPS data that is passively collected with smartphones introduce new opportunities to monitor SF. We conducted a five-day pilot study with five schizophrenic patients to assess the feasibility of this approach. Participants used a smartphone to continuously record their GPS location, and completed a paper-based SF diary to register out-of-home activities. We implemented a time-based method and a density-based method to identify the geolocations visited and then we clustered geolocations visited in places visited. Finally, we used semantic enrichment to classify places types and associated activities. We evaluated the performance of the two approaches by comparing the activities detected from the GPS data with those recorded in the SF diary. Recall was better for the density-based method, ranging from 0.686 (Standard Deviation [SD] 0.168) to 0.771 (SD 0.264) while precision was better for the time-based method (0.722 (SD 0.197) to 0.954 (SD 0.093)). To conclude, using routinely collected GPS data and relatively simple analytical methods we detected patients' out-of-home activities with moderate recall, more sophisticated analytical methods may obtain better performance.