As the impact of modern human activities on ecosystems intensifies, it becomes increasingly important to accurately assess this influence. Earth Observation, particularly through satellite imagery, serves as a key environmental conservation tool, providing a comprehensive overhead perspective for monitoring our planet’s ecosystems. This study formulates a pixel-wise regression task, guided by a novel set of naturalness annotations that quantify the modern human pressure on a landscape at the pixel level. We introduce a tailored framework that integrates geographical and contextual priors. These priors are represented by coordinate information and broader contextual information surrounding the immediate patch. Our approach improves the deep neural network’s capability to estimate naturalness from satellite imagery, enabling a deeper comprehension that promotes the safeguarding of our natural habitats.

Advancements in Earth observation (EO) have led to an increase in the volume of and easier access to multimodal geospatial data, making environmental monitoring and analysis more accessible. However, understanding the influence of each input modality on decision-making within deep learning models remains an open challenge. This letter proposes a deep occlusion framework to enhance the interpretability of a multimodal model for land naturalness assessment, using a supervised pixelwise regression task for naturalness mapping with the input modalities Sentinel-2 and Sentinel-1 imagery, land cover maps, and nighttime lights intensity data. The proposed framework systematically occludes individual input modalities to create modality-level influence scores. Influence scores are attributed to input modalities by measuring the distance between the embedding of the nonoccluded input and the embedding of the input with a single modality occluded, revealing how each modality influences predictions and clarifying their contributions (and, thus, importance) in the model’s decision-making process. The results provide further insights into how input modalities influence the model’s decision-making at both the sample level, enabling regional case studies, and the dataset level, allowing for data pruning and improving training and inference times. The code is available athttps://github.com/burakekim/embedding_occlusion.

In order to gain a better understanding of disturbances (i.e., anthropogenic pressure) in our environment, researchers have worked on methods for the mapping of wilderness areas given their crucial role in providing native habitat for many species, which are often endangered. In this work, we formulate the wilderness mapping task as a supervised learning problem. We focus on the joint use of potentially complementary features provided by multi-modal input data. Until now, the individual influences of different input modalities on the decision of a deep neural network have largely remained unclear. Therefore, we develop a framework for the modality-level interpretation of multi-modal Earth observation data in an end-to-end fashion. While leveraging an explainable machine learning method, namely Occlusion Sensitivity Maps, the proposed framework investigates the influence of modalities in an early-fusion setting, i.e. the modalities are fused before the learning process. With respect to the application, our results indicate that auxiliary data such as land cover and nighttime light data are important sources for the accurate classification of wilderness areas and the influence of a modality increases with the increasing number of spectral channels.

This paper introduces the MapInWild dataset, a multi-modal dataset tailored to mapping wilderness areas from satellite imagery and auxiliary geodata. MapInWild accommodates freely and globally available geodata layers that emerged from various remote sensing sensors, such as dualpol Sentinel-1 imagery, multi-spectral Sentinel-2 data, Visible Infrared Imaging Radiometer Suite night-time light data, and the ESA WorldCover map. Each sample of the Map-InWild dataset is annotated with labels derived from the World Database on Protected Areas, a most up-to-date and comprehensive global database on conservation areas. Protected areas are filtered through a sophisticated sampling process to ensure a representative coverage of the natural areas of the Earth. With MapInWild dataset, we hope to foster further research on deep learning applied to environmental remote sensing and conservation. MapInWild dataset is publicly available at https://dataverse.harvard.edu/dataverse/mapinwild.

The task of building footprint segmentation has been well-studied in the context of remote sensing (RS) as it provides valuable information in many aspects, however, difficulties brought by the nature of RS images such as variations in the spatial arrangements and in-consistent constructional patterns require studying further, since it often causes poorly classified segmentation maps. We address this need by designing a joint optimization scheme for the task of building footprint delineation and introducing two auxiliary tasks; image reconstruction and building footprint boundary segmentation with the intent to reveal the common underlying structure to advance the classification accuracy of a single task model under the favor of auxiliary tasks. In particular, we propose a deep multi-task learning (MTL) based unified fully convolutional framework which operates in an end-to-end manner by making use of joint loss function with learnable loss weights considering the homoscedastic uncertainty of each task loss. Experimental results conducted on the SpaceNet6 dataset demonstrate the potential of the proposed MTL framework as it improves the classification accuracy greatly compared to single-task and lesser compounded tasks.