In supervised learning, deep learning models demand a large corpus of annotated data for object detection and classification tasks. This constrains their utility in humanitarian emergency response. To overcome this problem, we have proposed an unsupervised dwelling counting from very high-resolution satellite imagery by combining a Variational Autoencoder(VAE) with an anomaly detection approach. When VAEs are applied in earth observation for dwelling localization and counting, we observed two critical limitations (1) the balance between reconstruction and good latent code, where in-favour of good reconstruction of dwellings leads to weak anomaly score maps that fail to properly localize dwellings (2) limited spatiotemporal invariance of the learned latent code. When the model is trained with datasets obtained from different geography and time, it fails to properly localize dwellings. For the first problem, we introduced self-supervision by creating synthetic anomalies. For the second problem, we introduced latent space conditioning. The approach is tested on 9 very high-resolution images obtained from six Forcibly Displaced People settlement areas. Results indicate that combining VAE with an anomaly detection approach has reached an AUC value ranging from 0.70 at complex settlements towards 0.98 at relatively less complex settlement areas. Similarly, an MAE value of 56.67 towards 5.03 is achieved for dwelling counting. Joint training of combined datasets with latent space conditioning and self-supervision enabled the achievement of results better than classical VAE, with improved spatiotemporal transferability of the model with more crisp and strong anomaly maps. Overall implementation code will be available at https://github.com/getch-geohum/SSL-VAE.

This research introduces a new method for predicting changes in corn yield and price, critical for food security. Instead of relying on difficult-to-access pre-harvest production data, our approach uses satellite-derived gross primary production (GPP) data and dimension-reduction techniques to forecast national corn yield and price changes. We conducted case studies in the U.S., Malawi, and South Africa to validate this approach, analyzing predictors from annual GPP variations during peak growing seasons. We applied dimension-reduction strategies, such as spatial averaging and empirical orthogonal functions (EOFs). Additionally, we used deep learning techniques such as autoencoders (AEs) and variational autoencoders (VAEs) to extract meaningful features from the high-dimensional GPP datasets. These features were then used as predictors in yield and price classification models based on generalized linear models (GLMs) and ElasticNet. We also considered a neural network model trained to predict yield and price variations from GPP input data directly. We evaluated the model performances using metrics such as area under the curve (AUC), brier skill score (BSS), and Matthew’s correlation coefficient (MCC). Our results indicate that dimension-reduction techniques based on AEs and VAEs provided better predictive capabilities across all three countries compared to EOF, particularly in Malawi, where the BSS increased from 0.26 with EOF to 0.81 with AE for yield and from −0.004 with EOF to 0.77 with VAE for price. This study demonstrates that integrating open-access satellite data with dimension-reduction techniques can significantly improve crop forecast accuracy, providing an accessible tool to enhance agricultural management and food security.

We propose a new model of Variational Autoencoder (VAE) for Anomaly Detection (AD) with improved modeling power. More precisely, we introduce a VAE model with a Gaussian Random Field (GRF) prior, namely VAE-GRF, which generalizes the classical VAE model. We show that, under some assumptions, the VAE-GRF largely outperforms the traditional VAE and some other probabilistic models developed for AD. Our experimental results suggest that the VAE-GRF could be used as a relevant VAE baseline in place of the traditional VAE with very limited additional computational cost. We provide competitive results on the public MVTec benchmark dataset for visual inspection, as well as on the public Livestock dataset dedicated to the task of unsupervised animal detection from aerial images.

Even though computer vision models are excellent for automatic scene segmentation and object identification from remotely sensed imagery, they demand a huge corpus of annotated data for the training and validation which is a huge challenge in humanitarian emergency response. To tackle this problem, we propose unsupervised dwelling object counting combining Variational Autoencoder (VAE) with an anomaly detection approach. The approach is tested in six Forcibly Displaced People (FDP) settlement areas situated in different parts of the world. Using an anomaly map computed with the VAE model, we demonstrated the possibility of properly locating dwelling objects using anomaly maps. Dwelling counts are obtained by further segmenting anomaly maps. Results show that, though it has strong spatio-temporal variation, the VAE model exhibits promising potential for locating and counting dwellings. It is also observed that in FDP settlements with dense buildings and extremely low contrast between buildings and ground or environment, the performance is relatively lower than the performance achieved in settlement areas with regularly spaced and less complex building structures.

This paper studies a reconstruction-based approach for weakly-supervised animal detection from aerial images in marine environments. Such an approach leverages an anomaly detection framework that computes metrics directly on the input space, enhancing interpretability and anomaly localization compared to feature embedding methods. Building upon the success of Vector-Quantized Variational Autoencoders in anomaly detection on computer vision datasets, we adapt them to the marine animal detection domain and address the challenge of handling noisy data. To evaluate our approach, we compare it with existing methods in the context of marine animal detection from aerial image data. Experiments conducted on two dedicated datasets demonstrate the superior performance of the proposed method over recent studies in the literature. Our framework offers improved interpretability and localization of anomalies, providing valuable insights for monitoring marine ecosystems and mitigating the impact of human activities on marine animals.

Anthropogenic activities pose threats to wildlife and marine fauna, prompting the need for efficient animal counting methods. This research study utilizes deep learning techniques to automate counting tasks. Inspired by previous studies on crowd and animal counting, a UNet model with various backbones is implemented, which uses Gaussian density maps for training, bypassing the need of training a detector. The new model is applied to the task of counting dolphins and elephants in aerial images. Quantitative evaluation shows promising results, with the EfficientNet-B5 backbone achieving the best performance for African elephants and the ResNet18 backbone for dolphins. The model accurately locates animals despite complex image background conditions. By leveraging artificial intelligence, this research contributes to wildlife conservation efforts and enhances coexistence between humans and wildlife through efficient object counting without detection from aerial remote sensing.

Anomaly Detection (AD) is an important research topic, with very diverse applications such as industrial defect detection, medical diagnosis, fraud detection, intrusion detection, etc. Within the last few years, deep learning-based methods have become the standard approach for AD. In many practical cases, the anomalies are unknown in advance. Therefore, most of challenging AD problems need to be addressed in an unsupervised or weakly supervised framework. In this context, deep generative models are widely used, in particular Variational Autoencoder (VAE) models. VAEs have been extended to Vector-Quantized VAEs (VQ-VAEs), a model increasingly popular because of its versatility enabled by the discrete latent space. We present for the first time a robust approach which takes advantage of the inner metrics of VQ-VAEs for AD. We show that the distance between the output of the encoder and the codebook vectors of a VQ-VAE provides a valuable information which can be used to localize the anomalies. In our approach, this metric complements a reconstruction-based metric to improve AD results. We compare our model with state-of-the-art AD models on three standards datasets, including the MVTec, UCSD-Ped1 and CIFAR-10 datasets. Experiments show that the proposed method yields high competitive results.

Hidden Markov Trees (HMTs) are successful probabilistic models [1] [2] [3] in image segmentation or genetic analysis for example. They offer a good compromise between the random variables that can be modeled and the tractability of the inference within these models. Indeed, the inference procedures can be conducted in a direct, exact fashion [4]. However the conditional independence restrictions in HMTs can be too simplistic with respect to the interactions we wish to model, particularly in the task of natural image segmentation. In this article we study an extension of HMTs called Spatial Triplet Markov Trees (STMTs) which is designed to greatly increase the correlations of the random variables while keeping the possibility of direct and exact inference procedures. The STMT model is tested and compared to other unsupervised segmentation techniques in the case of unsupervised image segmentation. Numerical experiments show that STMTs outperform HMTs and present rich spatial dependencies which are crucial for image segmentation.