While the pretraining of Foundation Models (FMs) for remote sensing (RS) imagery is on the rise, models remain restricted to a few hundred million parameters. Scaling models to billions of parameters has been shown to yield unprecedented benefits including emergent abilities, but requires data scaling and computing resources typically not available outside industry R&D labs. In this work, we pair high-performance computing resources including Frontier supercomputer, America's first exascale system, and high-resolution optical RS data to pretrain billion-scale FMs. Our study assesses performance of different pretrained variants of vision Transformers across image classification, semantic segmentation and object detection benchmarks, which highlight the importance of data scaling for effective model scaling. Moreover, we discuss construction of a novel TIU pretraining dataset, model initialization, with data and pretrained models intended for public release. By discussing technical challenges and details often lacking in the related literature, this work is intended to offer best practices to the geospatial community toward efficient training and benchmarking of larger FMs.

The design of a pretraining dataset is emerging as a critical component for the generality of foundation models. In the remote sensing realm, large volumes of imagery and benchmark datasets exist that can be leveraged to pretrain foundation models, however using this imagery in absence of a well-crafted sampling strategy is inefficient and has the potential to create biased and less generalizable models. Here, we provide a discussion and vision for the curation and assessment of pretraining datasets for remote sensing geospatial foundation models. We highlight the importance of geographic, temporal, and image acquisition diversity and review possible strategies to enable such diversity at global scale. In addition to these characteristics, support for various spatial-temporal pretext tasks within the dataset is also critical. Ultimately, our primary objective is to place emphasis on and draw attention to the data curation stage of the foundation model development pipeline. By doing so, we think it is possible to reduce biases of geospatial foundation models, as well as enable broader generalization to downstream remote sensing tasks and applications.

Population growth is increasingly happening in slum settlements of the large urban centers in the Global South. The term "slum" encompasses a wide range of communities, located mostly in underserved areas, and often exhibiting distinct structural and functional informalities with a relatively high concentration of marginalized populations. To address the issues confronting slums for effective planning and development, including the realistic estimation of the resident population, identifying them accurately is fundamental. Given the disagreements over a universal definition, diverse characteristic features, and socio-political limitations, global detection of slums is a veritable challenge. In this paper, we present experiments in slum detection using a scene classification algorithm and 3-meter spatial resolution satellite imagery. We train and evaluate the model for slum detection in Mumbai, India for the year 2023 and test the temporal generalization of the trained model on Mumbai in 2020 and 2018. In addition, we explore the pathways toward geographic generalization to Kolkata and Delhi (India). We discuss several limitations in the workflow and model, situate our findings in the existing literature, and suggest improvements and alternatives. With this, we establish baseline methods and experiments as a first step towards developing an image-based global slum detection framework and algorithm. This work adds to the community discussion on methods, data challenges, and open questions related to the detection of slums globally. With this research, we hope to improve our understanding of human settlements, especially in critical areas, improve population estimates, and help measure progress towards the sustainable development goals.

Rapid building damage assessment (BDA) is vital in guiding disaster response missions and estimating population distribution across impacted areas. While commercial satellite imagery providers have enabled near-daily monitoring of the Earth, near-realtime assessment of disaster scenarios frequently requires analysis of off-nadir imagery, as satellites are often far from impacted areas for at-nadir post-event imaging to occur Such scenarios are, however, underrepresented in existing BDA datasets and methodologies. With this motivation, we investigate generalization capabilities of current BDA practices across overhead view-angles and strategies for their improvement. Using a labeled dataset of images capturing conflict-related damages, we first train a baseline BDA architecture using imbalanced and balanced datasets with respect to view-angle. Then, we explore conditional convolutions parameterized on image features, image nadir, and their combination as a mechanism for conditioning on view-angles. Experiments demonstrate the limitations of current practice and the potential of conditional mechanisms to increase model robustness to view-angle variations.

Computer vision algorithms are increasingly leveraged to accelerate geospatial analysis for disaster response and recovery. As the diversity of remote sensing imagery grows with optical, SAR, and other modalities, a perquisite for analytics is cross-modal image registration. There is a high potential to harness computer vision for this pre-processing requirement toward enabling downstream analytics such as heterogeneous change detection, automated feature extraction, and data fusion. Advancement in these areas has the potential to simplify data wrangling tasks and further accelerate disaster response timelines. The SpaceNet 9 challenge (launching in mid-2024) focuses on addressing the cross-modal image registration problem and demonstrating the utility of such modules on earthquake impacted scenarios. This paper describes the motivation for the SpaceNet 9 and provides a first overview of the dataset, the baseline algorithm, and implications for seeking cross-modal image registration in Earth observation. Code is available at https://github.com/SpaceNetChallenge/SpaceNet9.

Machine learning models have been extensively used for analyzing Earth Observation images and have played a crucial role in advancing the field. While most studies focus on improving the model’s performance, some aim to understand the model’s output. These explainable approaches provide reasoning behind the model’s output, establishing trust and confidence in the results. However, the evaluation of these models’ performance is mainly based on accuracy. To enhance the fairness and transparency of machine learning models, the evaluation of these models on Earth Observation images should also focus on explainability. This work reflects on existing research on explaninable AI in Remote Sensing and further outlines the desirable properties of the gold standard metric for evaluating explainable machine learning models on EO images.

In recent years, deep learning has become an increasingly popular alternative for modeling in geoscience applications due to its scalability and efficiency. However, the interpretability, compute, data volume, and hyperparameter tuning requirements of deep learning models make development and monitoring difficult. Furthermore, model explainability and communicating results obtained by these models to users or domain experts is a challenge, as domain experts in geoscience also need to have a deep understanding of how those models function in order to support their scientific works. Here, we describe a science gateway and machine learning pipeline for predicting gravity anomalies from geophysical data. The gateway, built on open-source technologies, provides a holistic view of the pipeline through interactive visualizations aimed at enabling efficient exploratory data analysis. The repeatability, reproducibility, and monitoring capabilities of this overall system allow us to iterate and analyze at scale. Using this pipeline and gateway, we can repeatedly produce accurate high-resolution gravity anomaly datasets. By describing the underlying technologies, implementation, and results, we provide a foundation for the broader adoption of science gateways into cross-cutting geoscience and machine learning research projects as a means to improve the scientific discovery and collaboration in the geophysics and computational sciences community.

Archives of remote sensing (RS) data are increasing swiftly as new sensing modalities with enhanced spatiotemporal resolution become operational. While promising new breakthroughs, the sheer volume of RS archives stretches the limits of human analysts and existing AI tools, as most models are: i) limited to single data modalities; ii) task-specific; iii) heavily reliant on labeled data. The emerging Foundation Models (FMs) have the potential to address these limitations. Trained on vast unlabeled datasets through self-supervised learning, FMs enable generic feature extraction that facilitate specialization to a wide variety of downstream tasks. This paper describes a vision towards an FM for multimodal Earth Observation data (FM4EO), discussing key building blocks and open challenges. We put particular emphasis on multimodal reasoning, a topic underexplored in EO. Our ultimate goal is a practical path toward FM4EO with capacity to unlock breakthroughs in few-shot learning scenarios, multimodal geographic knowledge integration, synthesis, and hypothesis generation.

The development of algorithms to assess the effects of natural disasters plays an integral role in response efforts. There is a growing opportunity to leverage remote sensing data and computer vision to quickly analyze the scale of damage and organize a humanitarian response when extreme weather events occur. By automating the process of identifying damage to roads and infrastructure, we can significantly reduce response time, directing relief efforts on a time scale of minutes or hours rather than days. The SpaceNet 8 challenge featured a complex multi-class segmentation problem in the context of flood detection from remote sensing imagery. Competitors were tasked with leveraging both pre- and post-flooding event imagery to detect buildings and roads, as well as identify which of these object instances were affected by the flooding event. We examine the outcome of the SpaceNet 8 challenge and present an overview of the competition and a deeper look at the top-performing submissions.

With artificial intelligence breakthroughs permeating the Earth Science domain, there is an immediate need to advance the data, tools, and resulting technologies to broader societal challenges. Different efforts are emerging with fragmented best practices for making Earth Observation (EO) data Artificial Intelligence (AI)-ready, availing computer vision and image analysis tools for broader reuse across the remote sensing community. This paper will revisit current best practices and outline a guideline for advancing EO data and derivative AI products for broader community use. We mainly discuss the Analysis Ready Data (ARD) essentials and aim to forge their evolution with Findable, Accessible, Interoperable, Reusable (FAIR) principles to support cross-modal/cross-sensor/cross-provider opportunities that appear to be central to solving complex EO challenges.

Given the tremendous volume of accessible Earth Observation (EO) data, there is a need to develop scalable Geospatial Artificial Intelligence (GeoAI) solutions for time-sensitive applications. Scalability in this context refers to rapidly processing large-scale EO data using high performance computing resources. Accurate mapping of the built environment from remote sensing (RS) imagery has been one of the crucial components in GeoAI workflows for a wide spectrum of humanitarian applications. Derived vector data of built environment is often leveraged for disaster preparedness and response activities. However, factors such as differences in ortho-rectification, atmospheric conditions and human error, results in spatial misalignment between vector data and the timely available RS imagery. Model training for downstream tasks such as object detection, change analysis, etc., is negatively impacted due to such spatial misalignment. Although there has been progress towards automatic alignment of vector data, the lack of scalability remains an open research challenge. This paper proposes to leverage parallel computing to optimize an automatic vector data alignment workflow. It further employs CPU-level multi-core parallelism for improving the performance of the workflow for scalable built environment mapping. We report observations and discuss findings from the preliminary experiments performed on the Summit Supercomputer.

Deep learning has proven its effectiveness in numerous tasks for remote sensing scene understanding. However there is an increasing interest to explore fusion of domain-specific background information to the deep neural network to further improve its performance. Remote sensing researchers are also working towards developing models that generalize and adapt to multiple applications. Generalization challenges coupled with the scarcity of large corpora of high-quality noise-free labelled data, have together fueled an interest for leveraging background information. Knowledge graphs serve as excellent choice to represent domain-specific information in a structured, standardized and extensible manner. Integrating symbolic knowledge representations in the form of Knowledge Graph Embedding (KGE) to perform neuro-symbolic reasoning is an emerging research direction promising significant impacts. This vision paper seeks to position ideas and provoke early thoughts toward advancing neuro-symbolic artificial intelligence in the context of geospatial challenges. Specifically, it conceptualizes and elaborates on an architecture for infusing geospatial knowledge from knowledge graph in a deep neural network pipeline. As guiding case studies - land-use land-cover classification, object detection and instance segmentation can benefit from infusing spatio-contextual information with remote sensing imagery. The discussion further reflects on and articulates the challenges and explainable AI opportunities anticipated when scaling and maintaining large-scale geospatial knowledge graphs.

Understanding population at risks has been a focus of the LandScan program through its development of population estimates. With advancements in computer vision, deep learning technologies and access to High Performance Computing (HPC) and high resolution imagery, population estimates are now modeled at the building level. However, when those patterns are disrupted, rapid updates to population distribution estimates are needed to support humanitarian aid and response. Oak Ridge National Laboratory (ORNL) recently adapted an existing deep learning building footprint extraction model in development of a scalable approach to Building Damage Assessments (BDA). This new opportunity opens the possibility of automating BDA to support rapid population distribution estimate updates for geographic areas involved in geopolitical conflicts or natural events for humanitarian aid and response or where to focus recovery efforts. In addition, incorporate social surveys to further model human behavior under conflict or other scenarios that disrupt normal patterns of life.

As in many other research domains, Artificial Intelligence (AI) techniques have been increasing their footprint in Earth Sciences to extract meaningful information from the large amount of high-detailed data available from multiple sensor modalities. While on the one hand the existing success cases endorse the great potential of AI to help address open challenges in ES, on the other hand on-going discussions and established lessons from studies on the sustainability, ethics and trustworthiness of AI must be taken into consideration if the community is to ensure that its research efforts move into directions that effectively benefit the society and the environment. In this paper, we discuss insights gathered from a brief literature review on the subtopics of AI Ethics, Sustainable AI, AI Trustworthiness and AI for Earth Sciences in an attempt to identify some of the promising directions and key needs to successfully bring these concepts together.

Artificial intelligence (AI) algorithms have proven to be quite effective in Earth observation applications, often, when extensive amounts of representative training data are available. At large, processing large volumes of observation data can be challenging due to a myriad of reasons that include the cost of acquiring labeled samples, computing resources, identifying critical data features for model prototyping, standardization of model building, and deployment. Practical novel tools and approaches are emerging across different communities. In this paper, we discuss several such recent methods from machine learning and share lessons from advanced, scalable workflows that could impact the advancement of multimodal data fusion for Earth Science applications.

Human civilization faces existential threats in the forms of climate change, food insecurity, pandemics, international conflicts, forced displacements, and environmental injustice. These overarching humanitarian challenges disproportionately impact historically marginalized communities worldwide. UN OCHA estimates that 274 million people will need humanitarian support in 2022. Despite growing perils to human and environmental well-being, there remains a paucity of publicly-engaged computing research to inform the design of interventions. Data science efforts exist, but they remain isolated from socioeconomic, environmental, cultural, and policy contexts at local and international scales. Moreover, biases and privacy infringements in data-driven methods further amplify existing inequalities. The result is that proclaimed benefits of data-driven innovations may remain inaccessible to policymakers, practitioners, and underserved communities whose lives they intend to transform. To address gaps in knowledge and improve the livelihood of marginalized populations, we have established the Data-driven Humanitarian Mapping and Policymaking, an interdisciplinary initiative.

Understanding how people occupy open spaces is important for research in support of population modeling, policy, national security, emergency response, and sustainability. For the past decade, there has been an increase in research toward capturing and reporting population dynamics and patterns of life at the building level and in some open public spaces such as cemeteries and parks. This is done through observation models developed from local sociocultural information acquired at various spatiotemporal scales to inform night, day, and episodic population occupancy estimates (people/1000 sq ft). Sociocultural information for cemeteries and parks is scarcely available and often collected manually. The process is not only marred by inconsistencies but is laborious and time consuming. In this study, we leverage convolutional neural networks (CNNs) and satellite imagery to derive grave site counts as proxy variables to support scalable and accurate sociocultural data required in a population observation model. Through a hybrid workflow (weak localization plus regression model), we characterize a large scale automation process to counting of grave sites. We evaluate and demonstrate the efficacy of proposed workflow using out-of-data set large satellite imagery and establish its broader impact on cemetery observation models.

Studies on domain adaptation (DA) for remote sensing (RS) imagery analysis lack consistency in selection and description of evaluation scenarios. Without properly characterizing datasets, model assumptions, and evaluation scenarios, it is difficult to objectively compare DA methods and reach conclusions about their suitability across different applications. With this motivation, this work seeks to empirically assess to which extent the interaction between data characteristics and model assumptions influence the effectiveness of DA methods. Using the widely explored task of building footprint segmentation as case study, we perform a large-scale study across over 200 domain adaptation scenarios that include variations across view angles, areas observed, and sensors used for data acquisition. Rather than adopting different model architectures or optimization criteria, we contrast the performances of two DA methods based on adversarial learning that differ only in their assumptions about source and target domains. Informed by metadata and data characteristics unveiled using traditional computer vision techniques as well as pre-trained deep models, we provide a detailed meta-analysis of experiments highlighting the importance of accurately considering data assumptions for DA in RS segmentation tasks. As a “cherry-picking” exercise demonstrates, different claims regarding which model is best could be made by selecting different subsets of evaluation scenarios. While well-calibrated assumptions can be beneficial, mismatching assumptions can lead to negative biases in DA applications. This study intends to motivate the community towards more consistent evaluation protocols, while providing recommendations and insights toward creating novel benchmark datasets, documenting data characteristics, application-specific knowledge, and model assumptions.

Humanitarian challenges, including natural disasters, food insecurity, climate change, racial and gender violence, environmental crises, the COVID-19 coronavirus pandemic, human rights violations, and forced displacements, disproportionately impact vulnerable communities worldwide. Despite these growing perils, there remains a notable paucity of data science research to scientifically inform equitable public policy decisions for improving the livelihood of at-risk populations. Scattered data science efforts exist to address these challenges, but they remain isolated from practice and prone to algorithmic harms. Consequently, proclaimed benefits of data-driven innovations remain inaccessible to policymakers, practitioners, and marginalized communities at the core of humanitarian actions and global development. To help address this gap, we propose the Data-driven Humanitarian Mapping Research Program, which focuses on developing novel data science methodologies that harness human-machine intelligence for high-stakes public policy and resilience planning. As a part of the initiative, we host the second KDD workshop to continue fostering a global community of researchers, policymakers, and practitioners to advance a commonly shared data science research agenda for just humanitarian actions, resilience planning, and sustainable development. We envision the Data-driven Humanitarian Mapping will bring in new paradigms for equitable data science and policy decision-making while helping create a sustainable world.

Semantic segmentation of images is an important computer vision task that emerges in a variety of application domains such as medical imaging, robotic vision and autonomous vehicles to name a few. While these domain-specific image analysis tasks involve relatively small image sizes (∼ 102 × 102), there are many applications that need to train machine learning models on image data with extents that are orders of magnitude larger (∼ 104 × 104). Training deep neural network (DNN) models on large extent images is extremely memory-intensive and often exceeds the memory limitations of a single graphical processing unit, a hardware accelerator of choice for computer vision workloads. Here, an efficient, sample parallel approach to train U-Net models on large extent image data sets is presented. Its advantages and limitations are analyzed and near-linear strong-scaling speedup demonstrated on 256 nodes (1536 GPUs) of the Summit supercomputer. Using a single node of the Summit supercomputer, an early evaluation of a recently released model parallel framework called GPipe is demonstrated to deliver ∼ 2X speedup in executing a U-Net model with an order of magnitude larger number of trainable parameters than reported before. Performance bottlenecks for pipelined training of U-Net models are identified and mitigation strategies to improve the speedups are discussed. Together, these results open up the possibility of combining both approaches into a unified scalable pipelined and data parallel algorithm to efficiently train U-Net models with very large receptive fields on data sets of ultra-large extent images.

Model generalization remains a key challenge in the analysis of large amounts of heterogeneous satellite image data. One major limiting factor in developing generalizable models, in the context of supervised learning, is the lack of high quality training datasets. A model's capacity to perform well on new data is often inhibited by imbalance and bias in the data that was used for training. This is especially a problem when using convolutional neural networks to classify urban land-use in satellite images. Notable dataset imbalance issues in this application include land-use type imbalance and image scene imbalance. To begin understanding these dataset imbalance problems in more detail, we develop and test a number of sampling methods for generating training image datasets from subjective training polygons for urban land-use classification. We investigate sampling at different point densities as a means to reduce content repetition and therefore content imbalance and bias in the training image dataset.

OpenStreetMap(OSM) provides extensive coverage of road network that can be a source to prepare training samples for automated road mapping using very high resolution(VHR) satellite images and machine learning. However, several studies have shown that the pervasive spatial misalignment between OSM vector data and VHR images yields poor quality training samples and thereby compromises the performance of models. In this study, we undertake to address this shortcoming and develop an automated line segment shifting workflow to yield OSM vector data that aligns with VHR road features to generate high quality training samples. The approach leverages the standard deviation differences of road pixels and background information to guide the transformations. By taking into account trees, shadows, cars and water body on or beside the road when STD was calculated, our method is robust to various road obstacles. Experimental validations are conducted to confirm the correctness of aligned OSM data showing up to 338% improvement compared with original OSM. Finally, based on visual inspection, the road map generated by aligned OSM also presents obvious quality improvement in comparison with map created by original OSM.

Co-registrating is a common pre-processing step for existing change detection algorithms, but registering bi-temporal images is nontrivial. The use of image patch as input for deep learning techniques provides a natural avenue to apply them in the OBIA framework, and have shown successful performance in the object-based land cover mapping and change detection applications. Even though attempts of applying deep learning techniques for change detection applications have been made with varying success, its application under OBIA framework for change detection have not been conducted and its tolerance for misregistration among temporal images are neither known. This study performed change detection under OBIA framework using deep learning techniques for the first time, and evaluated its performance regarding their tolerance of image misregistration on training and testing dataset. Our results demonstrate the proposed change detection scheme is surprisingly robust to image misregistration on the testing dataset, while classifiers trained with the training dataset containing image misregistration errors suffer from slight decrease of overall accuracy.

Machine learning and deep learning provide a means for generating urban land-use maps with relatively little human effort compared to manually digitizing images. This is especially important for supporting global and regional initiatives focused on sustainability, planning, health, pro-poor policy, infrastructure, and population distribution estimates. Many of these initiatives work in areas where geospatial data is scarce, such as the global south, and often use land-use maps to help achieve their goals. In this study, we develop a typology for automated labeling of urban land-use data that captures the variation in structural patterns within cities. A comparison of classification accuracy between convolutional neural networks (CNNs) and support vector machines (SVMs) coupled with handcrafted features is conducted. Through experimental validation on two highly dense cities in Africa, we report on new insights and the potential benefits offered by both multiscale handcrafted features and multiscale-CNNs even with limited training data.

Most current state-of-the-art methods for semantic segmentation on remote sensing imagery require large labeled data, which is scarcely available. Due to the distribution shifting phenomenon inherent in remote sensing imagery, the reuse of pre-trained models on new areas of interest rarely yield satisfactory results. In this paper, we approach this problem from an adversarial learning perspective toward unsupervised domain adaptation. The core concept is to infuse fully convolutional neural networks and adversarial networks for semantic segmentation assuming the structures in the scene and objects of interest are similar in two set of images. Models are trained on a source dataset where ground truth is available and adapted to new target dataset iteratively via a adversarial loss on unlabeled samples. We use two real large scale datasets to validate the framework: 1) cross city road extraction and 2) cross country building extraction. The preliminary results show the usefulness of considering adversarial learning for indirect re-use of the pre-trained models. Experimental validation suggests significant benefits over models without adaptation.

Summary This paper presents a scalable object detection workflow for detecting objects, such as settlements, from remotely sensed (RS) imagery. We have successfully deployed this workflow on Titan supercomputer and utilized it for the task of mapping human settlement at a country scale. The performance of various stages in the workflow was analyzed before making it operational. The workflow implemented various strategies to address issues such as suboptimal resource utilization and long‐tail effects due to unbalanced image workload, data loss due to runtime failures, and maximum wall‐time constraints imposed by Titan's job scheduling policy. A mean shift clustering–based static load balancing strategy was implemented, which partitions the image load such that each partition contained similar‐sized images. Furthermore, a checkpoint‐restart strategy was added in the workflow as a fault‐tolerance mechanism to prevent the data losses due to unforeseen runtime failures. The performance of the above‐mentioned strategies was observed in various scenarios, such as node failure, exceeding wall time, and successful completion. Using this workflow, we have processed an RS data set that has a spatial resolution of 0.31 m and is comprised of 685 675 km2 of area of the Republic of Zambia in under six hours using 5426 nodes of the Titan supercomputer.

Finding where people live and the vulnerabilities of manmade facilities during natural disasters is not only critical for rescue efforts but also essential for damage assessment in the aftermath. New advances from machine learning and high performance computing are leveraging on the availability of high resolution satellite imagery to generate geographical maps for man-made facilities at scale. Mapping from satellite imagery can be a daunting task due to the enormous amount of data to be processed over large areas. In this short paper we take advantage of annotated satellite imagery and automate the semantic labeling of mobile home parks using an efficient framework rooted in patch-based and pixel-level classification. This multilevel labeling effort is a precursor for deploying very large scale deep convolutional neural networks toward broad and finer characterization of man-made structures from one-meter resolution NAIP images.

Two machine learning techniques were developed to extract human settlements from very high resolution (VHR) satellite images of 3 provinces in Afghanistan: Logar, Panjsher, and Wardak. The results were then compared with analyst verified reference data information known as the LandScan Settlement Layer (LandScan SL).[1] This study attempts to compare settlement mapping results from a support vector machine (SVM) classifier specifically integrated in a current settlement mapping framework and a deep learner utilizing a convolutional neural network (CNN) approach. By comparing the results from the SVM and the CNN to the reference data information we demonstrate that the CNN yields more accurate results overall, in terms of overall pixel cells, and the SVM performs more accurately in omission, based on derived statistics against the reference data information.

We test this premise and explore representation spaces from a single deep convolutional network and their visualization to argue for a novel unified feature extraction framework. The objective is to utilize and re-purpose trained feature extractors without the need for network retraining on three remote sensing tasks i.e. superpixel mapping, pixel-level segmentation and semantic based image visualization. By leveraging the same convolutional feature extractors and viewing them as visual information extractors that encode different settlement representation spaces, we demonstrate a preliminary inductive transfer learning potential on multiscale experiments that incorporate edge-level details up to semantic-level information.

Very large overhead imagery associated with ground truth maps has the potential to generate billions of training image patches for machine learning algorithms. However, random sampling selection criteria often leads to redundant and noisy-image patches for model training. With minimal research efforts behind this challenge, the current status spells missed opportunities to develop supervised learning algorithms that generalize over wide geographical scenes. In addition, much of the computational cycles for large scale machine learning are poorly spent crunching through noisy and redundant image patches. We demonstrate a potential framework to address these challenges specifically, while evaluating a human settlement detection task. A novel binary search tree sampling scheme is fused with a kernel based hashing procedure that maps image patches into hash-buckets using binary codes generated from image content. The framework exploits inherent redundancy within billions of image patches to promote mostly high variance preserving samples for accelerating algorithmic training and increasing model generalization.

Establishing up-to-date nationwide building maps is essential to understand urban dynamics, such as estimating population and urban planning and many other applications. However, an efficient and effective solution is yet to be developed. In this paper, for the first time we evaluate three state-of-the-art CNNs for detecting buildings across entire United States using aerial images. The three CNN architectures, fully convolutional neural network, conditional random field as recurrent neural network, and SegNet, support semantic pixel-wise labeling and focus on capturing textural information at multi-scale. We use 1-meter resolution NAIP images as the test data set, and compare the detection results across the three methods. In addition, we propose to combine signed distance function labels with SegNet, which is the preferred CNN architecture identified by our extensive evaluations. The results are further improved in terms of precision, recall rate and the number of building detected. On average, model inference on test images is less than one minute for an area of size ∼ 56 km2. With these promising results and the time required to process images, the framework offers great potential toward country scale building mapping with remote sensing imagery.

Multidimensional embedding is a technique useful for characterizing spectral signature relations in hyperspectral images. However, such images consist of disjoint similar spectral classes that are spatially sensitive, thus presenting challenges to existing graph embedding tools. Robust parameter estimation is often difficult when the image pixels contain several hundreds of bands. In addition, finding a corresponding high-quality lower dimensional coordinate system to map signature relations remains an open research question. We answer positively on these challenges by first proposing a combined kernel function of spatial and spectral information in computing neighborhood graphs. We further adapt a force field intuition from mechanics to develop a unifying nonlinear graph embedding framework. The generalized framework leads to novel unsupervised multidimensional artificial field embedding techniques that rely on the simple additive assumption of pair-dependent attraction and repulsion functions. The formulations capture long-range- and short-range-distance-related effects often associated with living organisms and help to establish algorithmic properties that mimic mutual behavior for the purpose of dimensionality reduction. In its application, the framework reveals strong relations to existing embedding techniques, and also highlights sources of weaknesses in such techniques. As part of evaluation, visualization, gradient field trajectories, and semisupervised classification experiments are conducted for image scenes acquired by multiple sensors at various spatial resolutions over different types of objects. The results demonstrate the superiority of the proposed embedding framework over various widely used methods.

Graph embedding techniques are useful to characterize spectral signature relations for hyperspectral images. However, such images consists of disjoint classes due to spatial details that are often ignored by existing graph computing tools. Robust parameter estimation is a challenge for kernel functions that compute such graphs. Finding a corresponding high quality coordinate system to map signature relations remains an open research question. We answer positively on these challenges by proposing a kernel function of spatial and spectral information in computing neighborhood graphs. Furthermore, a multidimensional artificial field graph embedding technique that relies on simple additive assumptions of pair-dependent attraction and repulsion functions is proposed. High quality visualizations and improved classification performance demonstrate the benefits of the approach.

In hyperspectral imagery, low-dimensional representations are sought in order to explain well the nonlinear characteristics that are hidden in high-dimensional spectral channels. While many algorithms have been proposed for dimension reduction and manifold learning in Euclidean spaces, very few attempts have focused on non-Euclidean spaces. Here, we propose a novel approach that embeds hyperspectral data, transformed into bilateral probability similarities, onto a nonlinear unit norm coordinate system. By seeking a unitl2-norm nonlinear manifold, we encode similarity representations onto a space in which important regularities in data are easily captured. In its general application, the technique addresses problems related to dimension reduction and visualization of hyperspectral images. Unlike methods such as multidimensional scaling and spherical embeddings, which are based on the notion of pairwise distance computations, our approach is based on a stochastic objective function of spherical coordinates. This allows the use of an Exit probability distribution to discover the nonlinear characteristics that are inherent in hyperspectral data. In addition, the method directly learns the probability distribution over neighboring pixel maps while computing for the optimal embedding coordinates. As part of evaluation, classification experiments were conducted on the manifold spaces for hyperspectral data acquired by multiple sensors at various spatial resolutions over different types of land cover. Various visualization and classification comparisons to five existing techniques demonstrated the strength of the proposed approach while its algorithmic nature is guaranteed to converge to meaningful factors underlying the data.