NASA has created the Science Discovery Engine, a search capability built using insight engine technology, to enable discovery of NASA’s open science data and information. In this paper, we present the SDE architecture and curation methodology developed using an insight engine capability. We also share our progress to date, lessons learned implementing an insight engine capability and planned future work.

Searching for Earth Science phenomena in large archives of Earth Observation Satellite Imagery data requires elaborate processing and spatio-temporal indexing of the images into categories of the said phenomena. Manual tagging is laborious as it needs constant monitoring through vast volumes of satellite data, the volume and velocity of which is ever-increasing. A complete re-indexing is also needed when a new phenomenon of interest is to be searched through the data archive. Previous efforts to automate tagging have leveraged Machine Learning (ML) techniques to classify images into phenomena of interest. In this method, multiple ML algorithms, each specifically trained for detecting a particular phenomenon, are used for spatio-temporal indexing. While doing so negates the need for human indexing, the process of creating ML models for identifying a class of phenomena involves significant time and computation overhead. Moreover, ML algorithms require vast amounts of extremely scarce labeled data. Furthermore, the computation needed for re-indexing the data whenever a new phenomenon is added to be tagged is not negligible. We propose an alternative, data-driven framework to search through vast amounts of satellite data, that eliminates the need for manual indexing, labeling, or creating purpose-built ML classifiers. The proposed method leverages Self-Supervised Learning (SSL) techniques to obtain feature vectors that are used for search and retrieval of satellite images. An Approximate Nearest Neighbors (ANN) algorithm is used to cluster and retrieve images exhibiting similar features, and by extension, similar Earth Science phenomena. Our unique contribution in this work is the orchestration of the methodology with various cloud services that facilitates searching through millions of images within a short span of time. To showcase the framework, we created a web interface to search through 21 years worth of daily satellite imagery with global coverage. In this paper, we discuss the progress we have made in enabling Embedding Based Search within Remote Sensing, and discuss the potential benefits and pitfalls involved in realizing this method. We also aim to provide insights and experiences we documented while developing such a system along with potential limitations of the current stage of the framework.

Supraglacial lakes (SGL)s result from melt water accumulation in topographic depressions on the surface of glaciers. SGLs primarily affect glacial dynamics through a positive feedback loop in which the albedo-lowering effect of SGLs can escalate surface melt leading to increases in lake extent and depth, amplifying the aforementioned albedo-lowering effect. The implications of accelerated glacial melt include increased sea level rise and modifications to ocean primary productivity. SGLs are critical indicators of surface melt and its downstream impacts and should be monitored efficiently. In situ observations and measurements of SGLs are time consuming, cost-prohibitive and difficult to scale. Earth observation data and machine learning enable scalable monitoring of SGLs through pattern detection and quantification of lake evolution over time [1]. This work presents a model developed by training a convolutional neural network with imagery and labels from NASA Operation IceBridge and predicting SGLs in high temporal and spatial resolution PlanetScope imagery.

NASA Earth Science Data Systems (ESDS) program manages a full lifecycle of data collected by all Earth science missions. ESDS also develops capabilities optimized to support rigorous science investigations. As technology landscapes evolve, ESDS has also evolved to transform its internal services and enhance external user centric services. This paper describes how ESDS is (i) adopting artificial intelligence (AI) technology to improve core services and (ii) enabling its users to advance AI driven research and build applications.

The use of deep learning techniques to build transformer language models such as SciBERT and GPT3 have transformed the natural language technology (NLT) landscape. These new NLTs are being used in speech to text and vice versa, automated text classification, sentiment analysis, topic modeling, text summarization, and cognitive assistants. While Earth science has no shortage of unstructured data such as journal and conference papers, little efforts have focused on harnessing NLTs for knowledge extraction and supporting the scientific process. This paper surveys the use of language models in different science. BERT-E, a new Earth science-specific language model, is presented. BERT-E is generated using a transfer learning solution. A language model that has already been trained for general Science (SciBERT) is fine-tuned using abstracts and full text extracted from various Earth science-related articles. A downstream keywords classification application is used for evaluation, and the use of BERT-E shows improved performance. The need to develop a robust set of benchmarks in evaluating the language model such as BERT-E is discussed. Finally, example applications are presented to inspire additional ideas for applications using domain-specific language models.

The COVID-19 pandemic caused authorities to limit or lockdown cities resulting in changes in human behaviors that impacted the Earth system. Studying such impacts on the Earth system requires an integrated study of relevant parameters using remotely sensed data. This paper discusses a unique dashboard that brings Earth observation datasets together to visualize, explore, and communicate the environmental effect of human behavior due to COVID-19.

One of the challenges of improving the search and use of complex Earth science data is designing and incorporating semantic components in existing Earth science data systems. Many projects have addressed this by using a knowledge engineering approach. However, using ontologies has inherent limitations as a practical and scalable approach. Data-driven strategies based on natural language processing, coupled with Machine Learning, provide an alternative approach. Data-driven approaches utilize existing corpus available as unstructured text. This paper describes a hybrid strategy that uses a data-driven approach to build an embedding from a large corpus of Earth science journal publications while leveraging existing ontologies to develop validation tests to evaluate the embedding's robustness and correctness. The paper also describes the use of this embedding in two different applications. The first application provides a semantic mapping service to bridge the gap between a science application need and the appropriate instruments or datasets required to address that need. The second application is keyword recommender to make the data set tagging process efficient for the data operators and ensure keyword consistency within a data catalog.

Earth science data archives have significantly increased in size due to the number of advanced sensors and science missions. In the meantime, Earth science data systems have not taken advantage of data driven technologies to provide advanced search capabilities. This paper discusses a machine learning-based approach, an enabling data driven technology, to detect Earth science events from image archives. The automated event detection is cataloged in an event database that provides a novel way to explore large archives of data. In addition, a phenomena portal to visually explore events and contextual information is discussed.

Recent review papers have discussed the opportunities and challenges of applying machine learning (ML) techniques to Earth science data. A common challenge cited in these papers is the lack of labeled training data. A literature review of Earth science papers over the last 10 years demonstrates that while there is rapid adoption of ML, particularly in biogeoscience and land surface research, the training datasets typically contain only hundreds of samples. This lack of training data limits the use of deep learning algorithms, which require larger volumes of labeled data. In situ training data are most frequently used in almost all domains, followed by model output and satellite data. The atmosphere and solid Earth domains use the largest training datasets, an order of magnitude larger than in biogeoscience papers. Random forest is the most commonly applied ML algorithm in all domains except atmospheric science and biogeoscience, which more frequently use fully connected neural networks.

Enterprises are making machine learning for production as an integral part of their future roadmaps and Earth science domain is no exception. However, there is common problem in transitioning machine learning from science to production due to a major difference in constructing a model versus deploying it for people to use to make decisions. Phases of machine learning lifecycle that includes model transition to production using a successful application is discussed.