Identifying and handling label errors can significantly enhance the accuracy of supervised machine learning models. Recent approaches for identifying label errors demonstrate that a low self-confidence of models with respect to a certain label represents a good indicator of an erroneous label. However, latest work has built on softmax probabilities to measure selfconfidence. In this paper, we argue that—as softmax probabilities do not reflect a model’s predictive uncertainty accurately— label error detection requires more sophisticated measures of model uncertainty. Therefore, we develop a range of novel, model-agnostic algorithms for Uncertainty Quantification-Based Label Error Detection (UQ-LED), which combine the techniques of confident learning (CL), Monte Carlo Dropout (MCD), model uncertainty measures (e.g., entropy), and ensemble learning to enhance label error detection. We comprehensively evaluate our algorithms on four image classification benchmark datasets in two stages. In the first stage, we demonstrate that our UQ-LED algorithms outperform state-of-the-art confident learning in identifying label errors. In the second stage, we show that removing all identified errors from the training data based on our approach results in higher accuracies than training on all available labeled data. Importantly, besides our contributions to the detection of label errors, we particularly propose a novel approach to generate realistic, class-dependent label errors synthetically. Overall, our study demonstrates that selectively cleaning datasets with UQ-LED algorithms leads to more accurate classifications than using larger, noisier datasets.

Increasing data volumes and migration of data to the cloud introduces challenges related to scalability and complexity to users of National Aeronautics and Space Administration (NASA) Earth science data. The Visualization, Exploration, and Data Analysis (VEDA) platform is an open-source modular cyberinfrastructure that leverages community standards and builds upon existing open-source capabilities for data services and geospatial data visualization. Four major components make up the VEDA platform – a cloud-optimized data store, backend data services, a web-based visualization and data-driven storytelling dashboard, and an analysis hub. The modular design of the VEDA platform enables the tailored use of the platform components or the platform in whole as has been demonstrated by other applications leveraging VEDA within NASA and the U.S. government.

An equitable and environmentally just community is essential in order to avoid disproportionate burden borne by vulnerable communities. This need becomes pressing in the aftermath of an extreme event such as disaster or hazard when it is difficult for the governing bodies to implement resource allocation as per the need. Artificial Intelligence (AI) algorithms can help surface Equity and Environmental Justice (EEJ) issues when trained on EEJ datasets. However, curating AI-ready EEJ training datasets is challenging due to differences in factors such as heterogeneity, resolution, modality, and level of expertise in labeling. Additionally, EEJ issues involve sensitive information where uncertainties and errors could degrade the performance of AI algorithms. For eg. error in seasonal crop yield information can highly effect the prediction of annual crop yield. To address these challenges, Data-centric AI (DCAI) methods are employed, which enhance AI algorithm performance even with limited training samples. DCAI prioritizes data quality, thereby reducing the adverse effects of uncertainties and errors during the model training process. This research proposes a novel dataset and benchmark for analyzing the effect of the Maui Wildfire of 2023 for Equity and Environmental Justice (EEJ) issues. The proposed dataset aligns with the concepts of DCAI such as annotation quality, data preprocessing, privacy, feature engineering, governance and provenance. The proposed AI-ready dataset is available on HuggingFace at https://huggingface.co/datasets/nasa-impact/ml4ej-maui-wildfire.

Data is a critical resource for machine learning (ML), yet working with data remains a key friction point. This paper introduces Croissant, a metadata format for datasets that creates a shared representation across ML tools, frameworks, and platforms. Croissant makes datasets more discoverable, portable, and interoperable, thereby addressing significant challenges in ML data management. Croissant is already supported by several popular dataset repositories, spanning hundreds of thousands of datasets, enabling easy loading into the most commonly-used ML frameworks, regardless of where the data is stored. Our initial evaluation by human raters shows that Croissant metadata is readable, understandable, complete, yet concise.

International collaboration among space agencies is central to the success of satellite Earth observations and data analysis, aiming at providing an accurate and timely information to decision-makers, main stakeholders and public. These partnerships foster more comprehensive measurements, robust datasets, and cost-effective missions. The European Space Agency (ESA), Japan Aerospace Exploration Agency (JAXA), and National Aeronautics and Space Administration (NASA) have combined their resources, technical knowledge, and expertise to produce the Earth Observing Dashboard [1], which strengthens our understanding of global environmental changes and other societal challenges impacting our planet [2].

Open science aims to foster transparent sharing of scientific processes including open access, incentivization, provenance, open source code and tools, metrics, and resource sharing. However, effective management of these processes remains a challenge. This paper explores the application of blockchain technology to address these key aspects of open science. Blockchain offers a decentralized and secure platform for information exchange and verification. By leveraging blockchain, open science can enhance transparency and reproducibility. In this paper, we present an implementation of blockchain for Earth science data synchronization across organizations, enabling tracking of data copying, citation, and download. The findings highlight the potential of blockchain in supporting open science objectives.

Searching for Earth Science phenomena in large archives of Earth Observation Satellite Imagery data requires elaborate processing and spatio-temporal indexing of the images into categories of the said phenomena. Manual tagging is laborious as it needs constant monitoring through vast volumes of satellite data, the volume and velocity of which is ever-increasing. A complete re-indexing is also needed when a new phenomenon of interest is to be searched through the data archive. Previous efforts to automate tagging have leveraged Machine Learning (ML) techniques to classify images into phenomena of interest. In this method, multiple ML algorithms, each specifically trained for detecting a particular phenomenon, are used for spatio-temporal indexing. While doing so negates the need for human indexing, the process of creating ML models for identifying a class of phenomena involves significant time and computation overhead. Moreover, ML algorithms require vast amounts of extremely scarce labeled data. Furthermore, the computation needed for re-indexing the data whenever a new phenomenon is added to be tagged is not negligible. We propose an alternative, data-driven framework to search through vast amounts of satellite data, that eliminates the need for manual indexing, labeling, or creating purpose-built ML classifiers. The proposed method leverages Self-Supervised Learning (SSL) techniques to obtain feature vectors that are used for search and retrieval of satellite images. An Approximate Nearest Neighbors (ANN) algorithm is used to cluster and retrieve images exhibiting similar features, and by extension, similar Earth Science phenomena. Our unique contribution in this work is the orchestration of the methodology with various cloud services that facilitates searching through millions of images within a short span of time. To showcase the framework, we created a web interface to search through 21 years worth of daily satellite imagery with global coverage. In this paper, we discuss the progress we have made in enabling Embedding Based Search within Remote Sensing, and discuss the potential benefits and pitfalls involved in realizing this method. We also aim to provide insights and experiences we documented while developing such a system along with potential limitations of the current stage of the framework.

Supraglacial lakes (SGL)s result from melt water accumulation in topographic depressions on the surface of glaciers. SGLs primarily affect glacial dynamics through a positive feedback loop in which the albedo-lowering effect of SGLs can escalate surface melt leading to increases in lake extent and depth, amplifying the aforementioned albedo-lowering effect. The implications of accelerated glacial melt include increased sea level rise and modifications to ocean primary productivity. SGLs are critical indicators of surface melt and its downstream impacts and should be monitored efficiently. In situ observations and measurements of SGLs are time consuming, cost-prohibitive and difficult to scale. Earth observation data and machine learning enable scalable monitoring of SGLs through pattern detection and quantification of lake evolution over time [1]. This work presents a model developed by training a convolutional neural network with imagery and labels from NASA Operation IceBridge and predicting SGLs in high temporal and spatial resolution PlanetScope imagery.

Developed by the researchers at the Global Hydrology Resource Center (GHRC) and Oak Ridge National Laboratory (ORNL) Distributed Active Archive Centers (DAACs), the Earthdata Pub system has been developed to improve the user experience for data providers archiving data at any of NASA's 12 DAACs. Earthdata Pub is a cloud-native, open-source system intended to create a enterprise-wide interface with NASA's DAACs. This will create a consistent user experience for submitting data to the DAACs. This presentation will discuss the origins and architecture of Earthdata Pub, its uses, and future development work.

NASA Earth Science Data Systems (ESDS) program manages a full lifecycle of data collected by all Earth science missions. ESDS also develops capabilities optimized to support rigorous science investigations. As technology landscapes evolve, ESDS has also evolved to transform its internal services and enhance external user centric services. This paper describes how ESDS is (i) adopting artificial intelligence (AI) technology to improve core services and (ii) enabling its users to advance AI driven research and build applications.

The use of deep learning techniques to build transformer language models such as SciBERT and GPT3 have transformed the natural language technology (NLT) landscape. These new NLTs are being used in speech to text and vice versa, automated text classification, sentiment analysis, topic modeling, text summarization, and cognitive assistants. While Earth science has no shortage of unstructured data such as journal and conference papers, little efforts have focused on harnessing NLTs for knowledge extraction and supporting the scientific process. This paper surveys the use of language models in different science. BERT-E, a new Earth science-specific language model, is presented. BERT-E is generated using a transfer learning solution. A language model that has already been trained for general Science (SciBERT) is fine-tuned using abstracts and full text extracted from various Earth science-related articles. A downstream keywords classification application is used for evaluation, and the use of BERT-E shows improved performance. The need to develop a robust set of benchmarks in evaluating the language model such as BERT-E is discussed. Finally, example applications are presented to inspire additional ideas for applications using domain-specific language models.

The measures to contain the Covid-19 pandemic have had worldwide impacts on our environment, societies and economies. The ‘COVID-19 Earth Observation Dashboard’ (https://eodashboard.org) jointly developed by ESA, NASA and JAXA, combines a wealth of data from the tri-agencies' Earth-observing satellites to monitor the worldwide impacts of COVID-19. Developed in an Open Science framework, the dashboard is openly available to users worldwide and allows to track changes in air and water quality, climate, economic activity and agriculture.

The COVID-19 pandemic has had substantial impacts on the Earth system and socioeconomic activities. Restrictions aimed at reducing the spread of COVID-19 by limiting human interaction have led to significant reductions in air pollution and CO2emissions, improvement in water quality, changes in agricultural output, and changes in economic activity for certain industries such as airlines and shipping, among others [1]–[5]. Those economic impact assessment related information are made available on the trilateral COVID-19 Earth Observing Dashboard (https://eodashboard.org) [6]. The presented use cases in economic activities make full use of the combined satellites fleet of NASA, ESA and JAXA as well as the expertise of the Earth Observation community.

The COVID-19 pandemic caused authorities to limit or lockdown cities resulting in changes in human behaviors that impacted the Earth system. Studying such impacts on the Earth system requires an integrated study of relevant parameters using remotely sensed data. This paper discusses a unique dashboard that brings Earth observation datasets together to visualize, explore, and communicate the environmental effect of human behavior due to COVID-19.

NASA's Commercial Smallsat Data Acquisition (CSDA) program was initiated with a goal of acquiring data from commercial sources that support NASA's Earth science research and application goals. Over the last several years, the CSDA program has evolved into a long-term sustained program. This paper presents an overview of the program, a featured innovative application, and data management capabilities.

One of the challenges of improving the search and use of complex Earth science data is designing and incorporating semantic components in existing Earth science data systems. Many projects have addressed this by using a knowledge engineering approach. However, using ontologies has inherent limitations as a practical and scalable approach. Data-driven strategies based on natural language processing, coupled with Machine Learning, provide an alternative approach. Data-driven approaches utilize existing corpus available as unstructured text. This paper describes a hybrid strategy that uses a data-driven approach to build an embedding from a large corpus of Earth science journal publications while leveraging existing ontologies to develop validation tests to evaluate the embedding's robustness and correctness. The paper also describes the use of this embedding in two different applications. The first application provides a semantic mapping service to bridge the gap between a science application need and the appropriate instruments or datasets required to address that need. The second application is keyword recommender to make the data set tagging process efficient for the data operators and ensure keyword consistency within a data catalog.

On May 30, 31 and June 1, 2018, a workshop on International Spaceborne SAR Missions Coordination and Collaboration was held at the California Institute of Technology to explore the interest, advantage and the significance of a more coordinated approach between the different organizations to achieve higher value to the user community. One of the main topics of this workshop is to make a recommendation to improve data visibility and accessibility of spaceborne SAR under the international coordination. During this Workshop, Working Group 1 (WG-1) was established to understand the issues related to data discovery and data access, as well as to discuss and coordinate this topic with good examples. This paper shows the status and progress of WG-1 related activities.

The COVID-19 pandemic has brought difficulties to daily lives and caused various changes in socio-economic activities and the social environment. Since the satellite-based Earth observation allows us to monitor the Earth's surface globally and periodically with various physical parameters, as space agencies, it is our responsibility to monitor the changes of the Earth from space, share the observation results with the public, and record them for future generations. Aiming to fulfill this obligation, JAXA has cooperated with NASA and ESA to develop the Earth Observing Dashboard, which provides information on the changes of the Earth environment before/after COVID-19 pandemic. This paper discusses the overview of the three agency's cooperation on COVID-19 dashboard.

The Global Hydrology Resource Center (GHRC) Distributed Active Archive Center (DAAC), developed the Field Campaign eXplorer (FCX) to address a limitation in available visualization resources. FCX is a cloud-native, open-source system capable of visualizing multiple datasets in three dimensions. This includes data from ground-, airborne-, and satellite-based observations. The open-source nature will further allow users of FCX to develop their own extensions for both visualizations and analyses. This paper will discuss the architecture of FCX, its uses, and future development work.

Collaborative open science practices are changing the way research is conducted. These changes affect how scientists work together on data, code and information. Data systems enhance open science by offering forward thinking technological solutions, such as providing data and computation on the cloud, to enable collaboration, sharing and analysis. In this paper, we present our vision for a conceptual data system on the cloud that enables open science. We also present our work on the Multi-Mission Algorithm and Analysis Platform (MAAP) which has served as a pathfinder data system for this conceptual approach.

Earth science data archives have significantly increased in size due to the number of advanced sensors and science missions. In the meantime, Earth science data systems have not taken advantage of data driven technologies to provide advanced search capabilities. This paper discusses a machine learning-based approach, an enabling data driven technology, to detect Earth science events from image archives. The automated event detection is cataloged in an event database that provides a novel way to explore large archives of data. In addition, a phenomena portal to visually explore events and contextual information is discussed.

New adaptive approaches to Earth observation data stewardship need to be adopted in order to allow for higher data volumes, heterogeneous data and constantly evolving technologies. The data ecosystem approach to stewardship offers a viable solution to this need by placing an emphasis on the relationships between data, technologies and people. In this paper, we present the Joint ESA-NASA Multi-Mission Algorithm and Analysis Platform's (MAAP) creation of a data ecosystem to support global aboveground terrestrial carbon dynamics research. We present the components needed to support the MAAP data ecosystem along with two data stewardship workflows used in the MAAP and the development of extended metadata for MAAP.

Diverse airborne and ground-based environmental observations are important technologies for disaster assessment and response, as well as for the validation of environmental satellite observations and atmospheric models which can improve forecasts. The VISAGE (Visualization for Integrated Satellite, Airborne and Ground-based data Exploration) project is working to provide three-dimensional visualization and basic analytics capabilities for such datasets in an interactive user interface. The use of cloud-native, serverless technologies and analysis optimized data storage will position VISAGE for integration with other technologies into a Data Analytic Center Framework.

Enterprises are making machine learning for production as an integral part of their future roadmaps and Earth science domain is no exception. However, there is common problem in transitioning machine learning from science to production due to a major difference in constructing a model versus deploying it for people to use to make decisions. Phases of machine learning lifecycle that includes model transition to production using a successful application is discussed.

Deep learning is a subset of machine learning that uses deep neural networks (DNNs) capable of learning representations and extracting valuable information from vast data sets. Similarly, weather phenomena are often identified by patterns in data that serve as precursor signatures. Therefore, deep learning networks can be used to identify signatures of the weather phenomena, or possibly signatures not yet established by forecasters in addition to aiding forecasters in synthesizing the growing amount of meteorological observations. In this article, we demonstrate the value of deep learning for atmospheric science applications by providing a proof of concept, using deep learning for the detection of hail-bearing storms as a test case study. The deep learning network presented in this article obtains a higher precision when presented with multisource data and is able to identify a common feature associated with hail storms-decreased infrared brightness temperatures. This network and case study illustrate the capability of deep networks for the detection of weather phenomena and contribute to the growing awareness of deep learning among atmospheric scientists.

Deep learning has revolutionized computer vision and natural language processing with various algorithms scaled using high-performance computing. The Data Science and Informatics Group (DSIG) at the NASA Marshall Space Flight Center (MSFC), has been using deep learning for a variety of Earth science applications. This paper provides examples of the applications and also addresses some of the challenges that have been encountered.

Tropical cyclone intensity estimation is a challenging task as it required domain knowledge while extracting features, significant pre-processing, various sets of parameters obtained from satellites, and human intervention for analysis. The inconsistency of results, significant pre-processing of data, complexity of the problem domain, and problems on generalizability are some of the issues related to intensity estimation. In this study, we design a deep convolutional neural network architecture for categorizing hurricanes based on intensity using graphics processing unit. Our model has achieved better accuracy and lower root-mean-square error by just using satellite images than 'state-of-the-art' techniques. Visualizations of learned features at various layers and their deconvolutions are also presented for understanding the learning process.

Service (API) discovery and recommendation is key to the wide spread of service oriented architecture and service oriented software engineering. Service recommendation typically relies on service linkage prediction calculated by the semantic distances (or similarities) among services based on their collection of inherent attributes. Given a specific context (mashup goal), however, different attributes may contribute differently to a service linkage. In this paper, instead of training a model for all attributes as a whole, a novel approach is presented to simultaneously train separate models for individual attributes. Meanwhile, a latent attribute modeling method is developed to reveal context-aware attribute distribution. Experiments over real-world datasets have demonstrated that this fine-grained method yields higher link prediction accuracy.

This paper presents two research applications exploiting unused metadata resources in novel ways to aid data discovery and exploration capabilities. The results based on the experiments are encouraging and each application has the potential to serve as a useful standalone component or service in a data system.

The role of a texture's directionality (i.e., Orientedness) in multivariate visualization is explored. A key emphasis here is determining if directional textures can be an effective component in the visualization of multiple attribute data, in particular weather data. Toward that end, a new directional texture-based data visualization technique is described and exhibited. Results of user-based evaluations of directional textures in visualization are also reported.

There is a dearth of software tools that allow users to easily visualize, analyze and mine satellite imagery. The few tools that are available are expensive commercial packages that provide limited functionality. As part of a NASA funded project, a software tool named GLIDER is currently being developed to fill this void. GLIDER allows users to visualize and analyze satellite data in its native sensor view. Users can enhance the image by applying different image processing algorithms on the data. GLIDER provides the users with a full suite of pattern recognition and data mining algorithms that can be applied to the satellite imagery to extract thematic information. The suite of algorithms includes both supervised and unsupervised classification algorithms. In addition, users can project satellite imagery and analysis/mining results onto a 3D globe for visualization. GLIDER also allows users to add additional layers to the globe along with the projected image. Users can open multiple views within GLIDER to manage, visualize and analyze many data files all at once. This paper describes the features of GLIDER version 1.0.

This paper describes approaches and methodologies facilitating the analysis of large amounts of distributed scientific data. The existence of full-featured analysis tools, such as the Algorithm Development and Mining (ADaM) toolkit and online data repositories now provide easy access and analysis capabilities to large amounts of data. However, there are obstacles to getting the analysis tools and the data together in a workable environment. Does one bring the data to the tools or deploy the tools close to the data? The large size of many current Earth science datasets incurs significant overhead in network transfer for analysis workflows, even with the current advanced networking capabilities. We are developing two solutions for this problem that address different analysis scenarios. The first is a Data Center Deployment of the analysis services for large data selections, orchestrated by a remotely defined analysis workflow. The second is a Data Mining Center approach of providing a cohesive analysis solution for smaller subsets of data. The two approaches can be complementary and thus provide flexibility for researchers to exploit the best solution for their data requirements.

Working closely with atmospheric scientists at the Marshall Space Flight Center, researchers at the University of Alabama in Huntsville are applying Sensor Web Enablement (SWE) technologies to the real world problem of efficiently assimilating NASA satellite data into weather forecast models in near real time. By implementing SWE protocols and services into our Data Assimilation System we expect to realize a processing framework that is distributed, interoperable and plug-and-play, thereby increasing access to scientific products in a more efficient, autonomous, and affordable way.

The Southeastern Universities Research Association (SURA) coastal ocean observing and prediction (SCOOP) program is a SURA Coastal Research initiative that is deploying cutting edge information technology to advance the science of environmental prediction and hazard planning for our nation's coasts. SCOOP is a distributed program, incorporating heterogeneous data, software and hardware; thus the use of standards to enable interoperability is key to SCOOP's success. Standards activities range from internal coordination among SCOOP partners to participation in national standards efforts. As the lead partner in the SCOOP program for both data management and data translation, the University of Alabama in Huntsville (UAH) is developing a suite of advanced technologies to provide core data and information management services for scientific data, including the SCOOP Catalog and a suite of standards-based web services providing Catalog access. Currently under development is a web service that will export information on SCOOP data collections in a schema compliant with the Federal Geographic Data Committee's Content Standard for Digital Geospatial Metadata. SCOOP is also a participant in the OpenlOOS Interoperability Demonstration, which leverages open geospatial consortium (OGQ standards such as the Web map service (WMS) and Web Feature Service (WFS) protocols to display near real time coastal observations together with water level, wave, and surge forecasts. SCOOP partners are also active participants in several data and metadata standards efforts, including the national ocean sciences data management and communications metadata studies and the marine metadata interoperability project. Continued close cooperation between the IT and coastal science modeling communities is producing positive results toward a real-time modeling environment that will benefit coastal stakeholders through better predictive capabilities.