Deploying multi-satellite constellations for Earth observation requires coordinating potentially hundreds of spacecraft. With increasing onboard capability for autonomy, we can view the constellation as a multi-agent system (MAS) and employ decentralized scheduling solutions. We analyze the multi-satellite constellation observation scheduling problem (COSP) and formulate it as a distributed constraint optimization problem (DCOP). COSP requires scalable inter-agent communication and computation and consists of millions of variables which, coupled with the assumptions and structure, make existing DCOP algorithms inadequate for this application. We develop a scheduling approach that employs a carefully constructed heuristic, referred to as the Geometric Neighborhood Decomposition (GND) heuristic, to decompose the global DCOP into sub-problems to enable the application of DCOP techniques. We present the Neighborhood Stochastic Search (NSS) algorithm, a decentralized algorithm to effectively solve COSP and other large-scale distributed problems, using decomposition. The experiments confirm the efficacy of the approach against baseline algorithms, and we discuss the generality of NSS, GND, and properties of COSP to other domains.

Deploying multi-satellite constellations for Earth observation requires coordinating potentially hundreds of spacecraft. With increasing on-board capability for autonomy, we can view the constellation as a multi-agent system (MAS) and employ decentralized scheduling solutions. We formulate the problem as a distributed constraint optimization problem (DCOP) and desire scalable inter-agent communication. The problem consists of millions of variables which, coupled with the structure, make existing DCOP algorithms inadequate for this application. We develop a scheduling approach that employs a well-coordinated heuristic, referred to as the Geometric Neighborhood Decomposition (GND) heuristic, to decompose the global DCOP into sub-problems as to enable the application of DCOP algorithms. We present the Neighborhood Stochastic Search (NSS) algorithm, a decentralized algorithm to effectively solve the multi-satellite constellation observation scheduling problem using decomposition. In full, we identify the roadblocks of deploying DCOP solvers to a large-scale, real-world problem, propose a decomposition-based scheduling approach that is effective at tackling large scale DCOPs, empirically evaluate the approach against other baseline algorithms to demonstrate the effectiveness, and discuss the generality of the approach.

Maximizing the utility of limited Earth observing satellite resources is a difficult ongoing problem. Dynamic Targeting is an approach to this challenge that intelligently plans and executes primary sensor observations based on information from a look-ahead sensor. However, current implementations have failed to account for realistic satellite operational constraints and have used static utility for repeat observations of the same target. To address these limitations, we implement a more general Dynamic Targeting framework that comprises a physics-based slew model, a dynamic model of observation utility, and an algorithm for gathering high-utility observations. To demonstrate this framework, we also supply complex dynamic utility models that are applicable to many missions and new algorithms for intelligently scheduling observations with slewing restrictions and changing utility, including a greedy algorithm and a depth-first search algorithm. To evaluate these algorithms, we test their performance across simulated runs through two datasets and compare to the performance of an algorithm representative of most scheduling algorithms aboard Earth science missions today as well as an intractable upper bound. We show that our algorithms have great potential to improve science return from Earth science missions.

Dynamic targeting (DT) is an emerging concept for improving science yield on Earth-observing missions limited by power-constrained sensors. DT uses a lookahead sensor together with on-board decision making to save resources for valuable observations in the future. Previous work has focused on developing DT mission use cases, such as storm hunting and cloud avoidance, that have relatively straightforward observation goals (i.e., look for storms, avoid clouds). However, DT has the potential to improve the science return of more complex missions and studies. To demonstrate this, we present and develop a new DT mission scenario to study the Planetary Boundary Layer (PBL). This paper describes the elements of our PBL mission scenario, which not only only involves multiple spacecraft, but also more sophisticated instruments, science models, and on-board decision making.

Recent developments in New Space companies have led to a dramatic increase in capabilities in Earth Observation. These advances in edge computing, low latency communications, and many new on orbit assets represent a unique opportunity for Earth Observation. NASA’s New Observation System (NOS) program aims to leverage these new capabilities to achieve global reach of science events such as volcanic eruption, wildfires, flooding, not by wide swath instruments but rather by intelligent, directed sensing, onboard analysis, and dissemination of knowledge rather than data using low latency communications links. We describe ongoing efforts to deploy NOS capabilities to the CogniSAT-6/HAMMER satellite launched in March 2024, with a currently projected flight demonstration of late summer or early fall 2024.

Flooding has serious environmental and humanitarian effects. To track these effects, previous work has used remote sensing to achieve global monitoring at low to moderate resolutions or regional monitoring at high resolutions. We proposed that implementing a new sensorweb that had previously been prototyped only regionally in Thailand could combine moderate-resolution flood detection with targeted high-resolution observations to enable worldwide high-resolution flood monitoring. Furthermore, we aimed to integrate both commercial and government satellites into this sensorweb to improve upon previous efforts, which only used government satellites. To this end, we first gather data from several moderate-resolution sensors to identify large, flooded regions. We then task high-resolution sensors to observe these floods and analyze the resulting data, thus enabling global high-resolution flood monitoring. Overall, our approach improves worldwide analysis of floods, which can be further improved by our ongoing efforts to incorporate more flood detection sensors and generate more products.

We collect alerts of volcanic activity from space remote sensing, volcano observatories, and meteorological organizations [1], [2]. We use these alerts to autonomously generate <1 m/pixel observation requests for Planet SkySat [3] and 70m/pixel thermal infrared data from the ECOSTRESS instrument on the ISS. We also automatically download Dove imagery corresponding to alerts to supplement the SkySat and ECOSTRESS imagery. One new feature is that we can then automatically generate thermal products from the Planet image data using newly-developed band-comparison classifiers.This system has been running continuously for nearly 1.5 years without a dedicated operations team. It has generated over 650,000 alerts, tasked over 80 SkySat observations, and resulted in more than 6,000 scenes scheduled for ECOSTRESS.

Dynamic Targeting (DT) will enable future Earth Observing instruments to intelligently reconfigure and point instruments to dramatically enhance science return. In this work we present a realistic simulation study of DT for tracking of storm features. To this end we have developed several algorithms from Operations Research and Artificial Intelli-gence/heuristic search. We benchmark these algorithms and show that DT is a powerful tool with the potential to significantly improve science yield.

Deep space missions can benefit from onboard image analysis. We demonstrate deep learning inference to facilitate such analysis for future mission adoption. Traditional space flight hardware provides modest compute when compared to today's laptop and desktop computers. New generations of commercial off the shelf (COTS) processors designed for embedded applications, such as the Qualcomm Snapdragon and Movidius Myriad X, deliver significant compute in small Size Weight and Power (SWaP) packaging and offer direct hardware acceleration for deep neural networks. We deploy neural network models on these processors hosted by Hewlett Packard Enterprise's Spaceborne Computer-2 onboard the International Space Station (ISS). We benchmark a variety of algorithms trained on imagery from Earth or Mars, as well as some standard deep learning models for image classification.

The Smart Ice Cloud Sensing (SMICES) instrument aims at providing onboard smart autonomous observation of upper tropospheric water vapor and ice particle size distribution in clouds at various local times. SMICES is an active/passive combined sensor with sounding channels at 380 GHz, radiometric channels at 250, 310 and 670 GHz, and a radar instrument operating at 239 GHz. A low-noise, low-power radiometer command and data handling (C&DH) subsystem has been designed to acquire the 24 analog radiometer channels and 8 analog thermistor data. A radiometric power regulation system provides the required power supplies for the other radiometric subsystems of the SMICES instrument. An on-board FPGA provides command and control of other instrument subsystems, performs synchronous data acquisition. The radiometer electronics are designed to fit into less than 2U horizontal dimensions of a CubeSat instrument. An AI controller unit directly interfacing with radar and radiometer C&DH subsystems performs on-board artificial intelligence operations for full system autonomy. The AI unit will control the radar instrument depending on the system health conditions, including the battery level, and based on the observed scene through the radiometer instrument.

A new observing strategy for floods was demonstrated and evaluated in a testbed environment. The strategy coordinates several observing platforms, including in situ and space based, to observe a flood from multiple vantage points and dynamically target predicted flood events with highresolution observations. The coordinated observations were assimilated back into the model to continuously improve forecasts and future observation selection. The demonstration shows the potential for coordinated, model-driven observing strategies and the feasibility of the NOS Testbed for demonstrating and evaluating new observing strategies.

Future space missions will process and analyze imagery onboard placing greater demands on flight computing. Traditional flight hardware provides modest compute, even when compared to laptop and desktop computers. A new generation of commercial off the shelf (COTS) processors, such as Qualcomm Snapdragon, deliver significant compute in small Size Weight and Power (SWaP) and offer direct hardware acceleration in the form of Graphics Processing Units (GPU) and Digital Signal Processors (DSP). We benchmark a variety of instrument processing and analysis software (including machine learned classifiers) on a Qualcomm Snapdragon SoC currently hosted by HPE’ s Spaceborne Computer-2 (SBC-2) onboard the International Space Station.

Long range space missions, such as Rosetta, require robust plans of data-acquisition activities and of the resulting data transfers. In this paper we revisit the problem of assigning priorities to data transfers in order to maximize safety margin of onboard memory. We propose a fast sweep algorithm to verify the feasibility of a given priority assignment and we introduce an efficient exact algorithm to assign priorities on a single downlink window. We prove that the problem is NP-hard for several windows, and we propose several randomized heuristics to tackle the general case. Our experimental results show that the proposed approaches are able to improve the plans computed for the real mission by the previously existing method, while the sweep algorithm yields drastic accelerations.

Planning for autonomous operation in unknown environments poses a number of technical challenges. The agent must ensure robustness to unknown phenomena, un-predictable variation in execution, and uncertain resources, all while maximizing its objective. These challenges are ex-acerbated in the context of space missions where uncertainty is often higher, long communication delays necessitate robust autonomous execution, and severely constrained computational resources limit the scope of planning techniques that can be used. We examine this problem in the context of a Europa Lander concept mission where an autonomous lander must collect valuable data and communicate that data back to Earth. We model the problem as a hierarchical task network, framing it as a utility maximization problem constrained by a strictly monotonically decreasing energy resource. We propose a novel deterministic planning framework that uses periodic replanning and sampling-based optimization to better handle model uncertainty and execution variation, while remaining computationally tractable. We demonstrate the efficacy of our framework through simulations of a Europa Lander concept mission in which our approach outperforms several baselines in utility maximization and robustness.

We introduce temporal multimodal multivariate learning, a new family of decision making models that can indirectly learn and transfer online information from simultaneous observations of a probability distribution with more than one peak or more than one outcome variable from one time stage to another. We approximate the posterior by sequentially removing additional uncertainties across different variables and time, based on data-physics driven correlation, to address a broader class of challenging time-dependent decision-making problems under uncertainty. Extensive experiments on real-world datasets ( i.e., urban traffic data and hurricane ensemble forecasting data) demonstrate the superior performance of the proposed targeted decision-making over the state-of-the-art baseline prediction methods across various settings.

Most existing works in Probabilistic Simple Temporal Networks (PSTNs) base their frameworks on well-defined, parametric probability distributions. Under the operational contexts of both strong and dynamic control, this paper addresses robustness measure of PSTNs, i.e. the execution success probability, where the probability distributions of the contingent durations are ordinary, not necessarily parametric, nor symmetric (e.g. histograms, PERT), as long as these can be discretized. In practice, one would obtain ordinary distributions by considering empirical observations (compiled as histograms), or even hand-drawn by field experts. In this new realm of PSTNs, we study and formally define concepts such as degree of weak/strong/dynamic controllability, robustness under a predefined dispatching protocol, and introduce the concept of PSTN expected execution utility. We also discuss the limitation of existing controllability levels, and propose new levels within dynamic controllability, to better characterize dynamic controllable PSTNs based on based practical complexity considerations. We propose a novel fixed-parameter pseudo-polynomial time computation method to obtain both the success probability and expected utility measures. We apply our computation method to various PSTN datasets, including realistic planetary exploration scenarios in the context of the Mars 2020 rover. Moreover, we propose additional original applications of the method.

Ocean Worlds represent one of the best chances for extra-terrestrial life in our solar system. A new mission concept must be developed to explore these oceans. This mission would require traversing the 10s of km thick icy shell and releasing a submersible into the ocean below. During the transit of the icy shell and the exploration of the ocean, the vehicle(s) would be out of contact with Earth for weeks or potentially months at a time. During this time the vehicle must have sufficient autonomy to locate and study scientific targets of interest. One such target of interest is hydrothermal venting. We have previously developed an autonomous nested search method to locate and investigate sources of hydrothermal venting by locating local maxima in hydrothermal vent emissions. In this work we demonstrate this approach on board an OceanServer Iver2 AUV in Chesapeake Bay, MD using simulated sensor data from a hydrothermal plume model. This represents the first step towards the deployment of this approach in conditions analogous to those that we might expect on an Ocean World.

Increased space and ground sensing is enabling dramatic new measurements of a wide range of Earth Science and Applied Earth Science phenomena, including: volcanism, flooding, wildfires, weather, and many other phenomena. New challenges exist to rapidly assimilate available data and to optimize measurements (e.g. direct assets) to best observe these complex and dynamic spatiotemporal phenomena. Artificial Intelligence offers the potential to assist in data interpretation and resource allocation to best allocate sensing assets. We describe efforts to build and experiment with such “sensorweb” systems and offer some direction for the future sensorweb observation systems.

Most existing works in Probabilistic Simple Temporal Networks (PSTNs) base their frameworks on well-defined probability distributions. This paper addresses on PSTN Dynamic Controllability (DC) robustness measure, i.e. the execution success probability of a network under dynamic control. We consider PSTNs where the probability distributions of the contingent edges are ordinary distributed (e.g. non-parametric, non-symmetric). We introduce the concepts of dispatching protocol (DP) as well as DP-robustness, the probability of success under a predefined dynamic policy. We propose a fixed-parameter pseudo-polynomial time algorithm to compute the exact DP-robustness of any PSTN under NextFirst protocol, and apply to various PSTN datasets, including the real case of planetary exploration in the context of the Mars 2020 rover, and propose an original structural analysis.

In February 2017, the Earth Observing One (EO-1) satellite mission successfully completed sixteen years and three months of Earth imaging by its two unique instruments, the Hyperion and the Advanced Land Imager (ALI). Both instruments have served as prototypes for new orbital sensors. Hyperion has provided the only available global sample of the Earth's surface with: (i) passive optical mid-morning observations at moderate spatial resolution (30 m) to match the Landsat series; and (ii) spectral coverage over almost the full optical spectrum in 10 nm contiguous bands, in visible through shortwave infrared (VSWIR, 0.4-2.5 μm) wavelengths. Consequently, Hyperion is a heritage platform for future full-spectrum VSWIR orbital spectrometers, including the German mission, EnMAP (2019), and the NASA pre-Phase A (yet unscheduled) mission, the Hyperspectral InfraRed Imager (HyspIRI), defined by the 2007 Decadal Survey conducted by the US National Research Council. We provide an overview of the mission's lifetime and Hyperion's scientific and application accomplishments, including calibration & validation activities, data quality evaluations during end of mission precession changes to the orbit and overpass time, and the development of a user-friendly science quality archive.

The Intelligent Payload Experiment (IPEX) is a CubeSat mission to flight validate technologies for onboard instrument processing and autonomous operations for NASA's Earth Science Technologies Office (ESTO). Specifically IPEX is to demonstrate onboard instrument processing and product generation technologies for the Intelligent Payload Module (IPM) of the proposed Hyperspectral Infra-red Imager (HyspIRI) mission concept. Many proposed future missions, including HyspIRI, are slated to produce enormous volumes of data requiring either significant communication advancements or data reduction techniques. IPEX demonstrates several technologies for onboard data reduction, such as computer vision, image analysis, image processing and in general demonstrates general operations autonomy. We conclude this paper with a number of lessons learned through operations of this technology demonstration mission on a novel platform for NASA.

Next-generation orbital imaging spectrometers will generate unprecedented data volumes, demanding new methods to optimize storage and communication resources. Here, we demonstrate that onboard analysis can excise cloud-contaminated scenes, reducing data volumes while preserving science return. We calculate optimal cloud-screening parameters in advance, exploiting stable radiometric calibration and foreknowledge of illumination and viewing geometry. Channel thresholds expressed in raw instrument values can be then uploaded to the sensor where they execute in real time at gigabit-per-second (Gb/s) data rates. We present a decision theoretic method for setting these instrument parameters and characterize performance using a continuous three-year image archive from the “classic” Airborne Visible/Infrared Imaging Spectrometer (AVIRIS-C). We then simulate the system onboard the International Space Station, where it provides factor-of-two improvements in data volume with negligible false positives. Finally, we describe a real-time demonstration onboard the AVIRIS Next Generation (AVIRIS-NG) flight platform during a recent science campaign. In this blind test, cloud screening is performed without error while keeping pace with instrument data rates.

Mapping localized spectral features in complex scenes demands sensitive and robust detection algorithms. This letter investigates two aspects of large images that can harm matched filter (MF) detection performance. First, multimodal backgrounds may violate normality assumptions. Second, outlier features can trigger false detections due to large projections onto the target vector. We review two state-of-the-art methods designed to resolve these issues. The background clustering of Funkmodels multimodal backgrounds, and the mixture-tuned (MT) MF of Boardman and Kruse addresses outliers. We demonstrate that combining the two methods has additional performance benefits. An MT cluster MF shows effective performance on simulated and airborne data sets. We demonstrate target detection scenarios that evidence multimodality, outliers, and their combination. These experiments explore the performance of the component algorithms and the practical circumstances that can favor a combined approach.

Imaging spectrometers are valuable instruments for space exploration, but their large data volumes limit the number of scenes that can be downlinked. Missions could improve science yield by acquiring surplus images and analyzing them onboard the spacecraft. This onboard analysis could generate surficial maps, summarizing scenes in a bandwidth-efficient manner to indicate data cubes that warrant a complete downlink. Additionally, onboard analysis could detect targets of opportunity and trigger immediate automated follow-up measurements by the spacecraft. Here, we report a first step toward these goals with demonstrations of fully automatic hyperspectral scene analysis, feature discovery, and mapping onboard the Earth Observing One (EO-1) spacecraft. We describe a series of overflights in which the spacecraft analyzes a scene and produces summary maps along with lists of salient features for prioritized downlink. The onboard system uses a superpixel endmember detection approach to identify compositionally distinctive features in each image. This procedure suits the limited computing resources of the EO-1 flight processor. It requires very little advance information about the anticipated spectral features, but the resulting surface composition maps agree well with canonical human interpretations. Identical spacecraft commands detect outlier spectral features in multiple scenarios having different constituents and imaging conditions.

Small satellites flight experiments, using CubeSats, are growing rapidly as a low-cost and quick turn-around platform for education, focused science observations, and advanced technology validation. This talk will describe recent investments by NASA's Earth Science Technology Office (ESTO) to rapidly advance the TRL of various hardware and software technologies targeted for future Earth Science Decadal Survey Instruments via the CubeSat platform.

No abstract available.

Current and proposed remote space missions, such as the proposed aerial exploration of Titan by an aerobot, often can collect more data than can be communicated back to Earth. Autonomous selective downlink algorithms can choose informative subsets of data to improve the science value of these bandwidth-limited transmissions. This requires statistical descriptors of the data that reflect very abstract and subtle distinctions in science content. We propose a metric learning strategy that teaches algorithms how best to cluster new data based on training examples supplied by domain scientists. We demonstrate that clustering informed by metric learning produces results that more closely match multiple scientists’ labelings of aerial data than do clusterings based on random or periodic sampling. A new metric-learning strategy accommodates training sets produced by multiple scientists with different and potentially inconsistent mission objectives. Our methods are fit for current spacecraft processors (e.g., RAD750) and would further benefit from more advanced spacecraft processor architectures, such as OPERA.

Orbital remote sensing provides a powerful way to efficiently survey targets such as the Earth and other planets and moons for features of interest. One such feature of astrobiological relevance is the presence of surface sulfur deposits. These deposits have been observed to be associated with microbial activity at the Borup Fiord glacial springs in Canada, a location that may provide an analogue to other icy environments such as Europa. This article evaluates automated classifiers for detecting sulfur in remote sensing observations by the hyperion spectrometer on the EO-1 spacecraft. We determined that a data-driven machine learning solution was needed because the sulfur could not be detected by simply matching observations to sulfur lab spectra. We also evaluated several methods (manual and automated) for identifying the most relevant attributes (spectral wavelengths) needed for successful sulfur detection. Our findings include (1) the Borup Fiord sulfur deposits were best modeled as containing two sub-populations: sulfur on ice and sulfur on rock; (2) as expected, classifiers using Gaussian kernels outperformed those based on linear kernels, and should be adopted when onboard computational constraints permit; and (3) Recursive Feature Elimination selected sensible and effective features for use in the computationally constrained environment onboard EO-1. This study helped guide the selection of algorithm parameters and configuration for the classification system currently operational on EO-1. Finally, we discuss implications for a similar onboard classification system for a future Europa orbiter.

We describe efforts to apply sensorweb technologies to the monitoring of forest fires in Thailand. In this approach, satellite data and ground reports are assimilated to assess the current state of the forest system in terms of forest fire risk, active fires, and likely progression of fires and smoke plumes. This current and projected assessment can then be used to actively direct sensors and assets to best acquire further information. This process operates continually with new data updating models of fire activity leading to further sensing and updating of models. As the fire activity is tracked, products such as active fire maps, burn scar severity maps, and alerts are automatically delivered to relevant parties. We describe the current state of the Thailand Fire Sensorweb which utilizes the MODIS-based FIRMS system to track active fires and trigger Earth Observing One / Advanced Land Imager to acquire imagery and produce active fire maps, burn scar severity maps, and alerts. We describe ongoing work to integrate additional sensor sources and generate additional products.

We describe efforts to integrate in-situ sensing, space-borne sensing, hydrological modeling, active control of sensing, and automatic data product generation to enhance monitoring and management of flooding. In our approach, broad coverage sensors and missions such as MODIS, TRMM, and weather satellite information and in-situ weather and river gauging information are all inputs to track flooding via river basin and sub-basin hydrological models. While these inputs can provide significant information as to the major flooding, targetable space measurements can provide better spatial resolution measurements of flooding extent. In order to leverage such assets we automatically task observations in response to automated analysis indications of major flooding. These new measurements are automatically processed and assimilated with the other flooding data. We describe our ongoing efforts to deploy this system to track major flooding events in Thailand.

This work investigates multiagent path planning in strong, dynamic currents using thousands of highly underactuated vehicles. We address the specific task of path planning for a global network of ocean-observing floats. These submersibles are typified by the Argo global network consisting of over 3000 sensor platforms. They can control their buoyancy to float at depth for data collection or rise to the surface for satellite communications. Currently, floats drift at a constant depth regardless of the local currents. However, accurate current forecasts have become available which present the possibility of intentionally controlling floats' motion by dynamically commanding them to linger at different depths. This project explores the use of these current predictions to direct float networks to some desired final formation or position. It presents multiple algorithms for such path optimization and demonstrates their advantage over the standard approach of constant-depth drifting.

This work addresses mission planning for autonomous underwater gliders based on predictions of an uncertain, time-varying current field. Glider submersibles are highly sensitive to prevailing currents so mission planners must account for ocean tides and eddies. Previous work in variable-current path planning assumes that current predictions are perfect, but in practice these forecasts may be inaccurate. Here we evaluate plan fragility using empirical tests on historical ocean forecasts for which followup data is available. We present methods for glider path planning and control in a time-varying current field. A case study scenario in the Southern California Bight uses current predictions drawn from the Regional Ocean Monitoring System (ROMS).

Future NASA missions will have instruments that generate enormous amounts of data. We describe an onboard processing mission concept for a possible Direct Broadcast capability for the HyspIRI mission - a mission under consideration for launch in the next decade carrying visible to short wave infrared (VSWIR) and thermal infrared (TIR) instruments. The VSWIR and TIR instruments will produce over 800 × 106bits per second of data however the Direct Broadcast downlink rate will be approximately 10×106bits per second, allowing only 1/80th of the data to be rapidly downlinked. Our onboard processing concept under development spectrally and spatially subsamples the data as well as generates science products onboard to enable return of key rapid response science and applications information despite limited downlink bandwidth. This rapid data delivery concept focuses on wildfires and volcanoes as primary applications but also has applications to vegetation, coastal, flooding, dust, and snow/ice applications.

Autonomous onboard processing of data allows rapid response to detections of dynamic, changing processes. Software that can detect volcanic eruptions from thermal emission has been used to retask the Earth Observing 1 spacecraft to obtain additional data of the eruption. Rapid transmission of these data to the ground, and the automatic processing of the data to generated images, estimates of eruption parameters and maps of thermal structure, has allowed these products to be delivered rapidly to volcanologists to aid them in assessing eruption risk and hazard. Such applications will enhance science return from future Earth-orbiting spacecraft and also from spacecraft exploring the Solar System, or beyond, which hope to image dynamic processes. Especially in the latter case, long communication times between the spacecraft and Earth exclude a rapid response to what may be a transient process - only using onboard autonomy can the spacecraft react quickly to such an event.

This paper describes work being performed under a NASA Earth Science Technology Office grant to develop a modular Sensor Web architecture based on Open Geospatial Consortium (OGC) standards, which enables discovery and generic tasking capability for sensors, both space-based and insitu. A series of increasingly complex demonstrations have been developed to prototype this architecture. Recent demonstrations have made use of the Hyperion and Advanced Land Imager instruments on the Earth Observing 1 (EO-1) satellite, the Moderate Resolution Imaging Spectroradiometer (MODIS) on the Terra and Aqua, the Advanced Space-borne Thermal Emission and Reflection Radiometer (ASTER) on the Terra satellite and the Wildfire sensor on the Ikhana Unmanned Aerial System (UAS). This Sensor Web was used in the recent Southern California fires during October 2007 to deliver key wildfire imagery to the San Diego county Emergency Operations Center (EOC) to assist emergency workers with situational awareness. Presently the team is in the process of prototyping the use of this sensor web for floods in collaboration with the International Federations of the Red Cross/Red Crescent for better flood disaster management. The paper will also describe a general overview of the modular architecture that has thus far been built and capabilities still needed to realize the full vision.

This paper describes the work being performed under a NASA Earth Science Technology Office (ESTO) Advanced Information System Technology (AIST) grant to develop a modular sensor web architecture which enables discovery and generic tasking capability for sensors, both space-based and in-situ. This effort seeks to demonstrate methods to facilitate interoperability and ease of use of a diverse set of sensors by hiding the details required to obtain sensor data, process the science data and deliver the science products. In particular, Web 2.0 and open geospatial consortium (OGC) Sensor Web Enablement (SWE) standard services are used. Thus, these capabilities serve to facilitate a user-centric approach to Global Earth Observing System of Systems (GEOSS). This work builds on previous sensor web efforts conducted at NASA/GSFC using the Earth Observing 1 (EO-1) and other satellites.

Analyzing data on-board a spacecraft as it is collected enables several advanced spacecraft capabilities, such as prioritizing observations to make the best use of limited bandwidth and reacting to dynamic events as they happen. In this paper, we describe how we addressed the unique challenges associated with on-board mining of data as it is collected: uncalibrated data, noisy observations, and severe limitations on computational and memory resources. The goal of this effort, which falls into the emerging application area of spacecraft-based data mining, was to study three specific science phenomena on Mars. Following previous work that used a linear support vector machine (SVM) on-board the Earth Observing 1 (EO-1)spacecraft, we developed three data mining techniques for use on-board the Mars Odyssey spacecraft. These methods range from simple thresholding to state-of-the-art reduced-set SVM technology. We tested these algorithms on archived data in a flight software testbed. We also describe a significant, serendipitous science discovery of this data mining effort: the confirmation of a water ice annulus around the north polar cap of Mars. We conclude with a discussion on lessons learned in developing algorithms for use on-board a spacecraft.

Acquisition of science in space applications is shifting from teleoperated gathering to an automated on-board analysis with improvements in the use of memory, CPU, bandwidth and data quality. In this paper, we describe algorithms to autonomously detect dust devils and clouds from a rover and summarize the results. The algorithms meet high hit-to-miss ratios and satisfy strict resource constraints. Both detectors have been uploaded to the Mars exploration rovers (MER). These are the first autonomous science processes in the rovers.

Typically, data collected by a spacecraft is downlinked to Earth and pre-processed before any analysis is performed. We have developed classifiers that can be used onboard a spacecraft to identify high priority data for downlink to Earth, providing a method for maximizing the use of a potentially bandwidth limited downlink channel. Onboard analysis can also enable rapid reaction to dynamic events, such as flooding, volcanic eruptions or sea ice break-up. Four classifiers were developed to identify cryosphere events using hyperspectral images. These classifiers include a manually constructed classifier, a Support Vector Machine (SVM), a Decision Tree and a classifier derived by searching over combinations of thresholded band ratios. Each of the classifiers was designed to run in the computationally constrained operating environment of the spacecraft. A set of scenes was hand-labeled to provide training and testing data. Performance results on the test data indicate that the SVM and manual classifiers outperformed the Decision Tree and band-ratio classifiers with the SVM yielding slightly better classifications than the manual classifier. The manual and SVM classifiers have been uploaded to the EO-1 spacecraft and have been running onboard the spacecraft for over a year. Results of the onboard analysis are used by the Autonomous Sciencecraft Experiment (ASE) of NASA’s New Millennium Program onboard EO-1 to automatically target the spacecraft to collect follow-on imagery. The software demonstrates the potential for future deep space missions to use onboard decision making to capture short-lived science events.

We describe a network of sensors linked by software and the Internet to an autonomous satellite observation response capability. This system of systems is designed with a flexible, modular, architecture to facilitate expansion in sensors, customization of trigger conditions, and customization of responses. This system has been used to implement a global surveillance program of science phenomena including: volcanoes, flooding, cryosphere events, and atmospheric phenomena. In this paper we describe the importance of the earth observing sensorweb application as well as overall architecture for the system of systems.

An Autonomous Science Agent has been flying onboard the Earth Observing One spacecraft since 2003. This software enables the spacecraft to autonomously detect and responds to science events occurring on the Earth such as volcanoes, flooding, and snow melt. This agent includes artificial intelligence software systems that perform science data analysis, deliberative planning, and run-time robust execution. This software is in routine use to fly the EO-1 mission. In this paper we discuss the architecture used to integrate these systems and lessons learned from its multi-year flight on EO-1.

Intelligent behavior for robotic agents requires a careful balance of fast reactions and deliberate consideration of long-term ramifications. The need for this balance is particularly acute in space applications, where hostile environments demand fast reactions, and remote locations dictate careful management of consumables that cannot be replenished. However, fast reactions typically require procedural representations with limited scope and handling long-term considerations in a general fashion is often computationally expensive. We describe three major areas for autonomous systems for space exploration: free-flying spacecraft, planetary rovers, and ground communications stations. In each of these broad applications areas, we identify operational considerations requiring rapid response and considerations of long-term ramifications. We describe these issues in the context of ongoing efforts to deploy autonomous systems using planning and task execution systems.

This paper discusses a proof-of-concept prototype for ground-based automatic generation of validated rover command sequences. This prototype is based on ASPEN (Automated Scheduling and Planning Environment). This Artificial Intelligence (AI) based planning and scheduling system will automatically generate a command sequence that will execute: within resource constraints and satisfy flight rules. An automated planning and scheduling system encodes rover design knowledge and uses the search and reasoning techniques to automatically generate low-level command sequences while respecting the rover operability constraints. This prototype planning system has been field-tested using the Rocky-7 rover at JPL, and will be field-tested on more complex rovers to prove its effectiveness before transferring the technology to flight operations for an upcoming NASA mission. The goal-driven commanding of planetary rovers greatly reduces the requirements for highly skilled rover engineering personnel. This in turn greatly reduces mission operations costs and permits a faster response to changes in rover states.

Work on generative planning systems has focused on two diverse approaches to plan construction. Hierarchical task network (HTN) planners build plans by successively refining high-level goals into lower-level activities. Operator-based planners employ means-end analysis to directly formulate plans consisting of low-level activities. While many have argued the universal dominance of a single approach, this paper presents an alternative view: that in different situations either may be most appropriate. To support this view, a number of advantages and disadvantages of these approaches are described in light of experiences in developing two real-world, fielded planning systems.

Automated planning and scheduling, including automated path planning, has been integrated with an Internet-based distributed operations system for planetary rover operations. The resulting prototype system enables faster generation of valid rover command sequences by a distributed planetary rover operations team. The Web Interface for Telescience provides Internet-based distributed collaboration, the automated scheduling and planning environment provides automated planning and scheduling, and an automated path planner provides path planning. The system was demonstrated on the Rocky7 research rover at JPL.

This paper describes the application of Artificial Intelligence planning techniques to the problem of antenna track plan generation for a NASA Deep Space Communications Station. Me described system enables an antenna communications station to automatically respond to a set of tracking goals by correctly configuring the appropriate hardware and software to provide the requested communication services. To perform this task, the Automated Scheduling and Planning Environment (ASPEN) has been applied to automatically produce antenna trucking plans that are tailored to support a set of input goals. In this paper, we describe the antenna automation problem, the ASPEN planning and scheduling system, how ASPEN is used to generate antenna track plans, the results of several technology demonstrations, and future work utilizing dynamic planning technology.

In recent times, improvements in imaging technology have made available an incredible array of information in image format. While powerful and sophisticated image processing software tools are available to prepare and analyze the data, these tools are complex and cumbersome, requiring significant expertise to properly operate. Thus, in order to extract (e.g., mine or analyze) useful information from the data, a user (in our case a scientist) often must possess both significant science and image processing expertise. This article describes the use of artificial intelligence (AI) planning techniques to represent scientific, image processing and software tool knowledge to automate knowledge discovery and data mining (e.g., science data analysis) of large image databases. In particular, we describe two fielded systems. The Multimission VICAR Planner (MVP) which has been deployed for since 1995 and is currently supporting science product generation for the Galileo mission. MVP has reduced time to fill certain classes of requests from 4 h to 15 min. The Automated SAR Image Processing system (ASIP) was deployed at the Department of Geology at Arizona State University in 1997 to support aeolian science analysis of synthetic aperture radar images. ASIP reduces the number of manual inputs in science product generation by ten-fold.

This paper considers the problem of learning the ranking of a set of stochastic alternatives based upon incomplete information (i.e., a limited number of samples). We describe a system that, at each decision cycle, outputs either a complete ordering on the hypotheses or decides to gather additional information (i.e., observations) at some cost. The ranking problem is a generalization of the previously studied hypothesis selection problem - in selection, an algorithm must select the single best hypothesis, while in ranking, an algorithm must order all the hypotheses. The central problem we address is achieving the desired ranking quality while minimizing the cost of acquiring additional samples. We describe two algorithms for hypothesis ranking and their application for the probably approximately correct (PAC) and expected loss (EL) learning criteria. Empirical results are provided to demonstrate the effectiveness of these ranking procedures on both synthetic and real-world datasets.

This paper describes a hierarchical scheduling, planning, control, and execution monitoring architecture for automating operations of the Deep Space Network, a worldwide network of communications antennas. We describe the network automation problem and current mode of operations. We then describe a three layer hierarchical architecture for automating network operations. In particular we describe the notion of plan/schedule generation, execution, and revision at each level. Finally, we describe the current state of deployment for each segment of the automation architecture.

Although most scheduling problems are NP-hard, domain specific techniques perform well in practice but are quite expensive to construct. In adaptive problem-solving solving, domain specific knowledge is acquired automatically for a general problem solver with a flexible control architecture. In this approach, a learning system explores a space of possible heuristic methods for one well-suited to the eccentricities of the given domain and problem distribution. In this article, we discuss an application of the approach to scheduling satellite communications. Using problem distributions based on actual mission requirements, our approach identifies strategies that not only decrease the amount of CPU time required to produce schedules, but also increase the percentage of problems that are solvable within computational resource limitations.

Describes the Multimission VICAR Planner (MVP): an AI planning system which uses knowledge about image processing steps and their requirements to construct executable image processing scripts to support high-level science requests made to the Jet Propulsion Laboratory (JPL) Multimission Image Processing Subsystem (MIPS). This article describes a general AI planning approach to automation and application of the approach to a specific area of image processing for planetary science applications involving radiometric correction, color triplet reconstruction, and mosaicing in which the MVP system significantly reduces the amount of effort required by image processing experts to fill a typical request.

This paper considers the decision-making problem of selecting a strategy from a set of alternatives on the basis of incomplete information (e.g. a finite number of observations). At any time the system can adopt a particular strategy or decide to gather additional information at some cost. Balancing the expected utility of the new information against the cost of acquiring the information is the central problem that the authors address. In the authors' approach, the cost and utility of applying a particular strategy to a given problem are represented as random variables from a parametric distribution. By observing the performance of each strategy on a randomly selected sample of problems, one can use parameter estimation techniques to infer statistical models of performance on the general population of problems. These models can then be used to estimate: (1) the utility and cost of acquiring additional information and (2) the desirability of selecting a particular strategy from a set of choices. Empirical results are presented that demonstrate the effectiveness of the hypothesis evaluation techniques for tuning system parameters in a NASA antenna scheduling application.>

This paper describes the Multimission VICAR Planner (MVP) system, which uses models of the image processing programs to automatically construct executable image processing procedures to fill science requests made to the JPL Multimission Image Processing Laboratory (MIPL). The MVP system allows the user to specify the image processing requirements in terms of the various types of correction required. Given this information, MVP which use Artificial Intelligence (AI) Planning techniques to derive unspecified required processing steps and determines appropriate image processing programs and parameters to achieve the specified image processing goals. This information is output as an executable image processing program which can then be executed to fill the processing request.>