The Airborne Visible/Infrared Imaging Spectrometer-4 (AVIRIS-4) represents the next generation in the series of airborne imaging spectrometers built by NASA JPL. Commissioned by the Swiss ARES research consortium, AVIRIS-4 is geared towards delivering cutting-edge imaging spectroscopy data for scientific and practical applications as a replacement for its predecessor APEX. AVIRIS-4 is based on a Dyson-type imaging spectrometer design, also employed by NASA-operated AVIRIS-3 and EMIT, and integrates a scaled two-mirror telescope housed in a compact vacuum vessel. This enables airborne measurements in unpressurized aircraft at altitudes ranging from 500 m to 7620 m, achieving image resolutions between 0.3 and 4.5 m with a field of view of 40.2° in 1241 spatial pixels. AVIRIS-4 surpasses previous state-of-the-art sensor heads in signal-to-noise ratio performance and features a spectral range of 375 to 2504 nm and 7.4 nm spectral sampling. The operation, data capture and mission control hardware as well as the calibration and data processing software is developed by UZH, EPFL, and ZHAW. This paper outlines the design and calibration strategies implemented in AVIRIS-4’s development and highlights its performance during its first year of operation in 2024.

The Airborne Visible / Infrared Imaging Spectrometer-3 (AVIRIS-3) instrument is the newest member of NASA’s AVIRIS airborne imaging spectrometer family. A Dyson pushbroom spectrometer similar to the satellite-based Earth Mineral Dust Source Investigation (EMIT) instrument, AVIRIS-3 offers higher throughput, a higher signal-to-noise ratio, and a more compact form factor than previous AVIRIS generations. AVIRIS-3 relies upon in-flight data to create updates to the wavelength, flatfield, and radiometric calibration using features from Earth’s surface and atmosphere. This technique of applying calibration updates derived from in-flight, solar-illuminated Earth scenes will be used in NASA’s upcoming Surface Biology and Geology (SBG) mission. We discuss the calibration method and first results from the first year of flights from AVIRIS-3.

Imaging spectrometers like EMIT and AVIRIS-3 have similar instrument parameters and methane and CO2 mapping capability that enables direct attribution of observed plumes to the oil and gas, waste, and agriculture sectors. Onboard the International Space Station, EMIT can constrain methane and CO2 emissions over a significant portion of the Earth’s surface. With improved spatial resolution, the airborne AVIRIS-3 instrument enables quantification of smaller emissions sources that compliment EMIT observations from space. These instruments offer the potential to improve understanding of greenhouse gas budgets, inform mitigation strategies, and in some cases lead to voluntary mitigation.

Recent advances in remote imaging spectroscopy have increased its utility for detecting and quantifying greenhouse gas emissions. In fact, multiple airborne and space-based instruments are actively used to estimate methane emissions. Many of these measurements are made using matched-filter-based detection and estimation algorithms. In this work, we present new methods for quantifying and improving the accuracy and uncertainty of these algorithms. Two new metrics are proposed that capture the biases and uncertainties in gas quantity measurements stemming from local surface and atmospheric variation, observation and solar geometries, and sensor noise. We show that one of these, termed the “sensitivity,” can be used to correct the bias in the gas concentration length estimates due to variable atmospheres and backgrounds, reducing the estimator’s root mean squared (rms) error in spectra that deviate from the mean spectrum. The second, termed the “uncertainty,” represents the bias-removed statistical uncertainty in the corrected estimator. Expressions for the rms error both with and without the correction are provided along with interpretation to help quantify the various noise sources. The utility of the metrics is demonstrated using data from the Earth Surface Mineral Dust Source Investigation (EMIT) imaging spectrometer currently collecting Earth observations onboard the International Space Station (ISS). The EMIT data also demonstrates the potential accuracy increase afforded by the sensitivity correction over variable surface types. These metrics and their concomitant estimator accuracy increases could prove valuable for future work in quantifying gas source emission rates and their uncertainties, instrument design, and machine learning-based detection methods.

The joint retrieval of surface reflectances and atmospheric parameters in visible/short-wave infrared (VSWIR) imaging spectroscopy is a computationally challenging high-dimensional problem. Using NASA’s Surface Biology and Geology mission (SBG) as the motivational context, the uncertainty associated with the retrievals is crucial for further application of the retrieved results for environmental applications. Although Markov chain Monte Carlo (MCMC) is a Bayesian method ideal for uncertainty quantification (UQ), the full-dimensional implementation of MCMC for the retrieval is computationally intractable. In this work, we developed a block Metropolis MCMC algorithm for the high-dimensional VSWIR surface reflectance retrieval that leverages conditional linearity structure in the forward radiative transfer model to enable tractable fully Bayesian computation. We use the posterior distribution from this MCMC algorithm to assess the limitations of optimal estimation (OE), the state-of-the-art Bayesian algorithm in operational retrievals which is more computationally efficient but uses a Gaussian approximation to characterize the posterior. Analyzing the differences in the posterior computed by each method, the MCMC algorithm was shown to give more physically sensible results and reveals the non-Gaussian structure of the posterior, specifically in the atmospheric aerosol optical depth (AOD) parameter and the low-wavelength surface reflectances.

Radiative forcing by small dust particles deposited on snow plays a key role in climate change. Detection and quantification of these particles is essential for predicting melt rates, and assessing the associated impacts on Earth’s climate. NASA’s Earth Surface Mineral Dust Source Investigation (EMIT) aims to improve our understanding of the Earth’s dust source and sink regions. The latter include snow surfaces in topographically challenging mountainous terrain. We present estimated snow reflectance, dust concentration, and radiative forcing from a new retrieval framework, and highlight their sensitivity to topographic characteristics. These findings will be essential for updating snow melt and climate models, but also for the conception of retrieval algorithms for upcoming global spaceborne imaging spectroscopy missions, including NASA’s Surface Biology and Geology (SBG).

Accurate atmospheric correction is critical for remote imaging spectroscopy of Earth’s surface. We present an overview of recent advances in the open-source atmospheric correction codebase ISOFIT that is designed for the application to NASA’s Earth Surface Mineral Dust Source Investigation (EMIT). The approach uses Bayesian Maximum A Posteriori (MAP) inference to simultaneously solve for the most likely surface and atmospheric state given a particular measurement. The main advantage is a rigorous uncertainty accounting and propagation, and the flexibility to incorporate diverse radiative transfer modeling assumptions. Released in 2018, the codebase has been used for a multitude of validation studies at several terrestrial and aquatic field sites, and is the current operational retrieval algorithm for the EMIT, AVIRIS-NG, and AVIRIS-C instruments.

We present the preliminary results of a study conducted by the Commonwealth Scientific and Industrial Research Organisation (CSIRO, Australia’s national science agency) and NASA’s Jet Propulsion Laboratory (JPL) to demonstrate the utility of imaging spectroscopy from space to provide actionable information on water quality and aquatic ecosystems for Australia and Western USA. Mission requirements are derived from three key application objectives: potentially harmful algal blooms and nutrient pollution, invasive aquatic vegetation, and coral reef habitat benthic cover. The proposed AquaSat-1 instrument is a state-of-the-art visible to near-infrared (VNIR) Dyson imaging spectrometer, which builds on over 30 years of imaging spectroscopy development at JPL.

Pursuant to recommendations by the National Academies of Science, Engineering and Medicine's Earth Science Decadal Survey [1], the National Aeronautics and Space Administration (NASA) has announced the development of an Earth System Observatory (ESO), a series of missions designed to observe processes across the Earth's interior, surface and atmosphere. A key component of this system is the Surface Biology and Geology (SBG) investigation. SBG will measure the composition and properties of Earth's land, inland waters, and coastal oceans. The notional architecture consists of multiple spacecraft slated for launch in the 2027–2028 timeframe (Figure 1). Target science questions and geophysical variables span diverse disciplines including terrestrial and aquatic ecology, geology, vulcanology, hydrology and cryospheric sciences (Figure 2). Beyond simply measuring geophysical variables for each discipline, SBG will provide information about the links between the different domains, enabling a more comprehensive understanding of the Earth as a connected system. SBG measurements will also benefit a wide range of societal applications including agriculture, terrestrial and aquatic biodiversity, natural hazards, public health, and management of water and other natural resources [2]. SBG will also coordinate measurements, data products, and analyses with other ESO elements to deliver an integrated Earth System perspective of Earth and its changing climate.

On Jan. 5, 2018, at the request of NASA, the National Oceanic and Atmospheric Administration (NOAA) and the U.S. Geological Survey (USGS), the Committee on the Decadal Survey for Earth Science and Applications from Space (ESAS) of the National Academies of Sciences, Engineering and Medicine (NASEM) Space Studies Board, Division on Engineering and Physical Sciences released the 2017 Decadal Survey, “Thriving on Our Changing Planet: A Decadal Strategy for Earth Observations from Space” [1]. The 700-page document is the second such Earth sciences survey produced by NASEM. The first, “Earth Science and Applications from Space: National Imperatives for the Next Decade and Beyond,” was released in 2007. The 2018 study designated a global “Surface Biology and Geology” (SBG) investigation that would include both imaging spectroscopy and thermal infrared observations [1]. This suite of measurements would address a wide range of global science questions. Its themes include: flows of energy, carbon, water, and nutrients sustaining terrestrial and marine ecosystems; the variability of the land surface and the fluxes of water and energy; inventory of the world's volcanoes, and the composition and temperature of volcanic products immediately following eruptions; other natural hazards including wildfires; snow accumulation and melt; water balance from the headwaters to the continent; land and water use effects on evapotranspiration; functional traits and diversity of terrestrial and aquatic ecosystems and vegetation; and more. Figure 1 shows example spectra from these surfaces, illustrating the enormous diversity of scene content that would be observed. Tables 1 and 2 show examples of the core and higher-level products that the SBG mission would produce.

The NASA Earth Surface Mineral Dust Source Investigation (EMIT) will use imaging spectroscopy to measure the mineral composition of the Earth's arid land regions from the International Space Station. The new directly observed surface composition products will be validated and used to initialize state-of-the-art Earth System Models to improve constraint of the sign and magnitude of dust-related radiative forcing at regional and global scales as well as predict the increase or decrease of available dust sources under future climate scenarios. EMIT is a NASA Earth Venture Instrument (EVI) Mission planned to launch in 2022. The instrument is a state-of-the-art optically fast Dyson imaging spectrometer that covers the spectral range from the visible to the short wavelength infrared (VSWIR). EMIT measurements and products will be openly available to the full science and applications communities for the range of additional investigations they enable.

Remote sensing instruments, both aircraft and on-orbit platforms, undergo extensive laboratory calibrations to determine their geometric, spectral, and radiometric responses. Additional in-flight radiometric calibrations can be performed using well-characterized earth targets. The Fire Influence on Regional to Global Environments and Air Quality (FIREX-AQ) campaign provided such an opportunity when the ER-2 aircraft overflew Railroad Valley on August 13 and 15, 2019. Surface reflectances were available from the August 4, 2019 field team and from the Radiometric Calibration Network (RadCalNet) portal, and spectral aerosol optical depths from an on-site AERosol RObotic NETwork (AERONET) sunphotometer. The Enhanced MODIS Airborne Simulator (eMAS), the Airborne Multiangle SpectroPolarimetric Imager (AirMSPI), and the “Classic” Airborne Visible/Infrared Imaging Spectrometer (AVIRIS-C) sensors individually performed a vicarious calibration using their respective methodologies and selection of input parameters. A comparison of the at-sensor radiances predicted from these independent analyses highlights some of the uncertainties in the inputs, including choice of solar irradiance model. Although good agreement, within 5%, is found at visible wavelengths, difference can be as large as 15% in the shortwave infrared (SWIR). This highlights the need for the remote sensing community to agree upon a standard solar model, to remove sensor-to-sensor biases derived from in-flight calibrations.

Earth and planetary sciences often rely upon the detailed examination of spectroscopic data for rock and mineral identification. This typically requires the collection of high resolution spectroscopic measurements. However, they tend to be scarce, as compared to low resolution remote spectra. This work addresses the problem of inferring high-resolution mineral spectroscopic measurements from low resolution observations using probability models. We present the Deep Gaussian Conditional Model, a neural network that performs probabilistic super resolution via maximum likelihood estimation. It also provides insight into learned correlations between measurements and spectroscopic features, allowing for the tractability and interpretability that scientists often require for mineral identification. Experiments using remote spectroscopic data demonstrate that our method compares favorably to other analogous probabilistic methods. Finally, we show and discuss how our method provides human-interpretable results, making it a compelling analysis tool for scientists.

Maintaining high levels of productivity for planetary rover missions is very difficult due to limited communication and heavy reliance on ground control. There is a need for autonomy that enables more adaptive and efficient actions based on real-time information. This paper presents an autonomous mapping and exploration approach for planetary rovers. We first describe a machine learning model that actively combines remote and rover measurements for mapping. We focus on spectroscopic data because they are commonly used to investigate surface composition. We then incorporate notions from information theory and non-myopic path planning to improve exploration productivity. Finally, we demonstrate the feasibility and successful performance of our approach via spectroscopic investigations of Cuprite, Nevada; a well-studied region of mineralogical and geological interest. We first perform a detailed analysis in simulations, and then validate those results with an actual rover in the field in Nevada.

The Earth Surface Mineral Dust Source Investigation (EMIT) has been selected as an Earth Venture mission by NASA in 2018. EMIT will measure surface mineralogy in the arid land dust source regions of the Earth using visible to short wavelength infrared (VSWIR) imaging spectroscopy. These new measurements will be used to initialize advanced Earth System Models (ESM) to reduce uncertainty in the understanding of the impact of mineral dust aerosols on radiative forcing. The EMIT imaging spectrometer is planned to operate on the International Space Station (ISS) beginning in 2021.

The last five years have seen dramatic growth in the use of Visible Shortwave Infrared (VSWIR) and Thermal Infrared (TIR) imaging spectrometers to detect and characterize greenhouse methane sources. Targets include: dairy and animal husbandry emissions; landfills; fossil fuel extraction, storage, and transport infrastructure; geologic sources; natural emissions associated with sensitive arctic ecosystems; and more. These campaigns have resulted in significant new discoveries and advances in our understanding of the North American CH4 budget. Recent algorithm improvements have been critical for these campaigns, enabling robust statistical CH4measurement, fully-automated image-space source identification, and quantification of flux. Here we survey recent campaigns by NASA's Next Generation Airborne Visible Infrared Imaging Spectrometer (AVIRIS-NG) and NASA's Hyperspectral Thermal Emission Spectrometer (HyTES). We describe their algorithmic advances and major findings.

The National Academies Decadal Survey for Earth Science recommended that NASA pursue global imaging spectroscopy and thermal infrared measurements in the coming decade [1]. Both measurements would offer repeat coverage on approximately five-day to biweekly cadence, with comprehensive coverage of the globe's coastal and terrestrial area. This would be an unprecedented volume of data with the potential to transform remote sensing practice. To address this recommendation, NASA has sponsored a concept study by NASA research centers and associated university partners (https://sbg.jpl.nasa.gov). This study is determining a family of architecture options - including launch vehicle, spacecraft, instrument, and suborbital components - that could address the Decadal Survey objectives. The architecture study is driven by science needs and builds on input of the research community. As of this writing, the study is entering a phase in which a large field of system possibilities is pared down to a representative handful for an ultimate decision by NASA.

The strong radiative forcing by atmospheric methane has stimulated interest in identifying natural and anthropogenic sources of this potent greenhouse gas. Point sources are important targets for quantification, and anthropogenic targets have the potential for emissions reduction. Methane point-source plume detection and concentration retrieval have been previously demonstrated using data from the Airborne Visible InfraRed Imaging Spectrometer-Next Generation (AVIRIS-NG). Current quantitative methods have tradeoffs between computational requirements and retrieval accuracy, creating obstacles for processing real-time data or large data sets from flight campaigns. We present a new computationally efficient algorithm that applies sparsity and an albedo correction to matched the filter retrieval of trace gas concentration path length. The new algorithm was tested using the AVIRIS-NG data acquired over several point-source plumes in Ahmedabad, India. The algorithm was validated using the simulated AVIRIS-NG data, including synthetic plumes of known methane concentration. Sparsity and albedo correction together reduced the root-mean-squared error of retrieved methane concentration-path length enhancement by 60.7% compared with a previous robust matched filter method. Background noise was reduced by a factor of 2.64. The new algorithm was able to process the entire 300 flight line 2016 AVIRIS-NG India campaign in just over 8 h on a desktop computer with GPU acceleration.

This paper presents the adaptive reflectance geometric correction (ARGC), a bidirectional reflectance distribution function (BRDF) correction algorithm to address intensity gradients across remotely sensed images. The ARGC is developed and tested on data from the Airborne Visible/Infrared Imaging Spectrometer-Next Generation (AVIRIS-NG) collected over Louisiana's Atchafalaya River Delta, an area of complex wetland vegetation and waterbodies suited to AVIRIS-NG's fine spatial and spectral resolutions. Changing view and solar geometry, in conjunction with surfaces' anisotropic properties, impact a scene's observed reflectance. As traditional BRDF corrections may not be appropriate for wetland environments that have distinctive vegetation and hydrologic structures, more flexible functional corrections are shown to improve results. We compared two existing methods and the ARGC. The first method fits a quadratic function over image column averages, and the second is based on the inversion of the Ross Thick and Li Sparse kernels. Building upon the principles of these methods, the ARGC uses a multiple regression-based BRDF correction whereby the image's solar and view geometric descriptors form the independent variables. Each BRDF correction method was applied to the set of six partially overlapping AVIRIS-NG scenes. Assuming the actual surface reflectance of a given land cover type is independent of geometry, we used adjacent images' overlapping regions to quantitatively assess each correction method's efficacy. The ARGC produced the lowest overall root-mean-square difference and the lowest overlap mean absolute difference across the vast majority of bands. The ARGC is proposed as a practical new BRDF correction option for investigators using AVIRIS-NG data.

Conventional VSWIR imaging spectrometer atmospheric correction evolved from multi-band approaches and generally does not exploit the full spectral measurement. We hypothesize that a pure spectroscopic approach can improve atmospheric inversion accuracy to minimize regional biases in global-scale investigations. Such techniques are pervasive in atmospheric remote sounding disciplines, where Optimal Estimation (OE) retrieval theory (Rodgers et al., 2000) inverts a radiance spectrum to recover a consistent physical model incorporating, for example, aerosol and H2O, the interactions between scattering, absorption, and the coupling between surface reflectance and atmosphere. This enables a statistically rigorous treatment of uncertainty with the potential to recover information on spectrally-broad signals. Here we demonstrate a proof of concept that overcomes the primary computational roadblock of these methods: fast execution of line-by-line Radiative Transfer (RT) models. Neural Network (NN) emulation can replicate the results of the MODTRAN 6.0 line-by-line A-band model to high accuracy and a five order of magnitude improvement in execution time. This demonstrates potential for OE to provide significant advances in the accuracy and modeling power of VSWIR atmospheric correction.

There is a pressing need to globally inventory and assess coastal and inland aquatic habitats; extremely valuable and productive regions that are vulnerable to global anthropogenic pressures and climatic change. Basic information about sessile communities (wetlands, coral reefs, and sea grasses) includes mapping their extent and distribution, which can be gleaned from spectral surface reflectance imagery at high spatial resolution, but moderate temporal resolution. Moderate to high temporal resolution is also required for detailed observations of sessile community change (e.g., phenology, disturbance) and high temporal resolution is required for environmental changes in the surrounding water, phytoplankton concentration and composition, and concentrations of sediment or chromophoric dissolved organic matter (CDOM). Current and upcoming satellite missions and technology could meet spatial and spectral challenges. Multiple orbiting and airborne platforms, along with a network of in situ measurements, could provide a more complete picture of how these vital resources are changing. This paper provides an overview of these resources.

Planetary exploration involves frequent scientific reformulation and replanning. It is limited by communication constraints and to overcome this limitation, this paper formulates the process as a collaboration in which the human scientist and the robot work together to fill in gaps in knowledge to make discoveries. It introduces the science hypothesis map as the probabilistic structure in which scientists initially describe their abstract beliefs and hypotheses, and in which the state of this belief evolves as the robot makes raw measurements. It discusses how to incorporate path planning for maximizing scientific information gain, which is efficiently computed. As proof of concept, this paper describes a geologic exploration problem where a robot uses a spectrometer to infer the geologic composition of different regions in a mining district at Cuprite, Nevada. It shows that the science hypothesis map can infer geologic units with high accuracy, and that exploration using information gain-based path planning has better performance than exploration with conventional science-blind algorithms.

Adaptive exploration uses active learning principles to improve the efficiency of autonomous robotic surveys. This work considers an important and understudied aspect of autonomous exploration: in situ validation of remote sensing measurements. We focus on high- dimensional sensor data with a specific case study of spectroscopic mapping. A field robot refines an orbital image by measuring the surface at many wavelengths. We introduce a new objective function based on spectral unmixing that seeks pure spectral signatures to accurately model diluted remote signals. This objective reflects physical properties of the multi-wavelength data. The rover visits locations that jointly improve its model of the environment while satisfying time and energy constraints. We simulate exploration using alternative planning approaches, and show proof of concept results with the canonical spectroscopic map of a mining district in Cuprite, Nevada.

The Intelligent Payload Experiment (IPEX) is a CubeSat mission to flight validate technologies for onboard instrument processing and autonomous operations for NASA's Earth Science Technologies Office (ESTO). Specifically IPEX is to demonstrate onboard instrument processing and product generation technologies for the Intelligent Payload Module (IPM) of the proposed Hyperspectral Infra-red Imager (HyspIRI) mission concept. Many proposed future missions, including HyspIRI, are slated to produce enormous volumes of data requiring either significant communication advancements or data reduction techniques. IPEX demonstrates several technologies for onboard data reduction, such as computer vision, image analysis, image processing and in general demonstrates general operations autonomy. We conclude this paper with a number of lessons learned through operations of this technology demonstration mission on a novel platform for NASA.

We demonstrate real-time model-based atmospheric correction onboard the Next Generation Airborne Visible/Infrared Imaging Spectrometer. We achieve a reduction in processing time from hours or days to seconds by modifying a standard physics-based atmospheric correction algorithm to support real-time execution. We achieved this reduction by modifying the physics-based ATmospheric REMoval algorithm to leverage a large lookup table of precomputed scattering and transmission coefficients, indexed by parameters specifying the aircraft operating conditions at capture time. Interpolation among the precomputed coefficients allows surface reflectance retrieval at the sensor acquisition rate of 500 Mb/s. Our system produced science-quality reflectance products during over 30 test flights and, to our knowledge, is the first reported demonstration of real-time model-driven visible shortwave infrared atmospheric correction onboard an aircraft.

Next-generation orbital imaging spectrometers will generate unprecedented data volumes, demanding new methods to optimize storage and communication resources. Here, we demonstrate that onboard analysis can excise cloud-contaminated scenes, reducing data volumes while preserving science return. We calculate optimal cloud-screening parameters in advance, exploiting stable radiometric calibration and foreknowledge of illumination and viewing geometry. Channel thresholds expressed in raw instrument values can be then uploaded to the sensor where they execute in real time at gigabit-per-second (Gb/s) data rates. We present a decision theoretic method for setting these instrument parameters and characterize performance using a continuous three-year image archive from the “classic” Airborne Visible/Infrared Imaging Spectrometer (AVIRIS-C). We then simulate the system onboard the International Space Station, where it provides factor-of-two improvements in data volume with negligible false positives. Finally, we describe a real-time demonstration onboard the AVIRIS Next Generation (AVIRIS-NG) flight platform during a recent science campaign. In this blind test, cloud screening is performed without error while keeping pace with instrument data rates.

In the era of large scientific data sets, there is an urgent need for methods to automatically prioritize data for review. At the same time, for any automated method to be adopted by scientists, it must make decisions that they can understand and trust. In this paper, we propose Discovery through Eigenbasis Modeling of Uninteresting Data (DEMUD), which uses principal components modeling and reconstruction error to prioritize data. DEMUD’s major advance is to offer domain-specific explanations for its prioritizations. We evaluated DEMUD’s ability to quickly identify diverse items of interest and the value of the explanations it provides. We found that DEMUD performs as well or better than existing class discovery methods and provides, uniquely, the first explanations for why those items are of interest. Further, in collaborations with planetary scientists, we found that DEMUD (1) quickly identifies very rare items of scientific value, (2) maintains high diversity in its selections, and (3) provides explanations that greatly improve human classification accuracy.

We address the problem of adaptive information-optimal data collection in time series. Here a remote sensor or explorer agent throttles its sampling rate in order to track anomalous events while obeying constraints on time and power. This problem is challenging because the agent has limited visibility - all collected datapoints lie in the past, but its resource allocation decisions require predicting far into the future. Our solution is to continually fit a Gaussian process model to the latest data and optimize the sampling plan on line to maximize information gain. We compare the performance characteristics of stationary and nonstationary Gaussian process models. We also describe an application based on geologic analysis during planetary rover exploration. Here adaptive sampling can improve coverage of localized anomalies and potentially benefit mission science yield of long autonomous traverses.

Communication blackouts and latency are significant bottlenecks for planetary surface exploration; rovers cannot typically communicate during long traverses, so human operators cannot respond to unanticipated science targets discovered along the route. Targeted data collection by point spectrometers or high-resolution imagery requires precise aim, so it typically happens under human supervision during the start of each command cycle, directed at known targets in the local field of view. Spacecraft can overcome this limitation using onboard science data analysis to perform autonomous instrument targeting. Two critical target selection capabilities are the ability to target priority features of a known geologic class, and the ability to target anomalous surfaces that are unlike anything seen before. This work addresses both challenges using probabilistic surface classification in traverse images. We first describe a method for targeting known classes in the presence of high measurement cost that is typical for power- and time-constrained rover operations. We demonstrate a Bayesian approach that abstains from uncertain classifications to significantly improve the precision of geologic surface classifications. Our results show a significant increase in classification performance, including a seven-fold decrease in misclassification rate for our random forest classifier. We then take advantage of these classifications and learned scene context in order to train a semi-supervised novelty detector. Operators can train the novelty detection to ignore known content from previous scenes, a critical requirement for multi-day rover operations. By making use of prior scene knowledge we find nearly double the number of abnormal features detected over comparable algorithms. We evaluate both of these techniques on a set of images acquired during field expeditions in the Mojave Desert.

Mapping localized spectral features in complex scenes demands sensitive and robust detection algorithms. This letter investigates two aspects of large images that can harm matched filter (MF) detection performance. First, multimodal backgrounds may violate normality assumptions. Second, outlier features can trigger false detections due to large projections onto the target vector. We review two state-of-the-art methods designed to resolve these issues. The background clustering of Funkmodels multimodal backgrounds, and the mixture-tuned (MT) MF of Boardman and Kruse addresses outliers. We demonstrate that combining the two methods has additional performance benefits. An MT cluster MF shows effective performance on simulated and airborne data sets. We demonstrate target detection scenarios that evidence multimodality, outliers, and their combination. These experiments explore the performance of the component algorithms and the practical circumstances that can favor a combined approach.

Imaging spectrometers are valuable instruments for space exploration, but their large data volumes limit the number of scenes that can be downlinked. Missions could improve science yield by acquiring surplus images and analyzing them onboard the spacecraft. This onboard analysis could generate surficial maps, summarizing scenes in a bandwidth-efficient manner to indicate data cubes that warrant a complete downlink. Additionally, onboard analysis could detect targets of opportunity and trigger immediate automated follow-up measurements by the spacecraft. Here, we report a first step toward these goals with demonstrations of fully automatic hyperspectral scene analysis, feature discovery, and mapping onboard the Earth Observing One (EO-1) spacecraft. We describe a series of overflights in which the spacecraft analyzes a scene and produces summary maps along with lists of salient features for prioritized downlink. The onboard system uses a superpixel endmember detection approach to identify compositionally distinctive features in each image. This procedure suits the limited computing resources of the EO-1 flight processor. It requires very little advance information about the anticipated spectral features, but the resulting surface composition maps agree well with canonical human interpretations. Identical spacecraft commands detect outlier spectral features in multiple scenarios having different constituents and imaging conditions.

We present a method to improve subpixel signal detection in airborne or orbital image sequences. The proposed technique recognizes stable interest point features in multiple overlapping frames. It estimates motion between consecutive frames and tracks candidate detections over time. The final detection decision combines signal strengths from multiple views to improve sensitivity. The algorithm is computationally tractable for real-time use on autonomous robotic platforms and spacecraft. Additionally, the interest points enable image-relative localization, obviating the need to transmit the entire image and reducing transmission bandwidth requirements by one or more orders of magnitude. This permits higher acquisition rates and potentially improved coverage for remote monitoring. Ground-based systems can reconstruct absolute positions from landmarks without measurements of sensor pose. We demonstrate the algorithm using airborne 4- μm imagery from multiple overpasses of a controlled wildfire.

The Autonomous Exploration for Gathering Increased Science (AEGIS) system enables automated data collection by planetary rovers. AEGIS software was uploaded to the Mars Exploration Rover (MER) mission’s Opportunity rover in December 2009 and has successfully demonstrated automated onboard targeting based on scientist-specified objectives. Prior to AEGIS, images were transmitted from the rover to the operations team on Earth; scientists manually analyzed the images, selected geological targets for the rover’s remote-sensing instruments, and then generated a command sequence to execute the new measurements. AEGIS represents a significant paradigm shift---by using onboard data analysis techniques, the AEGIS software uses scientist input to select high-quality science targets with no human in the loop. This approach allows the rover to autonomously select and sequence targeted observations in an opportunistic fashion, which is particularly applicable for narrow field-of-view instruments (such as the MER Mini-TES spectrometer, the MER Panoramic camera, and the 2011 Mars Science Laboratory (MSL) ChemCam spectrometer). This article provides an overview of the AEGIS automated targeting capability and describes how it is currently being used onboard the MER mission Opportunity rover.

Current and proposed remote space missions, such as the proposed aerial exploration of Titan by an aerobot, often can collect more data than can be communicated back to Earth. Autonomous selective downlink algorithms can choose informative subsets of data to improve the science value of these bandwidth-limited transmissions. This requires statistical descriptors of the data that reflect very abstract and subtle distinctions in science content. We propose a metric learning strategy that teaches algorithms how best to cluster new data based on training examples supplied by domain scientists. We demonstrate that clustering informed by metric learning produces results that more closely match multiple scientists’ labelings of aerial data than do clusterings based on random or periodic sampling. A new metric-learning strategy accommodates training sets produced by multiple scientists with different and potentially inconsistent mission objectives. Our methods are fit for current spacecraft processors (e.g., RAD750) and would further benefit from more advanced spacecraft processor architectures, such as OPERA.

Orbital remote sensing provides a powerful way to efficiently survey targets such as the Earth and other planets and moons for features of interest. One such feature of astrobiological relevance is the presence of surface sulfur deposits. These deposits have been observed to be associated with microbial activity at the Borup Fiord glacial springs in Canada, a location that may provide an analogue to other icy environments such as Europa. This article evaluates automated classifiers for detecting sulfur in remote sensing observations by the hyperion spectrometer on the EO-1 spacecraft. We determined that a data-driven machine learning solution was needed because the sulfur could not be detected by simply matching observations to sulfur lab spectra. We also evaluated several methods (manual and automated) for identifying the most relevant attributes (spectral wavelengths) needed for successful sulfur detection. Our findings include (1) the Borup Fiord sulfur deposits were best modeled as containing two sub-populations: sulfur on ice and sulfur on rock; (2) as expected, classifiers using Gaussian kernels outperformed those based on linear kernels, and should be adopted when onboard computational constraints permit; and (3) Recursive Feature Elimination selected sensible and effective features for use in the computationally constrained environment onboard EO-1. This study helped guide the selection of algorithm parameters and configuration for the classification system currently operational on EO-1. Finally, we discuss implications for a similar onboard classification system for a future Europa orbiter.

This work investigates multiagent path planning in strong, dynamic currents using thousands of highly underactuated vehicles. We address the specific task of path planning for a global network of ocean-observing floats. These submersibles are typified by the Argo global network consisting of over 3000 sensor platforms. They can control their buoyancy to float at depth for data collection or rise to the surface for satellite communications. Currently, floats drift at a constant depth regardless of the local currents. However, accurate current forecasts have become available which present the possibility of intentionally controlling floats' motion by dynamically commanding them to linger at different depths. This project explores the use of these current predictions to direct float networks to some desired final formation or position. It presents multiple algorithms for such path optimization and demonstrates their advantage over the standard approach of constant-depth drifting.

This work addresses mission planning for autonomous underwater gliders based on predictions of an uncertain, time-varying current field. Glider submersibles are highly sensitive to prevailing currents so mission planners must account for ocean tides and eddies. Previous work in variable-current path planning assumes that current predictions are perfect, but in practice these forecasts may be inaccurate. Here we evaluate plan fragility using empirical tests on historical ocean forecasts for which followup data is available. We present methods for glider path planning and control in a time-varying current field. A case study scenario in the Southern California Bight uses current predictions drawn from the Regional Ocean Monitoring System (ROMS).

Robotic explorers must be capable of autonomous navigation into unknown terrain as well as autonomous science to interpret their observations to guide exploration. In this research we have created a robot able to select science features, direct instruments, collect observations, build maps, and interpret this information to plan actions. We report on field experiments in California's Amboy Crater lava field and demonstrate fundamental capabilities for adaptive exploration in geologic mapping tasks. We show feature detection and instrument visual servoing that enables automated science observation of dozens of targets. Gaussian process models are used to discover spatial and cross-sensor structure including correlations between different locations and sensing scales. The rover develops these relationships on the fly with only on-board computation, reinterpreting remote sensing data in light of the surface materials it observes. The rover learns spatial models of physical phenomena and guides its exploration into informative areas using Maximum Entropy Sampling to improve exploration efficiency. The Amboy Crater experiments show that science autonomy can play a useful role in facilitating geologic survey on kilometer scales.

Superpixels are homogeneous image regions comprised of multiple contiguous pixels. Superpixel representations can reduce noise in hyperspectral images by exploiting the spatial contiguity of scene features. This paper combines superpixels with endmember extraction to produce concise mineralogical summaries that assist in browsing large image catalogs. First, a graph-based agglomerative algorithm oversegments the image. We then use segments' mean spectra as input to existing statistical endmember detection algorithms such as sequential maximum angle convex cone (SMACC) and N-FINDR. Experiments compare automatically detected endmembers to target minerals in an Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) scene of Cuprite, Nevada. We also consider a planetary science data set from the Compact Reconnaissance Imaging Spectrometer (CRISM) instrument that benefits from spatial averaging due to higher noise. In both cases, superpixel representations significantly reduce the computational complexity of later processing while improving endmembers' match to the target spectra.

Selective data return leverages onboard data analysis to allocate limited bandwidth resources during remote exploration. Here we present an adaptive method to subsample image sequences for downlink. We treat selective data return as a compression problem in which the explorer agent transmits the subset of measurements that are most informative with respect to the complete dataset. Experiments demonstrate selective downlink of navigation imagery by a rover during autonomous geologic investigations in the Atacama desert of Chile. Here automatic analysis identifies informative images using classifications based on natural image statistics. Image texture analysis, together with a context-sensitive Hidden Markov Model representation, permits adaptive downlink in response to geologic unit boundaries. Selective data return improves the science content of returned data for this geologic mapping task.

Geologists and planetary scientists will benefit from methods for accurate segmentation of rocks in natural scenes. However, rocks are poorly suited for current visual segmentation techniques - they exhibit diverse morphologies and have no uniform property to distinguish them from background soil. We address this challenge with a novel detection and segmentation method incorporating features from multiple scales. These features include local attributes such as texture, object attributes such as shading and two-dimensional shape, and scene attributes such as the direction of illumination. Our method uses a superpixel segmentation followed by region-merging to search for the most probable groups of superpixels. A learned model of rock appearances identifies whole rocks by scoring candidate superpixel groupings. We evaluate our method's performance on representative images from the Mars Exploration Rover catalog.

Mobile robots and robot teams can leverage multiple views of a scene to improve the accuracy of their maps. However non-uniform noise persists even when each sensor's pose is known, and the uncertain correspondence between detections from different views complicates easy "multiple view object detection." We present an algorithm based on expectation/maximization (EM) clustering that permits a principled fusion of the views without requiring an explicit correspondence search. We demonstrate the use of this algorithm to improve mapping performance of robots in simulation and in the field.