The NASA Planetary Data System (PDS) hosts millions of images of planets, moons, and other bodies collected throughout many missions. The ever-expanding nature of data and user engagement demands an interpretable content classification system to support scientific discovery and individual curiosity. In this paper, we leverage a prototype-based architecture to enable users to understand and validate the evidence used by a classifier trained on images from the Mars Science Laboratory (MSL) Curiosity rover mission. In addition to providing explanations, we investigate the diversity and correctness of evidence used by the content-based classifier. The work presented in this paper will be deployed on the PDS Image Atlas, replacing its non-interpretable counterpart.

The cosmic microwave background (CMB) is a significant source of knowledge about the origin and evolution of our universe. However, observations of the CMB are contaminated by foreground emissions, obscuring the CMB signal and reducing its efficacy in constraining cosmological parameters. We employ deep learning as a data-driven approach to CMB cleaning from multi-frequency full-sky maps. In particular, we develop a graph-based Bayesian convolutional neural network based on the U-Net architecture that predicts cleaned CMB with pixel-wise uncertainty estimates. We demonstrate the potential of this technique on realistic simulated data based on the Planck mission. We show that our model ac- accurately recovers the cleaned CMB sky map and resulting angular power spectrum while identifying regions of uncertainty. Finally, we discuss the current challenges and the path forward for deploying our model for CMB recovery on real observations.

The NASA Planetary Data System hosts millions of images acquired from the planet Mars. To help users quickly find images of interest, we have developed and deployed content-based classification and search capabilities for Mars orbital and surface images. The deployed systems are publicly accessible using the PDS Image Atlas. We describe the process of training, evaluating, calibrating, and deploying updates to two CNN classifiers for images collected by Mars missions. We also report on three years of deployment including usage statistics, lessons learned, and plans for the future.

Abstract Science teams for rover-based planetary exploration missions like the Mars Science Laboratory Curiosity rover have limited time for analyzing new data before making decisions about follow-up observations. There is a need for systems that can rapidly and intelligently extract information from planetary instrument datasets and focus attention on the most promising or novel observations. Several novelty detection methods have been explored in prior work for three-channel color images and non-image datasets, but few have considered multispectral or hyperspectral image datasets for the purpose of scientific discovery. We compared the performance of four novelty detection methods—Reed Xiaoli (RX) detectors, principal component analysis (PCA), autoencoders, and generative adversarial networks (GANs)—and the ability of each method to provide explanatory visualizations to help scientists understand and trust predictions made by the system. We show that pixel-wise RX and autoencoders trained with structural similarity (SSIM) loss can detect morphological novelties that are not detected by PCA, GANs, and mean squared error autoencoders, but that the latter methods are better suited for detecting spectral novelties—i.e., the best method for a given setting depends on the type of novelties that are sought. Additionally, we find that autoencoders provide the most useful explanatory visualizations for enabling users to understand and trust model detections, and that existing GAN approaches to novelty detection may be limited in this respect.

In this work, we present a system based on convolutional autoencoders for detecting novel features in multispectral images. We introduce SAMMIE: Selections based on Autoencoder Modeling of Multispectral Image Expectations. Previous work using autoencoders employed the scalar reconstruction error to classify new images as novel or typical. We show that a spatial-spectral error map can enable both accurate classification of novelty in multispectral images as well as human-comprehensible explanations of the detection. We apply our methodology to the detection of novel geologic features in multispectral images of the Martian surface collected by the Mastcam imaging system on the Mars Science Laboratory Curiosity rover.

Data analysis and machine learning methods have great potential to aid in planetary exploration. Spacecraft often operate at great distances from the Earth, and the ability to autonomously detect features of interest onboard can enable content-sensitive downlink prioritization to increase mission science return. We describe algorithms that we designed to assist in three specific scientific investigations to be conducted during flybys of Jupiter's moon Europa: the detection of thermal anomalies, compositional anomalies, and plumes of icy matter from Europa's subsurface ocean. We also share the unique constraints imposed by the onboard computing environment and several lessons learned in our collaboration with planetary scientists and mission designers.

We have constructed an information extraction system called the Mars Target Encyclopedia that takes in planetary science publications and extracts scientific knowledge about target compositions. The extracted knowledge is stored in a searchable database that can greatly accelerate the ability of scientists to compare new discoveries with what is already known. To date, we have applied this system to ~6000 documents and achieved 41-56% precision in the extracted information.

NASA has acquired more than 22 million images from the planet Mars. To help users find images of interest, we developed a content-based search capability for Mars rover surface images and Mars orbital images. We started with the AlexNet convolutional neural network, which was trained on Earth images, and used transfer learning to adapt the network for use with Mars images. We report on our deployment of these classifiers within the PDS Imaging Atlas, a publicly accessible web interface, to enable the first content-based image search for NASA’s Mars images.

The Intelligent Payload Experiment (IPEX) is a CubeSat mission to flight validate technologies for onboard instrument processing and autonomous operations for NASA's Earth Science Technologies Office (ESTO). Specifically IPEX is to demonstrate onboard instrument processing and product generation technologies for the Intelligent Payload Module (IPM) of the proposed Hyperspectral Infra-red Imager (HyspIRI) mission concept. Many proposed future missions, including HyspIRI, are slated to produce enormous volumes of data requiring either significant communication advancements or data reduction techniques. IPEX demonstrates several technologies for onboard data reduction, such as computer vision, image analysis, image processing and in general demonstrates general operations autonomy. We conclude this paper with a number of lessons learned through operations of this technology demonstration mission on a novel platform for NASA.

In the era of large scientific data sets, there is an urgent need for methods to automatically prioritize data for review. At the same time, for any automated method to be adopted by scientists, it must make decisions that they can understand and trust. In this paper, we propose Discovery through Eigenbasis Modeling of Uninteresting Data (DEMUD), which uses principal components modeling and reconstruction error to prioritize data. DEMUD’s major advance is to offer domain-specific explanations for its prioritizations. We evaluated DEMUD’s ability to quickly identify diverse items of interest and the value of the explanations it provides. We found that DEMUD performs as well or better than existing class discovery methods and provides, uniquely, the first explanations for why those items are of interest. Further, in collaborations with planetary scientists, we found that DEMUD (1) quickly identifies very rare items of scientific value, (2) maintains high diversity in its selections, and (3) provides explanations that greatly improve human classification accuracy.

Orbital remote sensing provides a powerful way to efficiently survey targets such as the Earth and other planets and moons for features of interest. One such feature of astrobiological relevance is the presence of surface sulfur deposits. These deposits have been observed to be associated with microbial activity at the Borup Fiord glacial springs in Canada, a location that may provide an analogue to other icy environments such as Europa. This article evaluates automated classifiers for detecting sulfur in remote sensing observations by the hyperion spectrometer on the EO-1 spacecraft. We determined that a data-driven machine learning solution was needed because the sulfur could not be detected by simply matching observations to sulfur lab spectra. We also evaluated several methods (manual and automated) for identifying the most relevant attributes (spectral wavelengths) needed for successful sulfur detection. Our findings include (1) the Borup Fiord sulfur deposits were best modeled as containing two sub-populations: sulfur on ice and sulfur on rock; (2) as expected, classifiers using Gaussian kernels outperformed those based on linear kernels, and should be adopted when onboard computational constraints permit; and (3) Recursive Feature Elimination selected sensible and effective features for use in the computationally constrained environment onboard EO-1. This study helped guide the selection of algorithm parameters and configuration for the classification system currently operational on EO-1. Finally, we discuss implications for a similar onboard classification system for a future Europa orbiter.

Given the large volume of images being sent back from remote spacecraft, there is a need for automated analysis techniques that can quickly identify interesting features in those images. Feature identification in individual images and automated change detection in multiple images of the same target are valuable for scientific studies and can inform subsequent target selection. We introduce a new approach to orbital image analysis called dynamic landmarking. It focuses on the identification and comparison of visually salient features in images. We have evaluated this approach on images collected by five Mars orbiters. These evaluations were motivated by three scientific goals: to study fresh impact craters, dust devil tracks, and dark slope streaks on Mars. In the process we also detected a different kind of surface change that may indicate seasonally exposed bedforms. These experiences also point the way to how this approach could be used in an onboard setting to analyze and prioritize data as it is collected.

We present Confidence-based Feature Acquisition (CFA), a novel supervised learning method for acquiring missing feature values when there is missing data at both training and test time. Previous work has considered the cases of missing data at training time (e.g., Active Feature Acquisition, AFA [8]), or at test time (e.g., Cost-Sensitive Naive Bayes, CSNB [2]), but not both. At training time, CFA constructs a cascaded ensemble of classifiers, starting with the zero-cost features and adding a single feature for each successive model. For each model, CFA selects a subset of training instances for which the added feature should be acquired. At test time, the set of models is applied sequentially (as a cascade), stopping when a user-supplied confidence threshold is met. We compare CFA to AFA, CSNB, and several other baselines, and find that CFA's accuracy is at least as high as the other methods, while incurring significantly lower feature acquisition costs.

Although there has been significant research on modelling and learning user preferences for various types of objects, there has been relatively little work on the problem of representing and learning preferences over sets of objects. We introduce a representation language, DD-PREF, that balances preferences for particular objects with preferences about the properties of the set. Specifically, we focus on the depth of objects (i.e. preferences for specific attribute values over others) and on the diversity of sets (i.e. preferences for broad vs. narrow distributions of attribute values). The DD-PREF framework is general and can incorporate additional object- and set-based preferences. We describe a greedy algorithm, DD-Select, for selecting satisfying sets from a collection of new objects, given a preference in this language. We show how preferences represented in DD-PREF can be learned from training data. Experimental results are given for three domains: a blocks world domain with several different task-based preferences, a real-world music playlist collection, and rover image data gathered in desert training exercises.

Spacecraft increasingly employ onboard data analysis to inform further data collection and prioritization decisions. However, many spacecraft operate in high-radiation environments in which the reliability of dataintensive computation is not known. This paper presents the first study of radiation sensitivity for k-means clustering. Our key findings are 1) k-means data structures differ in sensitivity, which is not determined solely by the amount of memory exposed; 2) no special radiation protection is needed below a data-set-dependent radiation threshold, enabling the use of faster, smaller, and cheaper onboard memory; and 3) subsampling improves radiation tolerance slightly, but the use of kd-trees unfortunately reduces tolerance. Our conclusions can help tailor k-means for use in future high-radiation environments.

Analyzing data on-board a spacecraft as it is collected enables several advanced spacecraft capabilities, such as prioritizing observations to make the best use of limited bandwidth and reacting to dynamic events as they happen. In this paper, we describe how we addressed the unique challenges associated with on-board mining of data as it is collected: uncalibrated data, noisy observations, and severe limitations on computational and memory resources. The goal of this effort, which falls into the emerging application area of spacecraft-based data mining, was to study three specific science phenomena on Mars. Following previous work that used a linear support vector machine (SVM) on-board the Earth Observing 1 (EO-1)spacecraft, we developed three data mining techniques for use on-board the Mars Odyssey spacecraft. These methods range from simple thresholding to state-of-the-art reduced-set SVM technology. We tested these algorithms on archived data in a flight software testbed. We also describe a significant, serendipitous science discovery of this data mining effort: the confirmation of a water ice annulus around the north polar cap of Mars. We conclude with a discussion on lessons learned in developing algorithms for use on-board a spacecraft.

Most work on preference learning has focused on pairwise preferences or rankings over individual items. In this paper, we present a method for learning preferences over sets of items. Our learning method takes as input a collection of positive examples---that is, one or more sets that have been identified by a user as desirable. Kernel density estimation is used to estimate the value function for individual items, and the desired set diversity is estimated from the average set diversity observed in the collection. Since this is a new learning problem, we introduce a new evaluation methodology and evaluate the learning method on two data collections: synthetic blocks-world data and a new real-world music data collection that we have gathered.

Clustering algorithms have become increasingly important in handling and analyzing data. Considerable work has been done in devising effective but increasingly specific clustering algorithms. In contrast, we have developed a generalized framework that accommodates diverse clustering algorithms in a systematic way. This framework views clustering as a general process of iterative optimization that includes modules for supervised learning and instance assignment. The framework has also suggested several novel clustering methods. In this paper, we investigate experimentally the efficacy of these algorithms and test some hypotheses about the relation between such unsupervised techniques and the supervised methods embedded in them.

A key pra\nti\nal obsta\nle in applying support ve\ntor ma-hines to many large-s\nale data mining tasks is that SVM training time generally s\nales quadrati\nally (or worse) in the number of examples or support ve\ntors. This \nomplexity is further ompounded when a spe\ni\n SVM training is but one of many, su\nh as in Leave-One-Out-Cross-Validation (LOOCV) for determining optimal SVM parameters or as in wrapper-based feature sele\ntion. In this paper we explore new te\nhniques for redu\ning the amortized \nost of ea\nh su\nh SVM training, by seeding su\n\nessive SVM trainings with the results of previous similar trainings.