NASA's Advanced Information Systems Technology (AIST) Program is one of several Technology programs managed by the Earth Science Technology Office (ESTO) in the Earth Science Division (ESD). The AIST Program focuses on advanced information systems and novel computer science technologies that will be needed by NASA Earth Science in the next 5 to 10 years. The three main thrusts of the AIST Program deal with New Observing Strategies, Analytic Collaborative Frameworks and Earth System Digital Twins. This paper describes these three thrusts and how they work together to create future Earth Science information systems.

We discuss the long-term evolution of an initiative being led by NASA's Earth Science Technology Office (ESTO) known as the “New Observing Strategy” [1] and describe a recent implementation of NOS in support of a virtual scientific field campaign. NOS represents a revolutionary change in the design and execution of Earth Science missions, replacing the monolithic “single satellite” approach with loosely coordinated or disaggregated, multi-platform observatories that have the potential for significant enhancement to scientific return. During late 2021 and early 2022, ES TO launched a pathfinder demonstration of NOS, in which observations of flooding events triggered by heavy rain over wildfire burn scars were intelligently targeted for collection by a numerical prediction system. The resulting “optimized” set of observations were used for characterizing deficiencies inherent in the accurate modeling of the debris flow runoff, which can result in deadly, fast-moving landslides.

Quantum Computing has been invigorated by the introduction of the first commercially available computers. Quantum Annealers (QAs) like the D-Wave are now housing ~5000 quantum-bits, and quantum computing, a potentially disruptive technology, is closer to becoming a reality than ever. Here we investigate quantum-assisted approaches to image registration of MODIS images using QAs. We present two approaches: (1) a mapping of image matching to a quadratic-unconstrained-binary optimization problem and (2) a machine learning approach that employs generative models trained using quantum annealing statistics. Results are shown and discussed.

The recent multiplication of open access initiatives to Big Data from Space is giving momentum to the field by widening substantially the spectrum of scientific communities and users, as well as awareness among the public, while offering new benefits at all levels from individual citizens to the whole society. Following a detailed and rigorous review process, 14 articles have been selected out of 48 submissions for this special issue. These are briefly summarized.

Because of the large variety of planetary sensors and spacecraft already collecting data and with many new and improved sensors being planned for future missions, planetary science needs to integrate numerous multimodal image sources, and, as a consequence, accurate and robust registration algorithms are required. In this article, we develop a new framework for crater detection based on marked point processes (MPPs) that can be used for planetary image registration. MPPs were found to be effective for various object detection tasks in Earth observation, and a new MPP model is proposed here for detecting craters in planetary data. The resulting spatial features are exploited for registration, together with fitness functions based on the MPP energy, on the mean directed Hausdorff distance, and on the mutual information. Two different methods—one based on birth–death processes and region-of-interest analysis and the other based on graph cuts and decimated wavelets—are developed within the proposed framework. Experiments with a large set of images, including 13 thermal infrared and visible images of the Mars surface, 20 semisimulated multitemporal pairs of images of the Mars surface, and a real multitemporal image pair of the Lunar surface, demonstrate the effectiveness of the proposed framework in terms of crater detection performance as well as for subpixel registration accuracy.

Seasonal snow on land is an important factor in water resources, natural disasters, water security, and weather and climate. While many types of sensors are sensitive to snow water equivalent or snow depth, no single sensor type works across the range of snow conditions and confounding factors (primarily forests and complex terrain) experienced globally. An attractive potential solution is to leverage existing and planned satellites with applicable sensors in a distributed spacecraft mission configuration. In this paper, we examine the basic question of the spatial-temporal coverage achievable by various combinations of satellites and sensors. The results will inform discussions on the advantages of specific future sensors, sensor parameters, as well as requirements for algorithms and models to help bridge the space-time gaps.

Emerging capabilities to integrate instruments on smallsats, airborne platforms and in situ devices into an intelligent, distributed observing strategy show great promise for measuring Earth science natural phenomena and physical processes that have not previously been characterized. To reduce the threshold for success in deploying such an intelligent, integrated observing strategy, a ground-based testbed system is proposed. Virtually all of the technologies needed for using such a tool have matured to the point of being used, individually. Virtually none of the technologies have been deployed, working together. The technologies to be deployed should be integrated into a working "breadboard" where the components can be debugged and performance and behavior characterized and tuned-up. A system of this complexity should not be expected to work without full integration and experimental characterization. Further, and perhaps more importantly, in order to successfully propose a space-based element to this strategy, teams must convince the relevant science community that the risk is low enough to warrant the investment.

One of the new thrusts of the Earth Science Technology Office (ESTO) Advanced Information Systems Technology (AIST) Program focused on New Observing Strategies (NOS) for future Earth Science missions and measurements. Its goal is to provide a framework for identifying technology advances needed to exploit newly available observational capabilities, particularly to enable the development of the information technologies needed to support planning, evaluating, implementing, and operating dynamic, multi-element sets of observing assets. In this paper, we will introduce relevant NOS terminology and some key concepts before describing the objectives, driving factors and technology goals of this new thrust.

The demonstration of high quality scientific observations from small satellites has signaled the opportunity to employ new Earth science observing strategies using constellations or swarms of these relatively inexpensive platforms. While replacement of the existing observing strategies with those based on smallsats is not indicated, they do offer the promise of completely new observing strategies that can measure characteristics of natural phenomena and physical processes that were not possible with the monolithic Battlestar Galactica type systems. Distributed Spacecraft Missions (DSM) are starting to take shape, as described in this Session. Variations in configurations are possible including geometry and composition and the degree of autonomy in operations. A number of sensor web information technologies have been matured, but mostly focused on ground based systems. Some useful technologies have also matured in other domains, such as machine learning and the Internet of Things. New technology development is needed, guided by a framework to prioritize new investments. Such a framework would also encourage division of labor and collaboration among Governments, companies and educational institutions.

In order to overcome spatial resolution limitations associated with physical sensor limitations when using smallsats and cubesats, we utilize an image processing technology referred to as Super-Resolution (SR). In general, software approaches are increasingly considered in connection with smaller satellites for which size, mass and power constraints limit the sensor capabilities. Being able to perform hardware vs. software trades might enable more capabilities for a lower cost. This paper describes recent experiments conducted to optimize the spatial enhancement of acquired observations using multiple sub-pixel shifted low resolution image.

For the last 5 years, NASA Goddard has been investigating Distributed Spacecraft Missions (DSM) system architectures, surveying past, current and potential mission concepts, developing several taxonomies and identifying some key technologies that will enable future DSM mission design, development, operations and management. This paper summarizes this Initiative and the talk will provide details about specific Goddard DSM projects that are currently underway and that are relevant to future Earth Science missions.

Distributed Spacecraft Missions (DSMs) have largely garnered interest due to lower cost and new technical capabilities. However, another important attribute in DSMs is the potential of graceful degradation where a failure of one or more spacecraft (or instruments) in the DSM can reduce total performance, but the remaining functional spacecraft can maintain a limited science return. The potential for retaining some mission value in the event of partial failures leads to enhanced lifetime value. Here, an approach is developed to analyze graceful degradation and performance of DSM used for Earth observation. The system performance is defined through metrics of coverage and average revisit time of points of interest. Graceful degradation is quantified through total change and rate of change in performance metrics with increasing sub-system failures over the lifetime of a mission. An Expected Performance measure is formulated for conducting comparison of DSM architectures for early stage conceptual mission design and trade studies.

This paper addresses the problem of semi-automatic image registration on planetary images. A joint feature-based and area-based approach is proposed. Firstly, the most relevant craters are extracted from the two images to register, and then, registration is performed in two steps. The first step matches the craters extracted from the images based on a generalized Hausdorff distance. In the second step, the mutual information between the two images is maximized to achieve high registration accuracy. Craters are detected by a stochastic-geometry approach based on a marked point process model and of a multiple-birth-and-cut energy minimization algorithm. The experimental validation is carried out with 13 images for the crater extraction stage, and with 20 semi-synthetic pairs of images with ground truth and several images extracted from actual multitemporal lunar scenes for the registration phase.

For many applications, accurate and fast image registration of large amounts of multi-source data is the first necessary step before subsequent processing and integration. Image registration is defined by several steps and each step can be approached by various methods which all present diverse advantages and drawbacks depending on the type of data, the type of applications, the a priori information known about the data and the type of accuracy that is required. This paper will first present a general overview of remote sensing image registration and then will go over a few specific methods and their applications.

While there is growing interest in implementing future NASA Earth Science missions as Distributed Spacecraft Missions (DSMs), there are currently very few tools available to help in the design of DSMs. The objective of our project is to provide a framework that facilitates DSM Pre-Phase A investigations and optimizes DSM designs with respect to a-priori Science goals. Our Trade-space Analysis Tool for Constellations (TAT-C) enables the investigation of questions such as: “Which type of constellations should be chosen? How many spacecraft should be included in the constellation? Which design has the best cost/risk value?”. This paper provides a description of the TAT-C tool and its components.

In this paper we propose to utilize a new concept of discrete directional Gabor frames for automatic image registration. The directional Gabor representations have been shown to provide more accurate feature extraction than directional wavelet transforms for images where texture is the dominant feature. Initial experimental results are presented here which indicate that discrete directional Gabor frames exhibit strong correlations, which indicates that they are likely to improve the existing image registration toolbox.

Because of the large variety of sensors and spacecraft collecting data, planetary science needs to integrate various multisensor and multitemporal images. These multiple data represent a precious asset, as they allow the study of target spectral responses and of changes in the surface structure. Because of their variety, they also require accurate and robust registration. A new crater detection algorithm, used to extract features to be integrated in an image registration framework, is presented. A marked point process-based method has been developed to model the spatial distribution of elliptical objects (i.e. the craters), and a birth-death Markov chain method, coupled with a region-based scheme aiming at computational efficiency, is used to find the optimal configuration fitting the image. The extracted features are exploited, together with a newly defined fitness function based on a modified Hausdorff distance, by an image registration algorithm whose architecture has been designed to minimize the computation time.

Automatic image registration is the process of aligning two or more images of approximately the same scene with minimal human assistance. Wavelet-based automatic registration methods are standard, but sometimes are not robust to the choice of initial conditions. That is, if the images to be registered are too far apart relative to the initial guess of the algorithm, the registration algorithm does not converge or has poor accuracy, and is thus not robust. These problems occur because wavelet techniques primarily identify isotropic textural features and are less effective at identifying linear and curvilinear edge features. We integrate the recently developed mathematical construction of shearlets, which is more effective at identifying sparse anisotropic edges, with an existing automatic wavelet-based registration algorithm. Our shearlet features algorithm produces more distinct features than wavelet features algorithms; the separation of edges from textures is even stronger than with wavelets. Our algorithm computes shearlet and wavelet features for the images to be registered, then performs least squares minimization on these features to compute a registration transformation. Our algorithm is two-staged and multiresolution in nature. First, a cascade of shearlet features is used to provide a robust, though approximate, registration. This is then refined by registering with a cascade of wavelet features. Experiments across a variety of image classes show an improved robustness to initial conditions, when compared to wavelet features alone.

We investigate the role of anisotropic feature extraction methods for automatic image registration of remotely sensed multitemporal images. Building on the classical use of wavelets in image registration, we develop an algorithm based on shearlets, a mathematical generalization of wavelets that offers increased directional sensitivity. Experimental results on multitemporal Land-sat images are presented, which indicate superior performance of the shearlet algorithm when compared to classical wavelet algorithms.

This paper evaluates the potential of embedded Graphic Processing Units in the Nvidia's Tegra K1 for onboard processing. The performance is compared to a general purpose multi-core CPU and full fledge GPU accelerator. This study uses two algorithms: Wavelet Spectral Dimension Reduction of Hyperspectral Imagery and Automated Cloud-Cover Assessment (ACCA) Algorithm. Tegra K1 achieved 51% for ACCA algorithm and 20% for the dimension reduction algorithm, as compared to the performance of the high-end 8-core server Intel Xeon CPU with 13.5 times higher power consumption.

The method presented in this paper utilizes Global Land Survey (GLS) maps to register Earth Observing-1 (EO-1) data, either using entire scenes or utilizing chips extracted from the GLS maps. The automated registration algorithm is based on the optimization of wavelet or wavelet-like features extracted from both reference and input image data. After testing the method on several ALI scenes, results and conclusions are presented.

With the launch of several planetary missions in the last decade, a large amount of planetary images has been already acquired and much more will be available for analysis in the coming years. The image data need to be analyzed, preferably by automatic processing techniques because of the huge amount of data. Although many automatic feature extraction methods have been proposed and utilized for earth remote sensing images, these methods are not always applicable to planetary data that often present low contrast and uneven illumination characteristics. Here, we propose a new unsupervised method for the extraction of different features of elliptical and geometrically compact shapes, such as craters and rocks of compact shape (e.g., boulders), to be used for image registration purposes. This approach is based on the combination of several image processing techniques, including watershed segmentation and the generalized Hough transform. The method potentially has application for extraction of craters, rocks, and other geological features.

The next generation of Earth-observing spacecraft are likely to generate enormous volumes of data. A major challenge lies in the conversion of these mountains of data into information useful to researchers and other users. Hierarchical segmentation is one way to detect relationships among regions in a hyperspectral image. We implemented this algorithm on a next-generation space-capable hardware platform, and studied its performance before and after adapting it to use the platform's unique computational resources. We found that these adaptations enable an order-of-magnitude increase in performance over our initial implementation, and our detailed analysis points to areas for additional improvement.

Image registration is an important element in data processing for remote sensing with many applications and a wide range of solutions. Despite considerable investigation the field has not settled on a definitive solution for most applications and a number of questions remain open. This article looks at selected research issues by surveying the experience of operational satellite teams, application-specific requirements for Earth science, and our experiments in the evaluation of image registration algorithms with emphasis on the comparison of algorithms for subpixel accuracy. We conclude that remote sensing applications put particular demands on image registration algorithms to take into account domain-specific knowledge of geometric transformations and image content.

The task of image registration can be divided into two major components, i.e., the extraction of control points or features from images, and the search among the extracted features for the matching pairs that represent the same feature in the images to be matched. Manual extraction of control features can be subjective and extremely time consuming, and often results in few usable points. On the other hand, automated feature extraction allows using invariant target features such as edges, corners, and line intersections as relevant landmarks for registration purposes. In this paper, we present an extension of a recently developed morphological approach for automatic extraction of landmark chips and corresponding windows in a fully unsupervised manner for the registration of multispectral images. Once a set of chip-window pairs is obtained, a (hierarchical) robust feature matching procedure, based on a multiresolution overcomplete wavelet decomposition scheme, is used for registration purposes. The proposed method is validated on a pair of remotely sensed scenes acquired by the Advanced Land Imager (ALI) multispectral instrument and the Hyperion hyperspectral instrument aboard NASA’s Earth Observing-I satellite.

In this letter, we propose hierarchical principal component analysis (HPCA) techniques for fusing spatial and spectral data, and compare them to direct principal component analysis (DPCA) over Multiangle Imaging SpectroRadiometer (MISR) data. It is shown that the proposed methods are significantly faster than DPCA. In case of DPCA, we merge the 20 different images resulting from the four spectral bands over the nadir and the four forward angles. In the hierarchical case, we first merge the information from the four spectral camera bands; then, we integrate the spatial information from the five cameras in the second step (or vice versa) by applying principal component analysis (PCA) twice. The classification results show that fused data using HPCA compare favorably to DPCA or to classification using the original data. This is because applying PCA to one particular data domain (e.g., spectral data followed by spatial data or vice versa) tends to better remove redundancies and enhance features within that domain. In addition, classification through hierarchical data fusion results in computational savings over the other methods.

The future of remote sensing will see the development of spacecraft formations, and with this development will come a number of complex challenges such as maintaining precise relative position and specified attitudes. At the same time, there will be increasing needs to understand planetary system processes and build accurate prediction models. One essential technology to accomplish these goals is the integration of multiple source data. For this integration, image registration and fusion represent the first steps and need to be performed with very high accuracy. In this paper, we describe studies performed in both image registration and fusion, including a modular framework that was built to describe registration algorithms, a Web-based image registration toolbox, and the comparison of several image fusion techniques using data from the EO-1/ALI and Hyperion sensors

The presence of cloud contamination can hinder the use of satellite data, and this requires a cloud detection process to mask out cloudy pixels from further processing. The trend for remote sensing satellite missions has always been towards smaller size, lower cost, more flexibility, and higher computational power. Reconfigurable Computers (RCs) combine the flexibility of traditional microprocessors with the power of Field Programmable Gate Arrays (FPGAs). Therefore, RCs are a promising candidate for on-board preprocessing. This paper presents the design and implementation of an RC-based real-time cloud detection system. We investigate the potential of using RCs for on-board preprocessing by prototyping the Landsat 7 ETM+ ACCA algorithm on one of the state-of-the art reconfigurable platforms, SRC-6E. Although a reasonable amount of investigations of the ACCA cloud detection algorithm using FPGAs has been reported in the literature, very few details/results were provided and/or limited contributions were accomplished. Our work has been proven to provide higher performance and higher detection accuracy.

Principal Component Analysis (PCA) has been widely used as a data reduction technique to overcome the curse of dimensionality. In this research we show a different use for PCA technique as a tool for data fusion. PCA as a data fusion technique is performed over the Multiangle Imaging Spectroradiometer (MISR) data, studying dust storms to better serve their identification. The multi-angle viewing capability of MISR is used to enhance our understanding of the Earth's environment that includes climate particularly of atmosphere and of land surfaces. In this research the multi angle MISR images clearly show a dust storm over the Liaoning region of China as well as parts of northern and western Korea on April 8, 2002. PCA is used to combine the obtained information from the different angle views and frequency bands of MISR datasets. Performing K-means clustering on the original and the assimilated products apply a quantitative measure that is introduced. Upon classifying the first 4 principal components (PCs) having 95% of the information content similar results were obtained as compared to the classification using original datasets.

The problem of image registration, or the alignment of two or more images representing the same scene or object, has to be addressed in various disciplines that employ digital imaging. In the area of remote sensing, just like in medical imaging or computer vision, it is necessary to design robust, fast, and widely applicable algorithms that would allow automatic registration of images generated by various imaging platforms at the same or different times and that would provide subpixel accuracy. One of the main issues that needs to be addressed when developing a registration algorithm is what type of information should be extracted from the images being registered, to be used in the search for the geometric transformation that best aligns them. The main objective of this paper is to evaluate several wavelet pyramids that may be used both for invariant feature extraction and for representing images at multiple spatial resolutions to accelerate registration. We find that the bandpass wavelets obtained from the steerable pyramid due to Simoncelli performs best in terms of accuracy and consistency, while the low-pass wavelets obtained from the same pyramid give the best results in terms of the radius of convergence. Based on these findings, we propose a modification of a gradient-based registration algorithm that has recently been developed for medical data. We test the modified algorithm on several sets of real and synthetic satellite imagery.

Hyperspectral imagery, by definition, provides valuable remote sensing observations at hundreds of frequency bands. Conventional image classification (interpretation) methods may not be used without dimension reduction preprocessing. Automatic wavelet reduction has been proven to yield better or comparable classification accuracy, while achieving substantial computational savings. However, the large hyperspectral data volumes remain to present a challenge for traditional processing techniques. Reconfigurable computers (RCs) can leverage the synergism between conventional processors and FPGAs to provide low-level hardware functionality at the same level of programmability as general-purpose computers. We investigate the potential of using RCs for on-board, i.e. aboard airborne/spaceborne carriers, preprocessing of hyperspectral imagery by prototyping for the first time the automatic wavelet dimension reduction algorithm. Our investigation exploits the fine and coarse grain parallelism provided by the RCs and has been experimentally verified on one of the state-of the art reconfigurable platforms, SRC-6E. An order of magnitude speedup over traditional processing techniques has been reported.

Recently developed hyperspectral sensors provide much richer information than comparable multispectral sensors. However traditional methods that have been designed for multispectral data are not easily adaptable to hyperspectral data. One way to approach this problem is to perform dimension reduction as pre-processing, i.e. to apply a transformation that brings data from a high order dimension to a low order dimension. Wavelet spectral analysis of hyperspectral images has been recently proposed as a method for dimension reduction and, when tested on the classification of AVIRIS data, has shown promising results over the traditional principal component analysis (PCA) technique. We propose to extend and apply the wavelet analysis reduction method to the Atmospheric Infrared Sounder (AIRS) instrument data, designed to measure the Earth's atmospheric water vapor and temperature profiles on a global scale. With more than 2,000 channels, the AIRS infrared data represent a good candidate for dimension reduction, and especially wavelet reduction, due to its computational efficiency and the large data sizes involved.

While automatic image registration algorithms are usually being evaluated with regards to their accuracy, it is often useful to relate this accuracy to the "initial conditions", i.e., the distance between the initial navigation geolocation and the correct result. This paper describes a modular framework that was built to describe registration algorithms, and utilize this framework to attempt to classify different registration components and algorithms in terms of their responses to the initial conditions. Performances would be evaluated on synthetic data, multitemporal and multisensor data. All results of the study would be presented at the conference and would be useful for two different purposes: (1) provide automatic quality assessment of the geolocation of remote sensing data by performing interalgorithm consistency studies; and (2) be the foundations for the design of future on-board applications including planetary exploration.

The goal of the project described in this paper is to build a prototype of an operational system, which will provide registration within subpixel accuracy of multitemporal Landsat data, acquired by either Landsat-5 or Landsat-7 Thematic Mapper instruments. Integrated within an automated mass processing system for Landsat data, the input to our registration system consists of scenes that have been geometrically and radiometrically corrected, as well as preprocessed for detection of clouds and cloud shadows. Such preprocessed scenes are then georegistered relative to a database of Landsat chips. This paper describes the entire registration process, including the use of landmark chips, feature extraction performed by an overcomplete wavelet representation, and feature matching using statistically robust techniques. Knowing the approximate longitudes and latitudes or the UTM coordinates of the four corners of each incoming scene, a subset of the chips that represent landmarks included in the scene are selected to perform the registration. For each of these selected landmark chips, a corresponding window is extracted from the incoming scene, and each chip-window pair is registered using a robust wavelet feature-matching methodology. Based on the transformations from the chip-window pairs, a global transformation is then computed for the entire scene using a variant of a robust least median of squares estimator. Empirical results of this registration process, which provided subpixel accuracy for several multitemporal scenes from different study areas, are presented and discussed.

We extend the stochastic gradient optimization used in a mutual information-based registration algorithm to include second-order (Hessian) effects with the aim of accelerating its convergence rate. We consider images, which are misaligned by a four parameter rigid transformation, consisting of scale, rotation and/or x- and y-translations, and we present the results of optimization using a second-order stochastic derivative. The algorithm is applied to a pair of multi-temporal satellite images, and is implemented in a multi-resolution manner using wavelets. Results are presented for an implementation, which switches after a fixed number of iterations, from the first-order scheme to the second-order one.

Unsupervised clustering is a fundamental building block in numerous image processing applications. One of the most popular and widely used clustering schemes for remote sensing applications is the ISOCLUS algorithm, which is based on the ISODATA method. The algorithm is given a set of n data points in d-dimensional space, an integer k indicating the initial number of clusters, and a number of additional parameters. The general goal is to compute the coordinates of a set of cluster centers in d-space, such that those centers minimize the mean squared distance from each data point to its nearest center. This clustering algorithm is similar to another well-known clustering method, called k-means. One significant feature of ISOCLUS over k-means is that the actual number of clusters reported might be fewer or more than the number supplied as part of the input. The algorithm uses different heuristics to determine whether to merge lor split clusters. As ISOCLUS can run very slowly, particularly on large data sets, there has been a growing .interest in the remote sensing community in computing it efficiently. We have developed a faster implementation of the ISOCLUS algorithm. Our improvement is based on a recent acceleration to the k-means algorithm of Kanungo, et al. They showed that, by using a kd-tree data structure for storing the data, it is possible to reduce the running time of k-means. We have adapted this method for the ISOCLUS algorithm, and we show that it is possible to achieve essentially the same results as ISOCLUS on large data sets, but with significantly lower running times. This adaptation involves computing a number of cluster statistics that are needed for ISOCLUS but not for k-means. Both the k-means and ISOCLUS algorithms are based on iterative schemes, in which nearest neighbors are calculated until some convergence criterion is satisfied. Each iteration requires that the nearest center for each data point be computed. Naively, this requires O(kn) time, where k denotes the current number of centers. Traditional techniques for accelerating nearest neighbor searching involve storing the k centers in a data structure. However, because of the iterative nature of the algorithm, this data structure would need to be rebuilt with each new iteration. Our approach is to store the data points in a kd-tree data structure. The assignment of points to nearest neighbors is carried out by a filtering process, which successively eliminates centers that can not possibly be the nearest neighbor for a given region of space. This algorithm is significantly faster, because large groups of data points can be assigned to their nearest center in a single operation. Preliminary results on a number of real Landsat datasets show that our revised ISOCLUS-like scheme runs about twice as fast.

The study of global environmental changes involves the comparison, fusion, and integration of multiple types of remotely-sensed data at various temporal, radiometric, and spatial resolutions. Results of this integration may be utilized for global change analysis, as well as for the validation of new instruments or for new data analysis. Furthermore, future multiple satellite missions will include many different sensors carried on separate platforms, and the amount of remote sensing data to be combined is increasing tremendously. For all of these applications, the first requires step is fast and automatic registration, and as this need for automating registration techniques is being recognized, it becomes necessary to survey all the registration methods which may be applicable to Earth and space science problems and to evaluate their performances on a large variety of existing remote sensing data as well as on simulated data of soon-to-be-flown instruments. In this paper we present one of the first steps toward such as exhaustive quantitative evaluation. First, the different components of image registration algorithms are reviewed, and different choices for each of these components are described. Then, the results of the evaluation of the corresponding algorithms combing these components are described. Then, the results of the evaluation of the corresponding algorithms combining these components are presented on several datasets. The algorithms are based on gray levels or wavelet features and compute rigid transformations (including scale, rotation, and shifts). Test datasets include synthetic data as well as data acquired over several EOS Land Validation Core Sites with the IKONOS and the Landsat-7 sensors.

Hyperspectral imagery provides richer information about materials than multispectral imagery. The new larger data volumes from hyperspectral sensors present a challenge for traditional processing techniques. For example, the identification of each ground surface pixel by its corresponding spectral signature is still difficult because of the immense volume of data. Conventional classification methods may not be used without dimension reduction preprocessing. This is due to the curse of dimensionality, which refers to the fact that the sample size needed to estimate a function of several variables to a given degree of accuracy grows exponentially with the number of variables. Principal component analysis (PCA) has been the technique of choice for dimension reduction. However, PCA is computationally expensive and does not eliminate anomalies that can be seen at one arbitrary band. Spectral data reduction using automatic wavelet decomposition could be useful. This is because it preserves the distinctions among spectral signatures. It is also computed in automatic fashion and can filter data anomalies. This is due to the intrinsic properties of wavelet transforms that preserves high- and low-frequency features, therefore preserving peaks and valleys found in typical spectra. Compared to PCA, for the same level of data reduction, we show that automatic wavelet reduction yields better or comparable classification accuracy for hyperspectral data, while achieving substantial computational savings.

Image registration is the process by which we determine a transformation that provides the most accurate match between two images. The search for the matching transformation can be automated with the use of a suitable metric, but it can be very time-consuming and tedious. We introduce a registration algorithm that combines a simple yet powerful search strategy based on a stochastic gradient with two similarity measures, correlation and mutual information, together with a wavelet-based multiresolution pyramid. We limit our study to pairs of images, which are misaligned by rotation and/or translation, and present two main results. First, we demonstrate that, in our application, mutual information may be better suited for sub-pixel registration as it produces consistently sharper optimum peaks than correlation. Then, we show that the stochastic gradient search combined with either measure produces accurate results when applied to synthetic data, as well as to multitemporal or multisensor collections of satellite data. Mutual information is generally found to optimize with one-third the number of iterations required by correlation. Results also show that a multiresolution implementation of the algorithm yields significant improvements in terms of both speed and robustness over a single-resolution implementation.

Hyperspectral Imagery can provide very rich information on land cover classes. However, it also presents many challenges in data analysis and interpretation, due to the large amount of data collected. For example, conventional methods for land use and land cover classifications may not be applicable, due to "the curse of dimensionality." Therefore, these conventional methods may need a preprocessing step to transform high dimensional data to low dimensional data, by eliminating data redundancy. Due to its conceptual simplicity, principal component analysis (PCA) has been widely used for decades to reduce dimensionality. It is a useful technique if the spectral class structure of the transformed data is such that it is distributed along the first few axes. Otherwise, the transformed data may be similar to the original data. In such a case, the wavelet decomposition technique might be a better approach. Wavelet decomposition can reduce hyperspectral data in the spectral domain for each pixel. This will not only reduce the data volume, but will also preserve the distinction among spectral signatures that is useful for most pixel-based classifiers. This characteristic is related to the intrinsic property of wavelet transforms that preserve high- and low-frequency features during the signal decomposition, and therefore preserve peaks and valleys found in typical spectra. In general, most classification errors occur at the boundary between classes. Since wavelet decomposition is applied to each local pixel, a wavelet-based reduction might not well differentiate classes among neighborhood pixels in the spatial domain. PCA, however, can provide more local spatial information among neighborhood class pixels than wavelet.

With the increasing importance of multiple multiplatform remote sensing missions, fast and automatic integration of digital data from disparate sources has become critical to the success of these endeavors. Our work utilizes maxima of wavelet coefficients to form the basic features of a correlation-based automatic registration algorithm. Our wavelet-based registration algorithm is tested successfully with data from the National Oceanic and Atmospheric Administration (NOAA) Advanced Very High Resolution Radiometer (AVHRR) and the Landsat Thematic Mapper (TM), which differ by translation and/or rotation. By the choice of high-frequency wavelet features, this method is similar to an edge-based correlation method, but by exploiting the multiresolution nature of a wavelet decomposition, our method achieves higher computational speeds for comparable accuracies. This algorithm has been implemented on a single-instruction multiple-data (SIMD) massively parallel computer, the MasPar MP-2, as well as on the CrayT3D, the Cray T3E, and a Beowulf cluster of Pentium workstations.

Recent advances in sensor technology have led to the development of hyperspectral sensors capable of collecting remote sensing imagery at several hundred bands over the spectrum. While these developments hold great promise for Earth science, they create new processing challenges. Therefore, processing hyperspectral data using new efficient techniques is a must.One such approach of effectively processing hyperspectral data is through dimension reduction prior to the use of data in applications such as land use/land cover classifications. Principal Component Analysis (PCA) is perhaps the most popular dimension reduction technique. The outcome is a number of principal components (PCs) in a descending order of information content. It is often the case that only a small number of these components contain the effective information needed. In this paper, we propose an adaptive parallel technique for determining the effective dimensionality of hyperspectral data on computer clusters. Based on a user-specified desired level of information content, the method selects adaptively the faster technique for solving the eigenproblem and computes only the needed components for that level of information.

This paper studies the effects of image translation on wavelet-based image registration. The main result is that the normalized correlation coefficients of low-pass Haar and Daubechies wavelet subbands are essentially insensitive to translations for features larger than twice the wavelet blocksize. The third-level low-pass subbands produce a correlation peak that varies with translation from 0.7 and 1.0 with an average in excess of 0.9. Translation sensitivity is limited to the high-pass subband and even this subband is potentially useful. The correlation peak for high-pass subbands derived from first and second-level low-pass subbands ranges from about 0.0 to 1.0 with an average of about 0.5 for Daubechies and 0.7 for Haar. We use a mathematical model to develop these results, and confirm them on real data.

Given two images of roughly the same scene, image registration is the process of determining the transformation that most nearly maps one image to another.This problem is of particular interest in remote sensing applications, where it is known that two images correspond to roughly the same gecgraphic region, but the exact alignment between the images io not known.There are many approaches to image registration.We will consider an approach based on extracting a Ret of point features from each of the two images, and thus reducing the problem to a point pattern matching problem.Because of measurement errors and the presence of outlying data points in either of the images, it is important that the diotance measure between two point sets be robust to theeo cffecto.We will measure distances using the partial Hauodorff distance, An important element of image registration applications is that the search begins with a priori information on the bounds of transformation, and a good algorithm should be able to take advantage of this information.Point matching can be a computationally intensive task, and there have been a number of algorithms and approaches proposed for solving this problem, both from theoretical and applied standpoints.One common approach is based on a *Dopartmont of Computer

A basic requirement for understanding the dynamics of the Earth's major ecosystems is accurate quantitative information about the distribution and areal extent of the Earth's vegetation formations. Some of this required information can be obtained through the analysis of remotely sensed data. Image segmentation is often one of the first steps of this analysis. This paper focuses on two particular types of segmentation: region-based and edge-based segmentations. Each approach is affected differently by various factors, and both types of segmentations may be improved by taking advantage of their complementary nature. Included among region-based segmentation approaches are region growing methods, which produce hierarchical segmentations of images from finer to coarser resolution. In this hierarchy, an ideal segmentation (ideal for a given application) does not always correspond to one single iteration, but map correspond to several different iterations. This, among other factors, makes it somewhat difficult to choose a stopping criterion for region growing methods. To find the ideal segmentation, the authors develop a stopping criterion for their Iterative Parallel Region Growing (IPRG) algorithm using additional information from edge features, and the Hausdorff distance metric. They integrate information from regions and edges at the symbol level, taking advantage of the hierarchical structure of the region segmentation results. Also, to demonstrate the feasibility of this approach in processing the massive amount of data that will be generated by future Earth remote sensing missions, such as the Earth Observing System (EOS), all the different steps of this algorithm have been implemented on a massively parallel processor.>

A visual navigation system for autonomous land vehicles has been designed at the Computer Vision Laboratory of the University of Maryland. This system includes several modules, among them a 'knowledge-based reasoning module' that is described. This module utilizes domain-dependent knowledge (in this case, 'road knowledge') to analyze and label the visual features extracted from the imagery by the image processing module. Knowledge and general hypotheses are given. The reasoning module itself is described and results are presented. Some conclusions and future extensions are proposed.>

The authors describe some of the important operational considerations and image processing tasks required to utilize a nonscanning structured-light range sensor in path planning for robot mobility. The efforts have concentrated on the geometric design of the projectional grid with regard to smoothing of fine-scale range texture, and the development of image processing techniques in order to extract the deformed grid from the image. Particular emphasis is placed on the issues of operating in ambient lighting, smoothing of range texture, grid pattern selection, albedo normalization, grid extraction, and coarse registration of images to the projected grid. Once the grid is extracted, its global order allows intersections to be labeled, and disparities assigned and converted to range data. This range map can then be converted to a 'topography map' to be used for planning short-range paths through the environment while avoiding obstacles.>

A modular system architecture has been developed to support visual navigation by an autonomous land vehicle. The system consists of vision modules performing image processing, three-dimensional shape recovery, and geometric reasoning, as well as modules for planning, navigating, and piloting. The system runs in two distinct modes, bootstrap and feedforward. The bootstrap mode requires analysis of entire images to find and model the objects of interest in the scene (e.g., roads). In the feedforward mode (while the vehicle is moving), attention is focused on small parts of the visual field as determined by prior views of the scene, to continue to track and model the objects of interest. General navigational tasks are decomposed into three categories, all of which contribute to planning a vehicle path. They are called long-, intermediate-, and short-range navigation, reflecting the scale to which they apply. The system has been implemented as a set of concurrent communicating modules and used to drive a camera (carried by a robot arm) over a scale model road network on a terrain board. A large subset of the system has been reimplemented on a VICOM image processor and has driven the DARPA Autonomous Land Vehicle (ALV) at Martin Marietta's test site in Denver, CO.

A modular system architecture has been developed to support visual navigation by an autonomous land vehicle. The system consists of vision modules performing image processing, 3-D shape recovery, and rule-based reasoning, as well as modules for planning, navigating and piloting. The system runs in two distinct modes, bootstrap and feed-forward. The bootstrap mode requires analysis of entire images in order to find and model the objects of interest in the scene (e.g. roads). In the feed-forward mode (while the vehicle is moving), attention is focused on small parts of the visual field as determined by prior views of the scene, in order to continue to track and model the objects of interest. We have decomposed general navigational tasks into three categories, all of which contribute to planning a vehicle path. They are called long, intermediate and short range navigation, reflecting the scale to which they apply. We have implemented the system as a set of concurrent, communicating modules and use it to drive a camera (carried by a robot arm) over a scale model road network on a terrain board.

This paper describes ongoing work at the Computer Vision Laboratory, in support of the Autonomous Land Vehicle project of the Defense Advanced Research Projects Agency's Strategic Computing Program. A visual navigation system is being developed to enable a vehicle to follow roads which, for the time being, are free of obstacles. We have developed an approach which consists of a boot-strap phase and a feed-forward phase. In each phase, visual processing is executed in three modules: an image processing module, a geometry module and a rule-based reasoning module. These modules and their interrelationships are described, with particular detail given to the image processing module.