Craters are among the most abundant features on the surface of many planets with great importance for planetary scientists. They reveal chronology information about planets and may be used for autonomous spacecraft navigation and landing. Although numerous research efforts have been carried out in the field of crater detection, existing crater detection algorithms (CDAs) are only helpful in a limited number of applications. A promising crater detection approach involves two main steps: 1) hypothesis generation (HG) and 2) hypothesis verification (HV). During HG, potential crater locations are detected. The validity of the hypothesized crater locations is then tested in a HV step. In this context, we discuss some commonly used algorithms for HG such as highlight-shadow region detection and Hough transform as well as our novel and enhanced algorithms based on interest point detection and convex grouping. A key objective of this paper is to analyze their performance while paying special attention to how they affect the accuracy of the verification step. To deal with different size craters, we focus on multiscale HG. For HV, we have chosen convolutional neural networks which have recently achieved state-of-the-art performance in many computer vision applications. Due to the variation of test sets in the literature, it is often challenging to compare the performance of different CDAs in a fair way. In this paper, we present a comprehensive performance evaluation and comparison of CDAs. Each algorithm has been trained/tested using common data sets generated by a systematic approach.

This paper describes a structured light-based sensor for hazard avoidance in planetary environments. The system presented here can also be used in terrestrial applications constrained by reduced onboard power and computational complexity and low illumination conditions. The sensor consists on a calibrated camera and laser dot projector system. The onboard hazard avoidance system determines the position of the projected dots in the image and through a triangulation process detects potential hazards. The paper presents the design parameters for this sensor and describes the image based solution for hazard avoidance. The system presented here was tested extensively in day and night conditions in Lunar analogue environments. The current system achieves over 97% detection rate with 1.7% false alarms over 2000 images.

Automatic crater detection in planetary images is an important task with many applications in planetary science, spacecraft navigation, landing, and control. Typically, crater detection algorithms consist of two main steps: candidate crater region extraction and crater verification. Various methods have been proposed for extracting candidate crater regions, ranging from detecting circular/elliptical regions to detecting highlight and shadow regions. For crater verification, powerful feature extraction and machine learning techniques have been employed. While this two-step approach can be efficient and robust, inaccuracies in the candidate crater region extraction step can result in mislocalized crater regions which could affect verification performance. In this paper, we investigate the robustness of various feature extraction methods to mislocalized crater regions. Using features which are robust to localization errors but also choosing a more representative training set has yielded significant performance improvements on an extensive dataset from the Lunar Reconnaissance Orbiter (LRO).

Planetary rovers navigate in extreme environments for which a Global Positioning System (GPS) is unavailable, maps are restricted to relatively low resolution provided by orbital imagery, and compass information is often lacking due to weak or not existent magnetic fields. However, an accurate rover localization is particularly important to achieve the mission success by reaching the science targets, avoiding negative obstacles visible only in orbital maps, and maintaining good communication connections with ground. This paper describes a horizon solution for precise rover orientation estimation. The detected horizon in imagery provided by the on board navigation cameras is matched with the horizon rendered over the existing terrain model. The set of rotation parameters (roll, pitch yaw) that minimize the cost function between the two horizon curves corresponds to the rover estimated pose.

The virtual bumper is a safeguarding mechanism for autonomous and remotely operated robots to conservatively avoid collisions even in the face of sensor uncertainty. In this paper we take a new approach to the virtual bumper system by applying a powerful but rarely examined statistical test. By using a modified version of Wald's sequential probability ratio test, we demonstrate that we can reduce the number of false positives reported by the virtual bumper, thereby saving valuable mission time. We further use the concept of sequential probability ratio to control vehicle speed in the presence of possible obstacles in order to increase certainty about whether or not obstacles are present. With this principled obstacle certainty measure, our new algorithm reduces the chances of collision by approximately 98% relative to traditional virtual bumper safeguarding without speed control.

Horizon line is a promising visual cue which can be exploited for robot localization or visual geo-localization. Prominent approaches to horizon line detection rely on edge detection as a pre-processing step which is inherently a non-stable approach due to parameter choices and underlying assumptions. We present a novel horizon line detection approach which uses machine learning and Dynamic Programming (DP) to extract the horizon line from a classification map instead of an edge map. The key idea is assigning a classification score to each pixel, which can be interpreted as the likelihood of the pixel belonging to the horizon line, and representing the classification map as a multi-stage graph. Using DP, the horizon line can be extracted by finding the path that maximizes the sum of classification scores. In contrast to edge maps which are typically binary (edge vs no-edge) and contain gaps, classification maps are continuous and contain no gaps, yielding significantly better solutions. Using classification maps instead of edge maps allows for removing certain assumptions such as the horizon is close to the top of the image or that the horizon forms a straight line. The purpose of these assumptions is to bias the DP solution but they fail to produce good results when they are not valid. We demonstrate our approach on three different data sets and provide comparisons with a traditional approach based on edge maps. Although our training set is comprised of a very small number of images from the same location, our results illustrate that our method generalizes well to images acquired under different conditions and geographical locations.

This paper introduces an advanced rover localization system suitable for autonomous planetary exploration in the absence of Global Positioning System (GPS) infrastructure. Given an existing terrain map (image and elevation) obtained from satellite imagery and the images provided by the rover stereo camera system, the proposed method determines the best rover location through visual odometry, 3D terrain and horizon matching. The system is tested on data retrieved from a 3 km traverse of the Basalt Hills quarry in California where the GPS track is used as ground truth. Experimental results show the system presented here reduces by over 60% the localization error obtained by wheel odometry.

In 2009, the Lunar Reconnaissance Orbiter (LRO) launched with the Lunar Orbiter Laser Altimeter (LOLA), an instrument that precisely measures the Moon's surface elevation. LOLA is recorded in Lunar polar tracks, making this data particularly sparse in the equatorial regions where NASA has large sets of orbital images captured over the last five decades. The coregistration of orbital images (often taken from an imprecise spacecraft camera pose) and the precise but sparse LOLA measurements is crucial in building large scale, accurate lunar maps that support current and near term NASA and international space agencies' missions to the Moon. In this paper we introduce a novel algorithm for matching orbital images captured during Apollo 15, 16 and 17 missions with LIDAR data captured by the LOLA instrument. The surface normals extracted from each LOLA shot, the Apollo Metric camera pose, Sun position at image caption time and the lunar albedo are used to estimate a synthetic orbital image used as reference. We then use the Gauss-Newton algorithm to precisely align the actual (Apollo) orbital image to this reference image to progressively higher resolution layers of the image pyramid.

Accurate photometric reconstruction of the Lunar surface is important in the context of upcoming NASA robotic missions to the Moon and in giving a more accurate understanding of the Lunar soil composition. This paper describes a novel approach for joint estimation of Lunar albedo, camera exposure time, and photometric parameters that utilizes an accurate Lunar-Lambertian reflectance model and previously derived Lunar topography of the area visualized during the Apollo missions. The method introduced here is used in creating the largest Lunar albedo map (16% of the Lunar surface) at the resolution of 10 meters/pixel.

Bundle adjustment (BA) is the problem of refining viewing and structure estimates in multi-view scene reconstruction subject to a scene model (e.g. a set of geometric constraints). Mismatched interest points cause serious problems for the standard least squares approach, as a single mismatch (i.e. outlier) will affect the entire reconstruction. We propose a novel robust Student's t BA algorithm (RST-BA), using the heavy tailed t-distribution to model reprojection errors. We design a custom algorithm to find the maximum a posteriori (MAP) estimates of the camera and viewing parameters. The algorithm exploits the same structure as L2-BA, matching the performance of fast L2implementations. RST-BA is more accurate than either L2-BA or L2-BA with a σ-edit outlier removal rule for a range of simulated error generation scenarios. RST-BA also achieved better median reproduction error recovery than SBA [1] or SBA with outlier removal for large publicly available datasets.

Vehicle detection from aerial images is becoming an increasingly important research topic in surveillance, traffic monitoring and military applications. The system described in this paper focuses on vehicle detection in rural environments and its applications to oil and gas pipeline threat detection. Automatic vehicle detection by unmanned aerial vehicles (UAV) will replace current pipeline patrol services that rely on pilot visual inspection of the pipeline from low altitude high risk flights that are often restricted by weather conditions. Our research compares a set of feature extraction methods applied for this specific task and four classification techniques. The best system achieves an average 85% vehicle detection rate and 1800 false alarms per flight hour over a large variety of areas including vegetation, rural roads and buildings, lakes and rivers collected during several day time illuminations and seasonal changes over one year.

Generating accurate three dimensional planetary models is becoming increasingly more important as NASA plans manned missions to return to the moon in the next decade. This paper describes a stereo correspondence system for orbital images and focuses on a novel approach for the sub-pixel refinement of the disparity maps. Our method uses a Bayesian formulation that generalizes the Lucas-Kanade method for optimal matching between stereo pair images. This approach reduces significantly the pixel locking effect of the earlier methods and reduces the influence of image noise. The method is demonstrated on a set of high resolution scanned images from the Apollo era missions.

While image clustering has many important applications ranging from personal to Web image management, its use is often limited by the difficulty of extracting reliable semantics from low level image features. The image clusters can be improved by using features extracted from image regions rather than the whole image. Region segmentation can be improved in turn, by considering all images within the same cluster rather than segmenting each image independently. This observation leads to the unified Bayesian framework for image clustering and segmentation presented in this paper. The experimental results, reported using several types of visual feature extractors on a database of Web documents containing over 6000 images, illustrates a significant improvement over existing techniques.

With the proliferation of digital cameras, the size of personal media data such as digital photos, videos, etc. has grown extremely large. The personal nature of the data has heightened the demands for a media management system on personal desktops. Existing solutions for media management target mostly server-based Web databases and rely on extensive metadata (i.e., labels) generation to aid retrieval. Personal media databases, on the other hand, have very limited labels generated by the end users themselves. This paper introduces a method for learning concept templates from web images to query personal image databases. The proposed method has the advantage of leveraging Web resources to ease personal photo retrieval in order to avoid costly annotation of personal image databases.

This paper describes a hierarchical Bayesian network used for segmenting desert images and detecting off road drivable corridors for autonomous navigation. Unlike the embedded hidden Markov model the Bayesian network presented in this paper can successfully account for natural dependencies between neighboring pixels in both image dimensions making it more suitable for a larger class of images. The method described here was developed within the Stanford racing team that won the DARPA Grand Challenge 2005 after driving over 130 miles autonomously in the Nevada desert.

In this paper, we investigate the use of audio and visual rather than only audio features for the task of speech separation in acoustically noisy environments. The success of existing independent component analysis (ICA) systems for the separation of a large variety of signals, including speech, is often limited by the ability of this technique to handle noise. In this paper, we introduce a Bayesian model for the mixing process that describes both the bimodality and the time dependency of speech sources. Our experimental results show that the online demixing process presented here outperforms both the ICA and the audio-only Bayesian model at all levels of noise.

We present the acquisition and content of a multi-stream audio-visual large-vocabulary database in Mandarin Chinese. The database consists of 17,000 utterances spoken by 225 people and captured by a set of seven cameras and 12 microphones. We also provide the label files that describe the endpoints of the utterances and the script files that represent the actual pronunciation of speech. The database can be used in audio-visual speech recognition (AVSR) for both large-vocabulary and small tasks, microphone array based speech recognition, audio-visual speaker identification and 3D face modeling.

This paper describes a Bayesian network for 3D articulated upper body segmentation and tracking from video sequences for which both color and depth information are available. In our upper body model the joints are represented as the parent nodes of the body components nodes which include the head, torso or arms. The upper body components are modeled using a set of planar, linear and Gaussian density functions. The model described in this paper segments and tracks accurately the upper body in different illumination conditions and in the presence of partial occlusions and self occlusions. In addition the current approach allows for automatic segmentation of the upper body without any human intervention allowing for further use of the system in hand gesture or human activity recognition.

In this paper, we investigate the use of the coupled hidden Markov models (CHMM) for the task of audio-visual text dependent speaker identification. Our system determines the identity of the user from a temporal sequence of audio and visual observations obtained from the acoustic speech and the shape of the mouth, respectively. The multi modal observation sequences are then modeled using a set of CHMMs, one for each phoneme-viseme pair and for each person in the database. The use of CHMMs in our system is justified by the capacity of this model to describe the natural audio and visual state asynchrony as well as their conditional dependency over time. To train a CHMM we first train a speaker independent model using expectation-maximization (EM), and then we build a speaker dependent model using maximum a posteriori (MAP) training. Experimental results on XM2VTS database show that our system improves the accuracy of audio-only or video-only speaker identification at all levels of acoustic signal-to-noise ratio (SNR) from 0 to 30 dB.

In recent years several speech recognition systems that use visual together with audio information showed significant increase in performance over the standard speech recognition systems. The use of visual features is justified by both the bimodality of the speech generation and by the need of features that are invariant to acoustic noise perturbation. The audio-visual speech recognition system presented in this paper introduces a novel audio-visual fusion technique that uses a coupled hidden Markov model (HMM). The statistical properties of the coupled-HMM allow us to model the state asynchrony of the audio and visual observations sequences while still preserving their natural correlation over time. The experimental results show that the coupled HMM outperforms the multistream HMM in audio visual speech recognition.

The increase in the number of multimedia applications that require robust speech recognition systems determined a large interest in the study of audio-visual speech recognition (AVSR) systems. The use of visual features in AVSR is justified by both the audio and visual modality of the speech generation and the need for features that are invariant to acoustic noise perturbation. The speaker independent audio-visual continuous speech recognition system presented relies on a robust set of visual features obtained from the accurate detection and tracking of the mouth region. Further, the visual and acoustic observation sequences are integrated using a coupled hidden Markov (CHMM) model. The statistical properties of the CHMM can model the audio and visual state asynchrony while preserving their natural correlation over time. The experimental results show that the current system tested on the XM2VTS database reduces by over 55% the error rate of the audio only speech recognition system at SNR of 0 dB.

The embedded Bayesian networks (EBN) introduced in this paper, are a generalization of the embedded hidden Markov models previously used for face and character recognition. An EBN is defined recursively as a hierarchical structure where the "parent" node is a Bayesian network (BN) that conditions the EBNs or the observation sequence that describes the nodes of the "child" layer. With an EBN, one can model complex N-dimensional data, avoiding the complexity of N-dimensional BNs while still preserving their flexibility and partial scale invariance. In this paper we present an application of the EBNs for face recognition and show the improvement of this approach versus the "eigenface" and the embedded HMM approaches.

With the increase in the computational complexity of recent computers, audio-visual speech recognition (AVSR) became an attractive research topic that can lead to a robust solution for speech recognition in noisy environments. In the audio visual continuous speech recognition system presented in this paper, the audio and visual observation sequences are integrated using a coupled hidden Markov model (CHMM). The statistical properties of the CHMM can describe the asyncrony of the audio and visual features while preserv-ing their natural correlation over time. The experimental re-sults show that the current system tested on the XM2VTS database reduces the error rate of the audio only speech recognition system at SNR of 0db by over 55%. 1.

This paper describes a hidden Markov model-based static and dynamic 3D gesture recognition system. The shape and position of the hands, segmented and tracked using a novel 3D statistical model for the upper body in stereo sequences, are used as observation vectors. The upper body model allows for accurate 3D localization of the hands in the presence of partial occlusions, self occlusions and different illumination conditions. The accuracy of our approach is reflected by the performance of our 3D gesture based editing system, that reaches 96% over 12 dynamic gestures and four static gestures.

The embedded hidden Markov model (HMM) is a statistical model that can be used in many pattern recognition and computer vision applications. This model inherits the partial size invariance of the standard HMM, and, due to its pseudo two-dimensional structure, is able to model two-dimensional data such as images, better than the standard HMM. We describe the maximum likelihood training for the continuous mixture embedded HMM and present the performance of this model for face detection and recognition. The experimental results are compared with other approaches to face detection and recognition.

We describe an embedded hidden Markov model (HMM)-based approach for face detection and recognition that uses an efficient set of observation vectors obtained from the 2D-DCT coefficients. The embedded HMM can model the two dimensional data better than the one-dimensional HMM and is computationally less complex than the two-dimensional HMM. This model is appropriate for face images since it exploits an important facial characteristic: frontal faces preserve the same structure of "super states" from top to bottom, and also the same left-to-right structure of "states" inside each of these "super states".

The work presented in this paper focuses on the use of hidden Markov models for face recognition. A new method based on the extraction of 2D-DCT feature vectors is described, and the recognition results are compared with other face recognition approaches. The method introduced reduces significantly the computational complexity of previous HMM-based face recognition system, while preserving the same recognition rate.

The work presented in this paper describes a hidden Markov model (HMM)-based framework for face recognition and face detection. The observation vectors used to characterize the states of the HMM are obtained using the coefficients of the Karhunen-Loeve transform (KLT). The face recognition method presented reduces significantly the computational complexity of previous HMM-based face recognition systems, while slightly improving the recognition rate. Consistent with the HMM model of the face, this paper introduces a novel HMM-based face detection approach using the same feature extraction techniques used for face recognition.