Road scene analysis is a challenging problem that has applications in autonomous navigation of vehicles. An integral component of this system is the robust detection and tracking of lane markings. It is a hard problem primarily due to large appearance variations in lane markings caused by factors such as occlusion (traffic on the road), shadows (from objects like trees), and changing lighting conditions of the scene (transition from day to night). In this paper, we address these issues through a learning-based approach using visual inputs from a camera mounted in front of a vehicle. We propose the following: 1) a pixel-hierarchy feature descriptor to model the contextual information shared by lane markings with the surrounding road region; 2) a robust boosting algorithm to select relevant contextual features for detecting lane markings; and 3) particle filters to track the lane markings, without knowledge of vehicle speed, by assuming the lane markings to be static through the video sequence and then learning the possible road scene variations from the statistics of tracked model parameters. We investigate the effectiveness of our algorithm on challenging daylight and night-time road video sequences.

Flash LADAR cameras based on continuous-wave, time-of-flight range measurement deliver fast 3D imaging for robot applications including mapping, localization, obstacle detection and object recognition. The accuracy of the range values produced depends on characteristics of the scene as well as dynamically adjustable operating parameters of the cameras. In order to optimally set these parameters during camera operation we have devised and implemented an optimization algorithm in a modular, extensible architecture for real-time applications including robot control. The approach uses two components: offline nonlinear optimization to minimize the range error for a training set of simple scenes followed by an online, real-time algorithm to reference the training data and set camera parameters. We quantify the effectiveness of our approach and highlight topics of interest for future research.

As unmanned ground vehicles take on more and more intelligent tasks, determination of potential obstacles and accurate estimation of their position become critical for successful navigation and path planning. The performance analysis of obstacle mapping and unmanned vehicle positioning in outdoor environments is the subject of this paper. Recently, the National Institute of Standards and Technology's (NIST) Intelligent Systems Division has been a part of the Defense Advanced Research Project Agency LAGR (Learning Applied to Ground Robots) Program. NIST's objective for the LAGR Project is to insert learning algorithms into the modules that make up the NIST 4D/RCS (Four Dimensional/Real-Time Control System) standard reference model architecture which has been successfully applied to many intelligent systems. We detail world modeling techniques used in the 4D/RCS architecture and then analyze the high precision maps generated by the vehicle world modeling algorithms as compared to ground truth obtained from an independent differential GPS system operable throughout most of the NIST campus. This work has implications, not only for outdoor vehicles but also, for indoor automated guided vehicles where future systems will have more and more onboard intelligence requiring non-contact sensors to provide accurate vehicle and object positioning

An iterative temporal registration algorithm is presented in this paper for registering 3D range images obtained from unmanned ground and aerial vehicles traversing unstructured environments. We are primarily motivated by the development of 3D registration algorithms to overcome both the unavailability and unreliability of Global Positioning System (GPS) within required accuracy bounds for unmanned ground vehicle (UGV) navigation. After suitable modifications to the well-known iterative closest point (ICP) algorithm, the modified algorithm is shown to be robust to outliers and false matches during the registration of successive range images obtained from a scanning LADAR rangefinder on the UGV. Towards registering LADAR images from the UGV with those from an unmanned aerial vehicle (UAV) that flies over the terrain being traversed, we then propose a hybrid registration approach. In this approach to air to ground registration to estimate and update the position of the UGV, we register range data from two LADARs by combining a feature-based method with the aforementioned modified ICP algorithm. Registration of range data guarantees an estimate of the vehicle's position even when only one of the vehicles has GPS information. Temporal range registration enables position information to be continually maintained even when both vehicles can no longer maintain GPS contact. We present results of the registration algorithm in rugged terrain and urban environments using real field data acquired from two different LADARs on the UGV.

In an outdoor, off-road mobile robotics environment, it is important to identify objects that can affect the vehicle's ability to traverse its planned path, and to determine their three-dimensional characteristics. In the paper, a combination of three elements is used to accomplish this task. An imaging ladar collects range images of the scene. A color camera, whose position relative to the ladar is known, is used to gather color images. Information extracted from these sensors is used to build a world model, a representation of the current state of the world. The world model is used actively in the sensing to predict what should be visible in each of the sensors during the next imaging cycle. The paper explains how the combined use of these three types of information leads to a robust understanding of the local environment surrounding the robotic vehicle for two important tasks: puddle/pond avoidance and road sign detection.

The lure of using motion vision as a fundamental element in the perception of space drives this effort to use flow features as the sole cues for robot mobility. Real-time estimates of image flow and flow divergence provide the robot's sense of space. The robot steers down a conceptual corridor, comparing left and right peripheral flows. Large central flow divergence warns the robot of impending collisions at "dead ends". When this occurs, the robot turns around and resumes wandering. Behavior is generated by directly using flow-based information in the 2D image sequence. Active mechanical gaze stabilization simplifies the visual interpretation problems by reducing camera rotation. By combining corridor following and dead-end deflection, the robot has wandered around the lab at 30 cm/s for as long as 20 min without collision. The ability to support this behavior in real-time with current equipment promises expanded capabilities as computational power increases in the future.

A real-time robot vision system is described which uses only the divergence of the optical flow field for both steering control and collision detection. The robot has wandered about the lab at 20 cm/s for as long as 26 minutes without collision. The entire system is implemented on a single ordinary UNIX workstation without the benefit of real-time operating system support. Dense optical flow data are calculated in real-time across the entire wide-angle image. The divergence of this optical flow field is calculated everywhere and used to control steering and collision behavior. Divergence alone has proven sufficient for steering past objects and detecting imminent collision. The major contribution is the demonstration of a simple, robust, minimal system that uses flow-derived measures to control steering and speed to avoid collision in real time for extended periods. Such a system can be embedded in a general, multi-level perception/control system.

This paper is concerned with the task of visual motion-based navigation. A critical requirement of the task is the ability to estimate 3D depth and motion from visual information. Recent studies have demonstrated that the relevant cues are contained in motion parallax or optical flow from which flow field divergence and hence time-to-contact can be extracted. We present a new concept called image gradient evolution, which utilizes the change of image spatial gradients over time as a threat cue; an approaching object induces 2D expanding motion and causes the image spatial structure to stretch so the image gradients decrease. Based on this idea, other method offers a one-step solution directly from image gradients, instead of optical flow and its derived properties. We use a technique that is local and linear so the implementation can be very fast. The threat map is expectedly noisy but sufficiently informative. As seen in demonstrations on several real images these two aspects, fast implementation and useful qualitative information, provide a viable solution to navigational tasks.

The lure of using motion vision as a fundamental element in the perception of space drives this effort to use flow features as the sole cues for robot mobility. Real-time estimates of image flow and flow divergence provide the robot's sense of space. The robot steers down a conceptual corridor comparing left and right peripheral flows. Large central flow divergence warns the robot of impending collisions at "dead ends." When this occurs, the robot turns around and resumes wandering. Behavior is generated by directly using flow-based information in the 2D image sequence; no 3D reconstruction is attempted. Active mechanical gate stabilization simplifies the visual interpretation problems by reducing camera rotation. By combining corridor following and dead-end deflection, the robot has wandered around the lab at 30 cm/s for as long as 20 minutes without collision. The ability to support this behavior in real-time with current equipment promises expanded capabilities as computational power increases in the future.>

An image is ideally a projection of the 3-D scene. However the imaging process is always imperfect and constrained by the physical environment, for example, viewing through a window with reflections. This paper is concerned with image sequences acquired in such situations, the so-called transparency. When it occurs, the image sequence contains undesirable transparent motion, for example, of the window reflections. This complicates the already difficult motion estimation problem. We present an algorithm to segment transparent motion based on a spatio-temporal filtering technique-3-D Hermite polynomial differentiation filters. With motion segmentation accomplished, we can then focus on the scene analysis. The implementation of our algorithm is fast and accurate.

Classic optical flow algorithms assume local image translational motion and apply some primitive image smoothing. Recent studies have taken two separate approaches toward improving accuracy: the application of spatio-temporal filtering schemes and the use of generalized motion models such as the affine model. Each has achieved improvement in its specialized situations. We analyze the interdependency between them and propose a unified theory. The generalized motion we adopt models arbitrary 3D steady motion. Under perspective projection, we derive an image motion equation that describes the spatio-temporal relation in an image sequence, thus making 3D spatio-temporal filtering possible. Hence we establish a theory of Hermite polynomial differentiation filters, whose orthogonality and Gaussian derivative properties ensure numerical stability. The use of higher order motion constraint equations to accommodate more complex motion is justified by the algorithm's reliable performance, as demonstrated by evaluating our algorithm in the scheme established by Barron, et al. (1994).

A technique for the calibration of an active camera system is presented. The calibration of manipulator, camera-to-manipulator, camera, and base-to-world is treated in a unified and elegant way. In this approach, the camera frames and manipulator link frames are all related to the world frame, therefore the camera-to-manipulator and base-to-world calibration is very straightforward. The approach is simple, since it uses the form of one equation solving one parameter. Two experiments that verify the accuracy of the technique are reported.>

To operate autonomous vehicles safely, obstacles must be detected before any path planning and obstacle avoidance activity is undertaken. In the paper, a novel approach to obstacle detection is developed. New visual linear invariants based on optical flow have been developed. Employing the linear invariance property, obstacles an be directly detected by using a reference flow line obtained from measured optical flow. This method can be used for ground vehicles to navigate through man-made roadways or natural outdoor terrain or for air vehicles to land on known or unknown terrain.>

Motion relative to a surface is addressed. Image flow can be caused either by motion of objects in the world or by motion of the eye through the world. If knowledge exists of eye motion, dense range maps can be computed locally for all stationary object pixels. Range from image flow can be computed either by gradient methods or by correlation methods. The former are simple but subject to quantization noise. The latter are complex but much more accurate. If eye rotation is known, image flow due to eye translation can be separated from flow due to rotation, and moving objects can be segmented from a stationary background. Topics discussed include the equations of image motion on the egosphere, the method for determining optical flow of B.K.P. Horn and B.G. Schunck (1981), cross correlation, image enhancement by scanning, obstacle avoidance navigation and the temporal integration of image flow.>

As part of the Multiple Autonomous Underwater Vehicle Project, a spatial mapping system has been developed to provide a model of the underwater environment suitable for autonomous navigation. The system is composed of multiresolution depth maps designed to integrate sensor data with an a priori model, an object/attribute database for storing information about detected objects, and a set of flags to monitor abnormal or emergency conditions in the environment. The structure of the mapping system and the algorithms used to map terrain and obstacles detected by acoustic sonar are described.>

This paper describes an algorithm for performing arbitrary translations and rotations of objects represented by octrees. Given an octree in a standard position and a transformation, the algorithm builds a new tree in a top down fashion, visiting each node in the new tree only once, and constructing only those nodes that appear in the final tree. It works by projecting the transformed space over the original tree, and labeling the new nodes according to the labels of the nodes in the underlying untransformed tree.

A robot sensing system is described that uses multiple sources of information to construct an internal representation of its environment. Initially, object models are used to form the basic representations. These are modified by processes that operate on sequences of sensory information, obtained from sensors that move about in the environment. Two representations are constructed. One is a description of the spatial layout of the environment, represented as an octree, while the other is an object- and feature-based representation. The system handles both expected and unexpected objects, and attempts to register its internal representation with the external world using a variety of predictive, sensory-processing, and matching procedures.

A method is described of representing curved or line-like segments in images using Chebyshev polynomials. The advantages of the approximations include fast computation minimum error over the sampling interval, and the reduction of the data to a concise description consisting of a few coefficients. An implementation is discussed that uses images obtained from a structured-light ranging system.

This correspondence describes a method of building and maintaining a spatial respresentation for the workspace of a robot, using a sensor that moves about in the world. From the known camera position at which an image is obtained, and two-dimensional silhouettes of the image, a series of cones is projected to describe the possible positions of the objects in the space. When an object is seen from several viewpoints, the intersections of the cones constrain the position and size of the object. After several views have been processed, the representation of the object begins to resemble its true shape. At all times, the spatial representation contains the best guess at the true situation in the world with uncertainties in position and shape explicitly represented. An octree is used as the data structure for the representation. It not only provides a relatively compact representation, but also allows fast access to information and enables large parts of the workspace to be ignored. The purpose of constructing this representation is not so much to recognize objects as to describe the volumes in the workspace that are occupied and those that are empty. This enables trajectory planning to be carried out, and also provides a means of spatially indexing objects without needing to represent the objects at an extremely fine resolution. The spatial representation is one part of a more complex representation of the workspace used by the sensory system of a robot manipulator in understanding its environment.

This correspondence describes a method of image segmentation based on a ``pyramid'' of reduced-resolution versions of the image. It defines link strengths between pixels at adjacent levels of the pyramid, based on proximity and similarity, and iteratively recomputes the pixel values and adjusts the link strengths. After a few iterations, the strengths stabilize, and the links that remain strong define subtrees of the pyramid; the leaves of each tree are the pixels belonging to a compact (piece of a) homogeneous region in the image.

This correspondence describes a method of extracting compact regions from images using three multiresolution (``pyramid'') representations of image features. The method makes use of a gray-level pyramid in which linking of similar nodes at successive levels smooths the image into uniform regions; an edge pyramid in which linking of similarly oriented edge elements serves to delineate the boundaries of the detected regions; and a ``surroundedness'' pyramid in which pixels surrounded by edges are detected.

Pictures containing lines and curves are often perceived by humans as being composed of a smaller number of more global figures. For example, a broken set of collinear line segments is perceived as a single straight line, even in the presence of other overlapping line segments. A method of extracting such figures from images automatically in a highly parallel manner is presented in this work. A pyramid of successively lower-resolution images is used to transform the problem from one of global search to one of local good continuation. Using the pyramid, figures that are both sparse and obscured can successfully be extracted and the background clutter can be suppressed.

A simple class of piecewise constant approximations to an image is constructed as follows: start with the entire image, subdivide it into quadrants if its gray level standard deviation is high, and repeat the process for each quadrant. This yields a decomposition of the image into blocks, each having low standard deviation, so that each of them can be approximated by a constant value, namely, its mean. The histogram of this approximated image tends to have sharper peaks than that of the original image since the block averaging reduces the variability of the gray levels within homogeneous regions. A possible way of further improving the histogram is based on the fact that small blocks tend to occur near region borders; thus, suppressing these blocks should tend to deepen the valleys on the histogram, making threshold selection (to separate regions of different types) easier. Conversely, the histogram of the small blocks only represents a population of pixels near region borders, and if there are only two types of regions (e.g., objects and background), the mean of this histogram should be a useful threshold for separating them; but in practice, this method is not very reliable since background fluctuations also give rise to border pixels.

A recently developed method of image smoothing and segmentation makes use of a "pyramid" of images at successively lower resolutions. It establishes links between pixels at successive levels of the pyramid; the subtrees of the pyramid defined by these links yield a segmentation of the image into regions over which the smoothing takes place. This paper investigates several variations on the basic linking process with regard to such factors as initialization, criteria for linking, and iteration scheme used. It also studies generalizations in which the links are weighted rather than forced, and in which interactions among the pixels at a given level are also allowed. Finally, it extends the approach to links based on more than one feature of a pixel, e.g., on color components or local property values.

The task of segmenting an image and that of estimating properties of image regions may be highly interdependent. The goal of segmentation is to partition the image into regions with more or less homogeneous properties; but the processes which estimate these properties should be confined within individual regions. A cooperative, iterative approach to segmentation and property estimation is defined; the results of each process at a given iteration are used to adjust the other process at the next iteration. A linked pyramid structure provides a framework for this process iteration. This hierarchical structure ensures rapid convergence even with strictly local communication between pyramid nodes.