Road scene analysis is a challenging problem that has applications in autonomous navigation of vehicles. An integral component of this system is the robust detection and tracking of lane markings. It is a hard problem primarily due to large appearance variations in lane markings caused by factors such as occlusion (traffic on the road), shadows (from objects like trees), and changing lighting conditions of the scene (transition from day to night). In this paper, we address these issues through a learning-based approach using visual inputs from a camera mounted in front of a vehicle. We propose the following: 1) a pixel-hierarchy feature descriptor to model the contextual information shared by lane markings with the surrounding road region; 2) a robust boosting algorithm to select relevant contextual features for detecting lane markings; and 3) particle filters to track the lane markings, without knowledge of vehicle speed, by assuming the lane markings to be static through the video sequence and then learning the possible road scene variations from the statistics of tracked model parameters. We investigate the effectiveness of our algorithm on challenging daylight and night-time road video sequences.

In this paper, we describe our recent efforts on symbolic representations enabling autonomous driving in dynamic environments such as on-road driving. In contrast to traditional bottom-up approaches where temporal accumulation of locally insignificant inaccuracies causes eventual failures in high-level scene interpretation, our proposed knowledge-driven top-down approach combined with vehicle's intentions can provide valuable information to guide low-level bottom-up tasks and vice-versa. We contend that such rich symbolic representations can reduce the burden on sensory processing thereby dynamically directing it to look for particular features in expected locations and subsequently facilitating the vehicle to better react to potentially dangerous situations, such as the appearance of pedestrians in the road. We demonstrate the proposed approaches in various scenarios pertaining to vehicle perception and control using field data obtained from a military unmanned ground vehicle (UGV) traversing urban environments

In an outdoor, off-road mobile robotics environment, it is important to identify objects that can affect the vehicle's ability to traverse its planned path, and to determine their three-dimensional characteristics. In the paper, a combination of three elements is used to accomplish this task. An imaging ladar collects range images of the scene. A color camera, whose position relative to the ladar is known, is used to gather color images. Information extracted from these sensors is used to build a world model, a representation of the current state of the world. The world model is used actively in the sensing to predict what should be visible in each of the sensors during the next imaging cycle. The paper explains how the combined use of these three types of information leads to a robust understanding of the local environment surrounding the robotic vehicle for two important tasks: puddle/pond avoidance and road sign detection.

We address the problem of retrieving images from a large database using an image as a query. The method is specifically aimed at databases that store images in JPEG format, and works in the compressed domain to create index keys. A key is generated for each image in the database and is matched with the key generated for the query image. The keys are independent of the size of the image. Images that have similar keys are assumed to be similar, but there is no semantic meaning to the similarity.

This paper describes an algorithm for performing arbitrary translations and rotations of objects represented by octrees. Given an octree in a standard position and a transformation, the algorithm builds a new tree in a top down fashion, visiting each node in the new tree only once, and constructing only those nodes that appear in the final tree. It works by projecting the transformed space over the original tree, and labeling the new nodes according to the labels of the nodes in the underlying untransformed tree.

A robot sensing system is described that uses multiple sources of information to construct an internal representation of its environment. Initially, object models are used to form the basic representations. These are modified by processes that operate on sequences of sensory information, obtained from sensors that move about in the environment. Two representations are constructed. One is a description of the spatial layout of the environment, represented as an octree, while the other is an object- and feature-based representation. The system handles both expected and unexpected objects, and attempts to register its internal representation with the external world using a variety of predictive, sensory-processing, and matching procedures.

The Sensory-Interactive Robotics Group at the National Bureau of Standards is producing PIPE, a pipelined image-processing engine, for research in low-level machine vision. PIPE processes sequences of images at field rates through a series of point and neighborhood operations. It is divided into a variable number of identical stages, each of which performs an independent set of operations on the image data stored in the stage. A stage control unit determines the sequence of operations performed within a stage on each image. This sequence is easily modified by a host computer during the inter-field interval when all of the stage control units can be totally reconfigured. Images flow through PIPE in several ways. In addition to the (standard pipeline) "forward" pathway, where an output image is sent to the next stage, an output image can also be sent to the same stage via a "recursive" pathway and to the previous stage via a "retrograde" pathway. As a result, PIPE can support relaxation operations, temporal neighborhood operations, and other local operations. Several processing modes are available in PIPE in addition to the usual "SIMD" mode of pipelined processors. In an "MIMD" mode, one of several operations is performed on a region of interest which can be defined by the host device or by previous image operations. PIPE also supports variable resolution pyramids where an image is compressed or expanded as it passes between stages.

A method is described of representing curved or line-like segments in images using Chebyshev polynomials. The advantages of the approximations include fast computation minimum error over the sampling interval, and the reduction of the data to a concise description consisting of a few coefficients. An implementation is discussed that uses images obtained from a structured-light ranging system.

This correspondence describes a method of building and maintaining a spatial respresentation for the workspace of a robot, using a sensor that moves about in the world. From the known camera position at which an image is obtained, and two-dimensional silhouettes of the image, a series of cones is projected to describe the possible positions of the objects in the space. When an object is seen from several viewpoints, the intersections of the cones constrain the position and size of the object. After several views have been processed, the representation of the object begins to resemble its true shape. At all times, the spatial representation contains the best guess at the true situation in the world with uncertainties in position and shape explicitly represented. An octree is used as the data structure for the representation. It not only provides a relatively compact representation, but also allows fast access to information and enables large parts of the workspace to be ignored. The purpose of constructing this representation is not so much to recognize objects as to describe the volumes in the workspace that are occupied and those that are empty. This enables trajectory planning to be carried out, and also provides a means of spatially indexing objects without needing to represent the objects at an extremely fine resolution. The spatial representation is one part of a more complex representation of the workspace used by the sensory system of a robot manipulator in understanding its environment.

This correspondence describes a method of extracting compact regions from images using three multiresolution (``pyramid'') representations of image features. The method makes use of a gray-level pyramid in which linking of similar nodes at successive levels smooths the image into uniform regions; an edge pyramid in which linking of similarly oriented edge elements serves to delineate the boundaries of the detected regions; and a ``surroundedness'' pyramid in which pixels surrounded by edges are detected.

A method of detecting blobs in images is described. The method involves building a succession of lower resolution images and looking for spots in these images. A spot in a low resolution image corresponds to a distinguished compact region in a known position in the original image. Further, it is possible to calculate thresholds in the low resolution image, using very simple methods, and to apply those thresholds to the region of the original image corresponding to the spot. Examples are shown in which variations of the technique are applied to several images.

Pictures containing lines and curves are often perceived by humans as being composed of a smaller number of more global figures. For example, a broken set of collinear line segments is perceived as a single straight line, even in the presence of other overlapping line segments. A method of extracting such figures from images automatically in a highly parallel manner is presented in this work. A pyramid of successively lower-resolution images is used to transform the problem from one of global search to one of local good continuation. Using the pyramid, figures that are both sparse and obscured can successfully be extracted and the background clutter can be suppressed.

A method of pairing antiparallel straight lines is presented and discussed. The pairing is based on the distance between the lines, the amount by which they overlap, and on whether or not other lines are interposed. Examples are shown of applying the method to high resolution aerial photographs. Results indicate that cultural features such as roads and buildings can be extracted and that a significant reduction in the complexity of the image description can be obtained.