The National Bureau of Standards robot sensory system employs multiple hierarchical levels of sensory interpretation that interact with matching levels of world modeling. At each level, the world-modeling processes generate hypotheses about the sensory data based on a priori knowledge, prior sensory input, and knowledge of robot motion. The sensory-interpretative processes use these hypotheses to facilitate their analyses of new data. The results of the analyses are used by the world-modeling processes to correct their models of the environment. This interaction requires the development of real-time algorithms for the analysis of sensory data that can usefully employ guidance from models. This paper presents an algorithm for accomplishing this at the level of object location and pose determination. Its desirable features include the ability to deal with underconstrained problems, the ability to employ all the data in a structured-light image, and robustness in the face of several types of error and noise.

A robot sensing system is described that uses multiple sources of information to construct an internal representation of its environment. Initially, object models are used to form the basic representations. These are modified by processes that operate on sequences of sensory information, obtained from sensors that move about in the environment. Two representations are constructed. One is a description of the spatial layout of the environment, represented as an octree, while the other is an object- and feature-based representation. The system handles both expected and unexpected objects, and attempts to register its internal representation with the external world using a variety of predictive, sensory-processing, and matching procedures.

The Sensory-Interactive Robotics Group at the National Bureau of Standards is producing PIPE, a pipelined image-processing engine, for research in low-level machine vision. PIPE processes sequences of images at field rates through a series of point and neighborhood operations. It is divided into a variable number of identical stages, each of which performs an independent set of operations on the image data stored in the stage. A stage control unit determines the sequence of operations performed within a stage on each image. This sequence is easily modified by a host computer during the inter-field interval when all of the stage control units can be totally reconfigured. Images flow through PIPE in several ways. In addition to the (standard pipeline) "forward" pathway, where an output image is sent to the next stage, an output image can also be sent to the same stage via a "recursive" pathway and to the previous stage via a "retrograde" pathway. As a result, PIPE can support relaxation operations, temporal neighborhood operations, and other local operations. Several processing modes are available in PIPE in addition to the usual "SIMD" mode of pipelined processors. In an "MIMD" mode, one of several operations is performed on a region of interest which can be defined by the host device or by previous image operations. PIPE also supports variable resolution pyramids where an image is compressed or expanded as it passes between stages.