This paper describes an interactive model-based vision system for vehicle tracking. A human specifies a limited amount of information in the form of object models, which establish a context for autonomous interpretation of scenes containing moving vehicles. Results are presented from several image sequences shot with hand-held uncalibrated cameras.

The history of the image understanding environment (IUE) project, a five-year program to develop a common software environment for the development of algorithms and application systems, is reviewed. An overview of some of the data structures that are currently evolving as a specification for the IUE is provided. The ultimate goal of the project is to provide the basic data structures and algorithms that are required to carry state-of-the-art research in image understanding.>

The architecture of an image understanding environment is presented. The use of object-oriented programming (OOP) principles in its design and programming constructs is described. It is based upon a small number of components that have been found useful in previous environments and stresses the use of OOP principles. The inheritance mechanisms associated with OOP provide an extendable hierarchy of image understanding objects. This allows the common operations over a class of object to be cleanly expressed. Another feature of OOP, encapsulation, allows the hiding of the implementation of a particular object with respect to its interactions with other objects.>

The organization and the functional components of software environments that have been developed for image understanding (IU) are described. The basic components of general IU environments are detailed with respect to representations, programming constructs, system-specific databases and user interfaces. How these components have been realized in different systems through examples and possible developments in the near future are discussed.>

We present an architecture for terrain recognition for an autonomous land vehicle. Basic components of this are a set of data bases for generic object models, perceptual structures, temporary memory for the instantiation of object and relational hypothesis, and a long term memory for storing stable hypothesis which are affixed to the terrain representation. Different inference processes operate over these data bases. We describe components of this architecture: the perceptual structure data base, the grouping processes that operate over this, and schemas. We conclude with a processing example for matching predictions from the long term terrain model to imagery and extracting significant perceptual structures for consideration as potential landmarks.