The paper gives a survey on the new Expectation-based Multifocal Saccadic vision (EMS-vision) system for autonomous vehicle guidance developed at the Universitat der Bundeswehr Munchen (UBM). EMS-Vision is the third generation dynamic vision system following the 4-D approach. Its core element is a new camera arrangement, mounted on a high bandwidth pan-tilt head for active gaze control. Central knowledge representation and a hierarchical system architecture allow efficient activation and control of behavioral capabilities for perception and action. The system has been implemented on commercial off-the-shelf (COTS) hardware components in both UBM test vehicles VaMoRs and VaMP Results from autonomous turnoff maneuvers, performed on army proving grounds, are discussed.

An intelligent cruise control system for automatic vehicle guidance on German 'autobahnen' is discussed. Recent developments have focused on the vision system, which has reached a new level of performance, compactness and robustness by the use of high performance computers. The system has been integrated in VaMP, the autonomous vehicle of UBM. Extensive test driving on German, French and Danish 'autobahnen' has been performed. Finally, a long-distance ride from Munich (Germany) to Odense (Denmark) is discussed.

In machine vision applications controllable vision sensors improve essentially the performance of the system. The design and realization of a camera platform is presented in this paper. Two modes of operation are important, fast gaze shifts to bring regions of interest into the field of view and smooth pursuit of a moving target. In addition, image stabilisation is needed for application of the camera platform in a vehicle. Using visual feedback in the control loop requires careful consideration of all effects like time and speed restrictions for real time control, image processing delays, interfacing the controller to image processing, smoothing the camera motion and rejection of disturbances. These control aspects are addressed in this paper.>

An integrated system for visual road vehicle guidance has been created on transputer-hardware and partly validated in normal Autobahn traffic environments. The computer vision and dynamic scene understanding components are discussed. The system is capable of recognizing horizontal and vertical curvature of the lane, lane width, number of lanes, the state of the vehicle relative to the lane, and the relative state of two other objects by recursive monocular estimation following the 4-D approach as a unifying scheme for 3-D dynamic scene understanding through motion stereo. The architecture realized on transputer hardware is described. Experimental results with a 5-ton van are discussed.>

An integrated spatio-temporal approach to real-time monocular vision in combination with a &dimensional fast symmetry analysis is presented for obstacle recognition and relative state estimation. This module is one pard of Q system for autonomous vehicle guidance. Obstacles i.e other vehicles are detected and tracked up to a distance of 100 meters. The system is able to handle multiple objects. The number is limited by hardware resources.

The 4D approach to dynamic machine vision has been validated for the application area of on-board autonomous landing approaches in the visual flight regime with computing technology available today; sensors are a video-camera, inertial gyros and an air velocity meter. The key feature of the method is the reconstruction and servo-maintained adjustment by prediction error feedback of an internal spatio-temporal model about the process to be controlled. This encompasses both the egomotion state of the aircraft carrying the sensors and the relevant geometric properties of the runway and its spatial environment. The efficiency of the approach is proved both in a hardware-in-the-loop simulation and in real test-flights with a twin-turbo-prop aircraft Do 128 of Dornier. For accuracy evaluation of the data gathered, the results of differential GPS and radiometric altitude measurements have been recorded simultaneously.>

The general problem of recognizing both horizontal and vertical road curvature parameters while driving along the road has been solved recursively. A differential geometry representation decoupled for the two curvature components has been selected. Based on the planar solution of E.D. Dickmanns and A. Zapp (1986) and its refinements, a simple spatio-temporal model of the driving process makes it possible to take both spatial and temporal constraints into account effectively. The estimation process determines nine road and vehicle state parameters recursively at 25 Hz (40 ms) using four Intel 80286 and one 386 microprocessors. Results with the test vehicle (VaMoRs), which is a 5-ton van, are given for a hilly country road.>

The 4-D approach to dynamic real-time machine vision is reviewed. The influence of control integration on the development of behavioral skills and intelligence are emphasized. This approach leads to a processing structure oriented towards the internal representation of physical objects in 3-D space and time. Results achieved in road vehicle guidance are discussed; this includes convoying with no direct range measurement.>

The Kalman filter approach to recursive state estimation making use of dynamic models for the motion of massive objects has been extended to image sequence processing. This confines image processing to the last frame of the sequence only, and derives a direct spatial interpretation including spatial velocity components by smoothing integrations of prediction errors. Results are presented for road-vehicle guidance at high speeds including obstacle detection and monocular relative spatial state estimation. The corresponding data-processing architecture is discussed; the system has been implemented on a MIMD parallel processing system. Speeds up to 100 km/h have been demonstrated.>

A novel method for dynamic scene analysis by computer vision is described that combines 3-D shape models, dynamical models as known from modern control theory and the laws of perspective projection. To arrive at numerically efficient real-time algorithms, the recursive state estimation by Kalman filtering is adapted to a feature-based image sequence analysis scheme. The spatial and temporal constraint propagation using an integral spatiotemporal model yields image evaluation cycle times of about 0.1 s for simple but realistic tasks with microprocessors available today. Motion control in the dynamic range of humans is thereby possible. Applications discussed are: three-degree-of-freedom planar docking, road vehicle guidance at speeds up to 60 mph and six-degree-of-freedom landing approach of a business jet plane (hardware in the loop simulation).>