A description is given of the system architecture of an autonomous vehicle and its real-time adaptive vision system for road-following. The vehicle is a 10-ton armored personnel carrier modified for robotic control. A color transformation that best discriminates road and nonroad regions is derived from labeled data samples. A maximum-likelihood pixel classification technique is then used to classify pixels in the transformed color image. The vision system adapts itself to road changes in two ways; color transformation parameters are updated infrequently to accommodate significant road color changes, and classifier parameters are updated every processing cycle to deal with gradual color and intensity changes. To reduce unnecessary computation, only the most likely road region in the segmented image is selected, and a polygonal representation of the detected road region boundary is transformed from the image coordinate system to the local vehicle coordinate system based on a flat-earth assumption.>