A real-time system for the control of an autonomous vehicle consisting of a nested hierarchy of control modules is described. The focus is on the PILOT module which provdies the real-time guidance of the system. It is responsible for the generation and tracking of dynamically feasible trajectories which follow the generally planned path and avoid local obstacles. The PILOT operation is based on a combination of simple rules and heuristic search. The state space consists of two parts: the lower level state space which represents the model of the world as a set of relationships among the units of vehicle state variables representing steering, acceleration, and velocity, as well as positions of obstacles, and the upper level state space where the relationships among generalized objects are represented as rules. Rules provide simple elementary maneuvers, and are applied when the upper level situation is easily recognized. Otherwise the situations are handled by the lower level search. Example runs of the vehicle are simulated to demonstrate the complexity of behavior. The algorithm has also been tested on a gas powered dune buggy developed at Drexel University.