Traditionally, path planning for field robotic systems is performed in Cartesian space: sensor readings are transformed into terrain costs in a (Cartesian) costmap, and a path to the goal is planned in that map. In this paper, we propose a new approach: planning a path for the robot in the image-space of an on-board camera. We apply a learned color- to-cost mapping to transform a raw image into a cost-image, which then undergoes a pseudo-configuration-space transform. We search in the resulting cost-image for a path to the projected goal point in the image. One benefit of our approach is the ability to react to obstacles at ranges well beyond our 3D sensor range - independent testing has confirmed our system has effectively reacted to obstacles at a range of 93 m while our stereo sensor provides reliable data only up to 5 m away. We describe the details of our technique and the results from testing under the DARPA LAGR and UPI programs.

Driving in unknown natural outdoor terrain is a challenge for autonomous ground vehicles. It can be difficult for a robot to discern obstacles and other hazards in its environment, and characteristics of this high cost terrain may change from one environment to another, or even with different lighting conditions. One successful approach to this problem is for a robot to learn from a demonstration by a human operator. In this paper, we describe an approach to calculating terrain costs from Bayesian estimates using feature vectors measured during a short teleoperated training run in similar terrain and conditions. We describe the theory, its implementation on two different robotic systems, and results of several independently conducted field tests.

Stereo matching in unstructured, outdoor environments is often confounded by the complexity of the scenery and thus may yield only sparse disparity maps. Two-dimensional visual imagery, on the other hand, offers dense information about the environment of mobile robots, but is often difficult to exploit. Training a supervised classifier to identify traversable regions within images that generalizes well across a large variety of environments requires a vast corpus of labeled examples. Autonomous learning of the traversable/untraversable distinction indicated by scene appearance is therefore a highly desirable goal of robot vision. We describe here a system for learning this distinction online without the involvement of a human supervisor. The system takes in imagery and range data from a pair of stereo cameras mounted on a small mobile robot and autonomously learns to produce a labeling of scenery. Supervision of the learning process is entirely through information gathered from range data. Two types of boosted weak learners, Nearest Means and naive Bayes, are trained on this autonomously labeled corpus. The resulting classified images provide dense information about the environment which can be used to fill-in regions where stereo cannot find matches or in lieu of stereo to direct robot navigation. This method has been tested across a large array of environment types and can produce very accurate labelings of scene imagery as judged by human experts and compared against purely geometric-based labelings. Because it is online and rapid, it eliminates some of the problems related to color constancy and dynamic environments.

This paper describes a vision-based perception system which has been used to guide an automated harvester cutting fields of alfalfa hay. The system tracks the boundary between cut and uncut crop; indicates when the end of a crop row has been reached; and identifies obstacles in the harvester's path. The system adapts to local variations in lighting and crop conditions, and explicitly models and removes noise due to shadow. In field tests, the machine has successfully operated in four different locations, at sites in Pennsylvania, Kansas, and California. Using the vision system as the sole means of guidance, over 60 acres have been cut at speeds of up to 4.5 mph (typical human operating speeds range from 3-6 mph). Future work largely centers around combining vision and GPS based navigation techniques to produce a commercially viable product for use either as a navigation aid or for a completely autonomous system.

Automation of agricultural harvesting equipment in the near term appears both economically viable and technically feasible. This paper describes a vision-based algorithm which guides a harvester by tracking the line between cut and uncut crop. Using this algorithm, a harvester has successfully cut roughly one acre of crop to date, at speeds of up to 4.5 miles an hour in an actual alfalfa field. A broad range of methods for detecting the crop cut boundary were considered, including both range-based and vision-based techniques; several of these methods were implemented and evaluated on data from an alfalfa field. The final crop-line detection algorithm is presented, which operates by computing the best-fit step function of a normalized-color measure of each row of an RGB image. Results of the algorithm on some sample crop images are shown, and potential improvements are discussed.