The visual robustness of biological systems is in part due to their ability to actively integrate (fuse) information from a number of visual cues. In addition to active integration, the perception-action nature of biological vision demands event-driven behavioral composition. Providing mechanical vision systems with similar capabilities therefore requires tools and techniques for cue integration and behavioral composition. In this paper, we address two issues. First, we present a unified approach for handling both active integration and behavioral composition. The approach combines a theoretical framework that handles uncertainty using a voting scheme with a set of behaviors that are committed to achieving a specific goal through common effort and a well-known process composition model. Secondly, we address the issue of integration in the active vision activity of smooth pursuit. We have experimented with the fusion of four smooth pursuit techniques (blob tracking, edge tracking, template matching and image differencing). We discuss each technique, highlighting their strengths and weaknesses, and then show that fusing the techniques according to our formal framework improves system tracking behavior.

In this paper we present a new active vision technique called zoom tracking. Zoom tracking is the continuous adjustment of a camera's focal length, to keep a constant-sized image of an object moving along the camera's optical axis. Two methods for performing zoom tracking are presented: a closed-loop visual feedback algorithm based on optical flow, and use of depth information obtained from an autofocus camera's range sensor. We show that the image stability provided by zoom tracking improves the performance of algorithms that are scale variant, such as correlation-based trackers. While zoom tracking cannot totally compensate an object's motion, due to the effect of perspective distortion, an analysis of this distortion provides a quantitative estimate of the performance of zoom tracking.

The purpose of this paper is twofold: we first present a novel architecture for real-time active vision systems, and then enhance the architecture with a unified approach to fault tolerance. Our system is designed modularly in order to enable the flexible addition of hardware and software redundancy and also to allow reconfiguration when and where needed. This gives us the ability to handle faults in the context of active vision.

In this paper we present a technique that can help us design system components (behaviors) with reliability in mind. Our approach is similar to N-version programming. A team of functionally redundant behaviors vote for a set of possible actions. A voter then fuses the votes and selects the action which has received the maximum number of votes. We draw some design guidelines and show that with careful design, the selected action will have higher probability of success (reliability) than the reliability of each of the behaviors. A comprehensive set of experiments together with results are presented in which a team of tracking behaviors control a stereo camera head in the active vision task of smooth pursuit. The results show that the approach dramatically improves tracking performance.

The active vision paradigm couples perception and action at several different levels. The effective use of active vision in complex robotic tasks requires that these levels operate both independently and cooperatively, reliably and in real-time. In this paper, we present a system for real-time active vision with fault tolerance. The system provides a vocabulary of active vision routines along with the means for composing the routines into continuously running perception-action processes. A novel architecture which enables the integration of the perceptive capabilities of real-time active vision with the active capabilities of other robotic devices is presented. We then enhance the architecture with a unified approach to fault tolerance and present results from experiments and simulations.

In this paper, we present an agent architecture/active vision research tool called the Active Vision Shell (AV-shell). The AV-shell can be viewed as a programming framework for expressing perception and action routines in the context of situated robotics. The AV-shelf is a powerful interactive C-shell style interface providing many capabilities important in an agent architecture such as the ability to combine perceptive capabilities of active vision with capabilities provided by other robotic devices, the ability to interact with a wide variety of active vision devices, a set of image routines and the ability to compose the routines into continuously running perception action processes. Finally, we present an application example of AV-shell.

In order for a robot to operate autonomously in its environment, it must be able to perceive its environment and take actions based on these perceptions. Recognizing the functionalities of objects is an important component of this ability. In this paper, we look into a new area of functionality recognition: determining the function of an object from its motion. Given a sequence of images of a known object performing some function, we attempt to determine what that function is. We show that the motion of an object, when combined with information about the object and its normal uses, provides us with strong constraints on possible functions that the object might be performing.