Virtual environments provide a whole new way of viewing and manipulating 3D data. Current technology moves the images out of desktop monitors and into the space immediately surrounding the user. Users can literally put their hands on the virtual objects.

Computer systems which analyse human face/head motion have attracted significant attention recently as there are a number of interesting and useful applications. Not least among these is the goal of tracking the head in real time. A useful extension of this problem is to estimate the subject's gaze point in addition to his/her head pose. This paper describes a real-time stereo vision system which determines the head pose and gaze direction of a human subject. Its accuracy makes it useful for a number of applications including human/computer interaction, consumer research and ergonomic assessment.

We have designed and implemented a real-time binocular tracking system which uses two independent cues commonly found in the primary functions of biological visual systems to robustly track moving targets in complex environments, without a-priori knowledge of the target shape or texture: a fast optical flow segmentation algorithm quickly locates independently moving objects for target acquisition and provides a reliable velocity estimate for smooth tracking. In parallel, target position is generated from the output of a zero-disparity filter where a phase-based disparity estimation technique allows dynamic control of the camera vergence do adapt the horopter geometry to the target location. The system takes advantage of the optical properties of our custom-designed foveated wide-angle lenses, which exhibit a wide field of view along with a high resolution fovea. Methods to cope with the distortions introduced by the space-variant resolution, and a robust real-time implementation on a high performance active vision head are presented.

Achieving the first step of a framework for human-robot interaction, we have designed a binocular tracking system which uses disparity and velocity information for the detection and pursuit of moving objects in cluttered environments without a-priori knowledge of the target shape or texture. The implemented system robustly tracks in real-time deformable objects such as human hands and faces, taking advantage of the mechanical and optical properties of ESCHeR, a high performances active vision head equipped with foveated wide-angle lenses.

Inspired by the properties of the human visual system, a new active vision system called ESCHeR (Etl Stereo Compact Head For Robot Vision) has been recently implemented with foveated wide angle lenses. The lenses exhibit a wide field of view along with a space-varying resolution for facilitating both detection and close observation. However, to handle such optical properties and achieve basic eye movement functions, new calibration methods are needed. Therefore, two novel and online techniques are presented that in one case perform a global identification of the optical process through artificial neural techniques and in the other case compute the physical parameters by using environmental feature-tracking and controlled rotations of the cameras. Self-alignment of the cameras is also achieved using a similar technique.

A novel framework for multiple robot cooperation called "cooperation by observation" is presented. It introduces many interesting issues such as a viewpoint constraint and role interchange, as well as novel concepts like "attentional structure". The framework has the potential to realize a high level of task coordination by decentralized autonomous robots allowing minimum explicit communication. Its source of power lies in an advanced capability given to each robot for recognizing other agent's actions by (primarily visual) observation. This provides rich information about the current task situation around each robot which facilitates highly-structured task coordination. The basic visuo-motor routines are described. Concrete examples and experiments using real mobile robots are also presented.>

This paper presents some advanced examples of reactive vision-based cooperative behaviors: 1) chasing and posing against another robot among others; 2) unblocking the path of another robot by removing an obstacle; and 3) passing an object from one to another. These behaviors are demonstrated using real mobile robots equipped with CCD cameras, in a complex environment, and with no central controller or explicit communication among the robots. The action observation is based on real time processing of optical flow analysis and stereo tracking. An extended behavior-based architecture for "cooperation by observation" is presented. The core extension consists of a mobile space buffer and an image space buffer with manipulable markers which control the internal flow of information, thereby coordinating parallel behaviors and achieving purposive tasks in complex and dynamic environments.>

This paper presents a stereo active vision system which performs tracking tasks on smoothly moving objects in complex backgrounds. Dynamic control of the vergence angle adapts the horopter geometry to the target position and allows to pick it up easily on the basis of stereoscopic disparity features. We introduce a novel vergence control strategy based on the computation of "virtual horopters" to track a target movement generating rapid changes of disparity. The control strategy is implemented on a binocular head, whose right and left pan angles are controlled independently. Experimental results of gaze holding on a smoothly moving target translating and rotating in a complex surrounding demonstrate the efficiency of the tracking system.>