In this paper, we propose a bottom-up approach to generating short descriptive sentences from images, to enhance scene understanding. We demonstrate automatic methods for mapping the visual content in an image to natural spoken or written language. We also introduce a human-in-the-loop evaluation strategy that quantitatively captures the meaningfulness of the generated sentences. We recorded a correctness rate of 60.34% when human users were asked to judge the meaningfulness of the sentences generated from relatively challenging images. Also, our automatic methods compared well with the state-of-the-art techniques for the related computer vision tasks.

The problem of a moving robot tracking a moving object with its cameras, without requiring the ability to recognize the target to distinguish it from distracting surroundings, is examined. A novel aspect of the approach taken is the use of controlled camera movements to simplify the visual processing necessary to keep the cameras locked on the target. A gaze-holding system implemented on a robot's binocular head demonstrates this approach. Even while the robot is moving, the cameras are able to track an object that rotates and moves in three dimensions. The central idea is that localizing attention on 3D space makes simple precategorical visual processing sufficient to hold gaze.>

Multibehavior control is explored in a simulation of gaze control using saccadic, vergence, pursuit, vestibulo-ocular reflex, and head control. In gaze control, several simple reflexes (or behaviors) cooperate to achieve effects such as gaze shifts, object tracking, and gaze stabilization. In the simulation, the behaviors acted like a multirate, multiinput, multioutput, time-delayed, saturated control system. Explicit coordination between behaviors was needed to let them cooperate effectively (stably, accurately) in the presence of time delays and control interactions. The coordination in this case is through a shared data structure that predicts the future state of the active agent. A behavior is assumed to know the maximum time delay of any behavior with which it shares an output. The resulting control is a form of the Smith predictor and seems to offer one general solution for this type of problem.>