Communication involves more than simply spoken information. Typical interactions use gestures to accurately and efficiently convey ideas that are more easily expressed with actions than words. A more intuitive interface with machines should involve not only speech recognition, but gesture recognition as well. One of the most frequently used and expressively powerful gestures is pointing. It is far easier and more accurate to point to an object than give a verbal description of its location. To produce a more efficient, accurate, and natural human-machine interface we use the Perseus architecture to interpret the pointing gesture. Perseus uses a variety of techniques to reliably solve this complex visual problem in non-engineered worlds. Knowledge about the task and environment is used at all stages of processing to best interpret the scene for the current situation. Once the visual operators are chosen, contextual knowledge is used to tune them for maximal performance. Redundant interpretation of the scene provides robustness to errors in interpretation. Fusion of independent types of information results in increased tolerance when assumptions about the environment fail. Windows of attention are used to improve speed and remove distractions from the scene. Furthermore, reuse is a major issue in the design of Perseus. Information about the environment and task is explicitly represented so it can easily be re-used in tasks other than pointing. A clean interface to Perseus is provided for symbolic higher level systems like the RAP reactive execution system. In this paper we describe Perseus in detail and show how it is used to locate objects pointed to by people.

As autonomous robots become increasingly adept at performing simple tasks like moving from place to place and picking up and delivering objects, it is becoming apparent that an important area of robotic research is that of developing natural interfaces for controlling them. In the context of building a robot "waiter", the authors demonstrate the use of the Perseus architecture for gesture recognition, teamed with the Animate Agent architecture for tightly coupled perception and action. Of particular significance is the ease of implementing this task utilizing the architectures and routines they have already created for other tasks.

One of the difficult aspects of building an autonomous system, such as a service robot, is representing the plans needed to carry out tasks in unstructured environments. We outline the plans used by the University of Chicago entry in the 1994 AAAI robot competition. The robot was controlled using the Animate Agent Architecture, and the skills and RAPs presented constitute its task knowledge.

Color histogram matching has been shown to be a promising way of quickly indexing into a large image database. Yet, few experiments have been done to test the method on truly large databases, and even if they were performed, they would give little guidance to a user wondering if the technique would be useful with his or her database. In this paper we define and analyze a measure relevant to extending color histogram indexing to large databases: capacity (how many distinguishable histograms can be stored).>

The high-resolution field of view of the human eye only covers a tiny fraction of the total field of view, which allows for great economy in computational resources but forces the visual system to solve other problems that would not exist with uniformly high resolution. One of these is how to determine where to redirect the fovea, given only the low-resolution information available in the periphery. The advent of spatially variant receptor arrays for cameras has made it imperative that computational solutions to this problem be found. Color has been traditionally associated with foveal vision, but it is shown that color cues are well preserved under low resolution, and an algorithm for locating objects based on color histograms that is both effective under low resolution and computationally efficient is illustrated.>

This paper shows color histograms to be stable object representations over change in view, and demonstrates that they can differentiate among a large number of objects. The authors introduce a technique called histogram intersection for efficiently matching model and image histograms. Color can also be used to search for the location of an object. An algorithm called histogram backprojection performs this task efficiently in crowded scenes.>

A geometrical theorem prover based on algebraic techniques has been used to prove and derive theorems in computer vision. The results show that theorems relevant to persepective viewing and the properties of shadows can be proven, including some which contain inequalities. Deriving theorems by stating incomplete hypotheses and letting the prover suggest the missing constraints can provide valuable new insights into vision probiems. However, there is no guarantee that the suggestions made by the proof algorithm are interpretable in terms of primitive geometric concepts.