Learning to recognize new objects in real time in unconstrained environments presents significant challenges for robotic platforms. We present a meta-learning solution to this problem as well as a registered image and events dataset to facilitate work in this domain. Our solution uses interactive motion to isolate the object, and motion-based saliency (from events) to select relevant keypoints from a high-resolution RGB image. Salient keypoints are then passed to a meta-learner to classify the object type. We show that using our interactive isolation and keypoint selection approach, we outperform existing techniques by 6-20%.

Tracking people and objects is an enabling technology for many robotic applications. From human-robot-interaction to SLAM, robots must know what a scene contains and how it has changed, and is changing, before they can interact with their environment. In this paper, we focus on the tracking necessary to record the 3D position and pose of objects as they change in real time. We develop a tracking system that is capable of recovering object locations and angles at speeds in excess of 60 frames per second, making it possible to track people and objects undergoing rapid motion and acceleration. Results are demonstrated experimentally using real objects and people and compared against ground truth data.

How do children learn how to play hide and seek? At age 3-4, children do not typically have perspective taking ability, so their hiding ability should be extremely limited. We show through a case study that a 3 1/2 year old child can, in fact, play a credible game of hide and seek, even though she does not seem to have perspective taking ability. We propose that children are able to learn how to play hide and seek by learning the features and relations of objects (e.g., containment, under) and use that information to play a credible game of hide and seek. We model this hypothesis within the ACT-R cognitive architecture and put the model on a robot, which is able to mimic the child's hiding behavior. We also take the "hiding" model and use it as the basis for a "seeking" model. We suggest that using the same representations and procedures that a person uses allows better interaction between the human and robotic system.

In conversation, people often use spatial relationships to describe their environment, e.g., "There is a desk in front of me and a doorway behind it," and to issue directives, e.g., "go around the desk and through the doorway." In our research, we have been investigating the use of spatial relationships to establish a natural communication mechanism between people and robots, in particular, for novice users. In this paper, the work on robot spatial relationships is combined with a multimodal robot interface. We show how linguistic spatial descriptions and other spatial information can be extracted from an evidence grid map and how this information can be used in a natural, human-robot dialog. Examples using spatial language are included for both robot-to-human feedback and also human-to-robot commands. We also discuss some linguistic consequences in the semantic representations of spatial and locative information based on this work.

This paper explores the development of a two and one-half dimensional (2 1/2 -D) map structure to provide an autonomous mobile robot with a more three-dimensional (3-D) model of its environment than those afforded by current map structures. The 2 1/2 -D map structure was created by modifying the widely used evidence grid to store a height, along with a probability value, in each cell location to record the varying elevations of a 3-D environment. Results show that this map structure is capable of providing an autonomous mobile robot with a representation of a limited 3-D environment that will allow it to perform obstacle detection, path planning, and to an extent, localization.

In our research on intuitive means for humans and intelligent, mobile robots to collaborate, we use a multimodal interface that supports speech and gestural inputs. As a preliminary step to evaluate our approach and to identify practical areas for future work, we conducted a wizard-of-Oz pilot study with five participants who each collaborated with a robot on a search task in a separate room. The goal was to find a sign in the robot's environment with the word "FOO" printed on it. Using a subset of our multimodal interface, participants were told to direct the collaboration. As their subordinate, the robot would understand their utterances and gestures, and recognize objects and structures in the search space. Participants conversed with the robot through a wireless microphone and headphone and, for gestural input, used a touch screen displaying alternative views of the robot's environment to indicate locations and objects.

In conversation, people often use spatial relationships to describe their environment, e.g., "There is a desk in front of me and a doorway behind it", and to issue directives, e.g., "Go around the desk and through the doorway." In our research, we have been investigating the use of spatial relationships to establish a natural communication mechanism between people and robots, in particular, for novice users. In this paper, the work on robot spatial relationships is combined with a multimodal robot interface developed at the Naval Research Lab. We show how linguistic spatial descriptions and other spatial information can be extracted from an evidence grid map and how this information can be used in a natural, human-robot dialog.

The major themes of our research include the creation of mobile robot systems that are robust and adaptive in rapidly changing environments and the view of integration as a basic research issue. Where reasonable, we try to use the same representations to allow different components to work more readily together and to allow better and more natural integration of and communication between these components. In this paper, we describe our most recent work in integrating mobile robot exploration, localization, navigation, and planning through the use of a common representation, evidence grids.

: We suggest some novel variable ordering techniques based upon the notion of sampling. Such techniques can produce highly e#ective static variable orders, and can thus be employed in numerous problems where current static variable ordering techniques prove totally inadequate. They can also augmentvarious reordering techniques thereby helping to produce far superior variable orders in a comparable, or lesser, amount of time. Importantly,wehave been able to build BDDs of circuits which could not be represented previously using numerous other reordering packages. 1 Introduction Ordered Binary Decision Diagrams #3# have found extensive use in various algorithms for analyzing Boolean functions, and thus play an important role in numerous CAD related applications. Unfortunately, in many applications, very large sized BDDs can be generated which can render a BDD based analysis scheme impractical or ine#cient. In particular, OBDD sizes are very sensitive to the order chosen on input variabl...

Evidence grids provide a uniform representation for fusing temporally and spatially distinct sensor readings. However, the use of evidence grids requires that the robot be localized within its environment. Odometry errors typically accumulate over time, making localization estimates degrade, and introducing significant errors into evidence grids as they are built. We have addressed this problem by developing a method for "continuous localization", in which the robot corrects its localization estimates incrementally and on the fly. Assuming the mobile robot has a map of its environment represented as an evidence grid, localization is achieved by building a series of "local perception grids" based on localized sensor readings and the current odometry, and then registering the local and global grids. The registration produces an offset which is used to correct the odometry. Results are given on the effectiveness of this method, and quantify the improvement of continuous localization over dead reckoning. We also compare different techniques for matching evidence grids and for searching registration offsets.

Our research addresses how to integrate exploration and localization for mobile robots. A robot exploring and mapping an unknown environment needs to know its own location, but it may need a map in order to determine that location. In order to solve this problem, we have developed ARIEL, a mobile robot system that combines frontier based exploration with continuous localization. ARIEL explores by navigating to frontiers, regions on the boundary between unexplored space and space that is known to be open. ARIEL finds these regions in the occupancy grid map that it builds as it explores the world. ARIEL localizes by matching its recent perceptions with the information stored in the occupancy grid. We have implemented ARIEL on a real mobile robot and tested ARIEL in a real-world office environment. We present quantitative results that demonstrate that ARIEL can localize accurately while exploring, and thereby build accurate maps of its environment.

The tripod operator is a class of feature extraction operators for range images. It facilitates the recognition and/or localization of objects by extracting a few sparse point samples in a regimented way; n sampled surface points yield only n-3 scalar features containing all the surface shape information in these points and no other information. They have complete six degrees-of-freedom (DOF) isometry invariance. Isolated objects are recognized in noisy range images using a statistical treatment of a few isolated tripod operator placements. Experiments show fast, reliable recognition.>

The tripod operator, a class of feature extraction operators for range images which facilitate the recognition and localization of objects is described. It consists of three points in 3-space fixed at the vertices of an equilateral triangle and a procedure for making several scalar measurements in the coordinate frame of the triangle. The triangle is moved as a rigid body until the three vertices lie on the surface of some range image or modeled object. The resulting measurements are local shape features which are invariant under rigid motions. These features contain all the shape information in the surface points involved, and no other information. Tripod operators are applicable to all 3-D shapes. A cloud of points in feature space was generated for each object by random placement of the operator. Then new feature measurements were made by operator placements in a range image containing one of those objects. Using a simple nearest-neighbor approach, the authors determined which objects were rejected and which remained as recognition candidates. Experiments showed that tripod operators had excellent discriminating power.>