This paper reports on a study of human participants with a robot designed to participate in a collaborative conversation with a human. The purpose of the study was to investigate a particular kind of gestural feedback from human to the robot in these conversations: head nods. During these conversations, the robot recognized head nods from the human participant. The conversations between human and robot concern demonstrations of inventions created in a lab. We briefly discuss the robot hardware and architecture and then focus the paper on a study of the effects of understanding head nods in three different conditions. We conclude that conversation itself triggers head nods by people in human-robot conversations and that telling participants that the robot recognizes their nods as well as having the robot provide gestural feedback of its nod recognition is effective in producing more nods.

Head pose and gesture offer several key conversational grounding cues and are used extensively in face-to-face interaction among people. We investigate how dialog context from an embodied conversational agent (ECA) can improve visual recognition of user gestures. We present a recognition framework which (1) extracts contextual features from an ECA's dialog manager, (2) computes a prediction of head nod and head shakes, and (3) integrates the contextual predictions with the visual observation of a vision-based head gesture recognizer. We found a subset of lexical, punctuation and timing features that are easily available in most ECA architectures and can be used to learn how to predict user feedback. Using a discriminative approach to contextual prediction and multi-modal integration, we were able to improve the performance of head gesture detection even when the topic of the test set was significantly different than the training set.

This paper reports on a study of human subjects with a robot designed to mimic human conversational gaze behavior in collaborative conversation. The robot and the human subject together performed a demonstration of an invention created at our laboratory; the demonstration lasted 3 to 3.5 minutes. We briefly discuss the robot architecture and then focus the paper on a study of the effects of the robot operating in two different conditions. We offer some conclusions based on the study about the importance of engagement for 3D IUIs. We will present video clips of the subject interactions with the robot at the conference.

This paper reports on research on developing the ability for robots to engage with humans in a collaborative conversation for hosting activities. It defines the engagement process in collaborative conversation. The paper then presents the analysis of a study of human-human "look tracking" and discusses rules that allow a robot to track and fail to track humans so that engagement is maintained.

We present a derivation for a spline smoother which takes into account local velocity information. This smoother is well suited for finding a best-fit trajectory from multiple example trajectories and is thus useful in applications such as programming by demonstration and online gesture recognition for teleoperation. Currently available smoothers are designed to consider only position information and not local velocity information, and are thus less suited for smoothing trajectories over time of dynamic systems.

In this paper, we present a method for high-level control of robots whose low-level software is based on dynamically reconfigurable, reusable real-time software modules. Our approach is to use an embedded interpreter for a general-purpose programming language to direct the operation of the low-level modules toward meeting the task-level goals of the robot. To this end, we present RSK, a virtual-machine kernel implementing a scheme interpreter capable of hard real-time operation, and employing a method of code execution we call "message-based evaluation" (MBE). MBE is a novel combination of a traditional code execution model and a message-passing architecture, which simplifies the process of writing code for managing the robot's reconfigurable subsystem.

Despite the large amount of research currently directed toward programming robots by demonstration, a significant problem with this method of human-to-robot skill transfer has not yet been addressed: developing representations of human performances which isolate the intrinsic dimensions of the performances (and thus the skills which guide them) within high-dimensional, raw human performance data. In this paper we propose the use of three methods for representing high-dimensional human performance data within lower-dimensional spaces: principal component analysis (PCA), nonlinear principal component analysis (NLPCA), and sequential nonlinear principal component analysis (SNLPCA). We compare the appropriateness of these methods for modeling a simple human grasping operation.

We have developed a gesture recognition system, based on hidden Markov models, which can interactively recognize gestures and perform online learning of new gestures. In addition, it is able to update its model of a gesture iteratively with each example it recognizes. This system has demonstrated reliable recognition of 14 different gestures after only one or two examples of each. The system is currently interfaced to a Cyberglove for use in recognition of gestures from the sign language alphabet. The system is being implemented as part of an interactive interface for robot teleoperation and programming by example.

We present a novel mobile manipulator concept called the dual-use mobile detachable manipulator, or (DM)/sup 2/, for early construction and maintenance tasks in lunar stations. The robot consists of a wheeled rover, or mobile base and a detachable manipulator arm. The arm is symmetric, with a gripper at each end. When the arm attaches to the mobile base by grasping a handle with one of its grippers, the robot becomes a mobile manipulator and can perform exploration tasks such as collecting soil samples, surveying the lunar surface, and transporting tools and supplies. When the robot nears a lunar center structure such as a manufacturing center or a fuel tank, the manipulator arm can detach from the base and walk hand-over-hand, by grasping a series of handles on the structure, to perform tasks such as structure inspection, parts delivery, and simple assembly tasks. The paper discusses the concept and its advantages, the system under development, and its software architecture.

Underactuated manipulators are a class of robotic mechanisms where passive joints are present. By controlling only the motion of the active joints, it is possible to control the entire system. Our goal is to develop control schemes using both classical nonlinear and modem learning techniques for underactuated manipulators. To examine the validity of the approaches, we developed an experimental setup known as U-ARM, or underactuated robot manipulator. In this paper we present the hardware development, dynamic parameters, control software and experimental results of real-time control of the U-ARM.