This article reports on the development of capabilities for (on-screen) virtual agents and robots to support isolated older adults in their homes. A real-time architecture was developed to use a virtual agent or a robot interchangeably to interact via dialog and gesture with a human user. Users could interact with either agent on 12 different activities, some of which included on-screen games, and forms to complete. The article reports on a pre-study that guided the choice of interaction activities. A month-long study with 44 adults between the ages of 55 and 91 assessed differences in the use of the robot and virtual agent.

A collaborative robot needs to be able to regulate and manage shared goals during collaboration. Emotion has a crucial influence on this goal management process. In this paper, we provide a cost function that we use to choose the goal in the shared plan with the lowest cost value out of a set of alternative goals. This cost function is a) based on the goal attributes, b) with respect to the reverse appraisal of the perceived emotion, and c) the appraisal of the collaborative environment.

We have developed learning and interaction algorithms to support a human teaching hierarchical task models to a robot using a single demonstration in the context of a mixed-initiative interaction with bi-directional communication. In particular, we have identified and implemented two important heuristics for suggesting task groupings based on the physical structure of the manipulated artifact and on the data flow between tasks. We have evaluated our algorithms with users in a simulated environment and shown both that the overall approach is usable and that the grouping suggestions significantly improve the learning and interaction.

We have developed a systematic methodology for designing emotional facial expressions for humanoid robots, especially those with limited degrees of freedom. The methodology is firmly grounded in the psychological literature on human static and dynamic emotional facial expressions. We demonstrate the methodology by applying it to a recent humanoid robot and evaluate the results, confirming that the observed confusion matrix agrees qualitatively with the predictions of the methodology. We also investigate how robot facial emotion recognition compares for dynamic versus static expressions.

We report on an experiment in which a human collaborates with a small, autonomous, humanoid robotic toy. The experiment demonstrates that the robot's use of two simple gestures, namely orienting its head toward the addressee when it speaks and raising its arm in the direction of objects it refers to, significantly improve the human's perception of the robot's interaction skills and quality as a collaborator.

We have developed and tested a reusable Robot Operating System (ROS) module that supports engagement between a human and a humanoid robot by generating appropriate directed gaze, mutual facial gaze, adjacency pair and backchannel connection events. The module implements policies for adding gaze and pointing gestures to referring phrases (including deictic and anaphoric references), performing end-of-turn gazes, responding to human-initiated connection events and maintaining engagement. The module also provides an abstract interface for receiving information from a collaboration manager using the Behavior Markup Language (BML) and exchanges information with our previously developed engagement recognition module.

Based on a study of the engagement process between humans, we have developed and implemented an initial computational model for recognizing engagement between a human and a humanoid robot. Our model contains recognizers for four types of connection events involving gesture and speech: directed gaze, mutual facial gaze, conversational adjacency pairs and backchannels. To facilitate integrating and experimenting with our model in a broad range of robot architectures, we have packaged it as a node in the open-source Robot Operating System (ROS) framework. We have conducted a preliminary validation of our computational model and implementation in a simple human-robot pointing game.

This paper reports on a study of human participants with a robot designed to participate in a collaborative conversation with a human. The purpose of the study was to investigate a particular kind of gestural feedback from human to the robot in these conversations: head nods. During these conversations, the robot recognized head nods from the human participant. The conversations between human and robot concern demonstrations of inventions created in a lab. We briefly discuss the robot hardware and architecture and then focus the paper on a study of the effects of understanding head nods in three different conditions. We conclude that conversation itself triggers head nods by people in human-robot conversations and that telling participants that the robot recognizes their nods as well as having the robot provide gestural feedback of its nod recognition is effective in producing more nods.

Head pose and gesture offer several key conversational grounding cues and are used extensively in face-to-face interaction among people. We investigate how dialog context from an embodied conversational agent (ECA) can improve visual recognition of user gestures. We present a recognition framework which (1) extracts contextual features from an ECA's dialog manager, (2) computes a prediction of head nod and head shakes, and (3) integrates the contextual predictions with the visual observation of a vision-based head gesture recognizer. We found a subset of lexical, punctuation and timing features that are easily available in most ECA architectures and can be used to learn how to predict user feedback. Using a discriminative approach to contextual prediction and multi-modal integration, we were able to improve the performance of head gesture detection even when the topic of the test set was significantly different than the training set.

This paper reports on a study of human subjects with a robot designed to mimic human conversational gaze behavior in collaborative conversation. The robot and the human subject together performed a demonstration of an invention created at our laboratory; the demonstration lasted 3 to 3.5 minutes. We briefly discuss the robot architecture and then focus the paper on a study of the effects of the robot operating in two different conditions. We offer some conclusions based on the study about the importance of engagement for 3D IUIs. We will present video clips of the subject interactions with the robot at the conference.

This paper reports on research on developing the ability for robots to engage with humans in a collaborative conversation for hosting activities. It defines the engagement process in collaborative conversation. The paper then presents the analysis of a study of human-human "look tracking" and discusses rules that allow a robot to track and fail to track humans so that engagement is maintained.

To participate in conversations with people, robots must not only see and talk to people, but must also make use of the conventions of conversation and connection to their human counterparts. This paper reports on research on engagement in human-human interaction and applications to (non-autonomous) robots interacting with humans in hosting activities.

This paper reports on experiments with subjects who must learn to use a small artificially constructed subset language (of English) to interact with a conversational spoken language system. The subjects converse with a collaborative interface agent about tasks involving TV recording and schedule navigation. The subjects perform their tasks in two conditions, one with contextually appropriate help on what to say always available on the screen and those with only a help sheet that they request to see. Our experiments indicate that users can perform their tasks in either condition, but demonstrate limits in remembering the language in a subsequent session.

This paper discusses hosting activities. Hosting activities are a general class of collaborative activity in which an agent provides guidance in the form of information, entertainment, education or other services in the user's environment (which may be an artificial or the natural world) and may also request that the human user undertake actions to support the fulfillment of those services. This paper reports on experience in building a robot agent for hosting activities, both the architecture and applications being used. The paper then turns to a range of issues to be addressed in creating hosting agents, especially robotic ones. The issues include the tasks and capabilities needed for hosting agents, and social relations, especially human trust of agent hosts. Lastly the paper proposes a new evaluation metric for hosting agents.

This paper addresses the issue of designing embodied conversational agents that exhibit appropriate posture shifts during dialogues with human users. Previous research has noted the importance of hand gestures, eye gaze and head nods in conversations between embodied agents and humans. We present an analysis of human monologues and dialogues that suggests that postural shifts can be predicted as a function of discourse state in monologues, and discourse and conversation state in dialogues. On the basis of these findings, we have implemented an embodied conversational agent that uses Collagen in such a way as to generate postural shifts.

Computer tutors and intelligent software assistants have traditionally been thought of as different kinds of systems. However tutors and assistants share many properties. We have incorporated tutorial strategies into an intelligent assistant based on the COLLAGEN architecture. We are working on an agent, named Triton, which teaches and helps users with the graphical user interface of an air travel planning system. We found that the collaborative model underlying COLLAGEN is an excellent foundation for both an assistant and a tutor, and that both modes of interaction can be implemented in the same system with different parameter settings.

No abstract available.

No abstract available.

We have developed an application-independent toolkit, called Collagen, based on the SharedPlan theory of collaborative discourse, in which interaction histories are hierarchically structured according to a user´s goals and intentions. We have used Collagen to implement an example collaborative interface agent with discourse processing, but not natural language understanding. In this paper, we concentrate on how a segmented interaction history supports user orientation, intelligent assistance, and transformations, such as returning to earlier points in the problem solving process and replaying segments in a new context. Superseded by TR97-21.

Communication theories and technology have tended to focus on extended, formal meetings and have neglected a prevalent and vital form of workplace communication—namely, lightweight communication. Unlike formal, extended meetings, lightweight interaction is brief, informal, unplanned, and intermittent. We analyze naturalistic data from a study of work-place communication and derive five design criteria for lightweight interaction systems. These criteria require that systems for lightweight interaction support conversational tracking, rapid connection , the ability to leave a message, context management, and shared real-time objects . Using these criteria, we evaluate existing interpersonal communications technologies. We then describe an implementation of a system (TeleNotes) that is designed to support lightweight interaction by meeting these criteria. The interface metaphor allows communications to be based around desktop objects, resembling “sticky notes.” These objects are also organized into “desktop piles” to support conversational threads and provide mechanisms for initiating real-time audio, video, and application sharing. We conducted informal user testing of several system prototypes. Based on our findings, outstanding issues concerning theory and systems design for communication systems are outlined—in particular, with regard to the issue of managing conversations over time.

Article Free Access Share on Email overload: exploring personal information management of email Authors: Steve Whittaker Lotus Development Corp. Lotus Development Corp.View Profile , Candace Sidner Lotus Development Corp. Lotus Development Corp.View Profile Authors Info & Claims CHI '96: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsApril 1996 Pages 276–283https://doi.org/10.1145/238386.238530Published:13 April 1996Publication History 498citation11,623DownloadsMetricsTotal Citations498Total Downloads11,623Last 12 Months957Last 6 weeks131 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteView all FormatsPDF

Article Adding a collaborative agent to graphical user interfaces Share on Authors: Charles Rich Mitsubishi Electric MERL, 201 Broadway, Cambridge, MA Mitsubishi Electric MERL, 201 Broadway, Cambridge, MAView Profile , Candace L. Sidner Lotus Development Corporation, One Rogers Street, Cambridge, MA Lotus Development Corporation, One Rogers Street, Cambridge, MAView Profile Authors Info & Claims UIST '96: Proceedings of the 9th annual ACM symposium on User interface software and technologyNovember 1996 Pages 21–30https://doi.org/10.1145/237091.237094Online:01 November 1996Publication History 14citation548DownloadsMetricsTotal Citations14Total Downloads548Last 12 Months16Last 6 weeks0 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

In a discourse the hearer must recognize the response intended by the speaker. To perform this recognition, the hearer must ascertain what plans the speaker is undertaking and how the utterances in the discourse further that plan. To do so, the hearer can parse the initial intentions (recoverable from the utterance) and recognize the plans the speaker has in mind and intends the hearer to know about. This paper reports on a theory of parsing the intentions in discourse. It also discusses the role of another aspect of discourse, discourse markers, that are valuable to intended response recognition.

No abstract available.