Natural language instructions are effective at tasking autonomous robots and for teaching them new knowledge quickly. Yet, human instructors are not perfect and are likely to make mistakes at times and will correct themselves when they notice errors in their own instructions. In this article, we introduce a complete system for robot behaviors to handle such corrections, during both task instruction and action execution. We then demonstrate its operation in an integrated cognitive robotic architecture through spoken language in two tasks: a navigation and retrieval task and a meal assembly task. Verbal corrections occur before, during, and after verbally taught sequences of tasks, demonstrating that the proposed methods enable fast corrections not only of the semantics generated from the instructions but also of overt robot behavior in a manner shown to be reasonable when compared to human behavior and expectations.

Transparent task-based communication between human instructors and robot instructees requires robots to be able to determine whether a human instruction can and should be carried out, i.e., whether the human is authorized, and whether the robot can and should do it. If the instruction is not appropriate, the robot needs to be able to reject it in a transparent manner by including its reasons for the rejection. In this article, we provide a brief overview of our work on natural language understanding and transparent communication in the Distributed Integrated Affect Reflection Cognition (DIARC) architecture and demonstrate how the robot can perform different inferences based on context to determine whether it should reject a human instruction. Specifically, we discuss four task-based dialogues and show videos of the interactions with fully autonomous robots that are able to reject human commands and provide succinct explanations and justifications for their rejection. The proposed approach can form the basis of further algorithmic developments for adapting the robot’s level of transparency for different interlocutors and contexts.

Turn-entry timing is an important requirement for conversation, and one that few spoken dialogue systems consider.In this paper we introduce a computational framework, based on work from psycholinguistics, which is aimed at achieving proper turn-entry timing for situated agents.Our approach involves incremental processing and lexical prediction of the turn in progress, which allows a situated dialogue agent to start its turn and initiate actions earlier than would otherwise be possible.We evaluate the framework by integrating it within a cognitive robotic architecture and testing performance on a corpus of situated, task-oriented human-robot directives.We demonstrate that: 1) the system is superior to a non-incremental system in terms of faster responses, reduced gap between turns, and the ability to perform actions early, 2) the system can time its turn to come in immediately at a turn transition, or earlier to produce several types of overlap, and 3) the system is robust to various forms of disfluency in the input.Overall, this domainindependent framework can be integrated into existing dialogue systems to improve responsiveness, and is another step toward more natural and fluid turn-taking behavior.

We present an approach to generating natural language justifications of decisions derived from norm-based reasoning.Assuming an agent which maximally satisfies a set of rules specified in an object-oriented temporal logic, the user can ask factual questions (about the agent's rules, actions, and the extent to which the agent violated the rules) as well as "why" questions that require the agent comparing actual behavior to counterfactual trajectories with respect to these rules.To produce natural-sounding explanations, we focus on the subproblem of producing natural language clauses from statements in a fragment of temporal logic, and then describe how to embed these clauses into explanatory sentences.We use a human judgment evaluation on a testbed task to compare our approach to variants in terms of intelligibility, mental model and perceived trust.

Expressive motion can situate a robot's attitude in its task motions, illustrating real-time reactions. Inspired by acting movement training, we construct path shape features that layer expression into a mobile robot's motion traversal. Our video-study results show that simple variations of path shape and orientation can influence human perceptions of a robot's task, focus, and confidence. We further find that sequencing path features is a useful way to create expressions that are pinpointed in time without requiring changes in velocity. Our quantitative features represent the Laban Space Effort: using path shape and orientation along the path to communicate the direct or indirect attitude of the robot toward its target destination (acting vocabulary italicized). These features illustrate expressive or stylistic aspects of the robot's inner state, filling a gap in the pre-existing literature that has mostly focused on task legibility. Our future work will evaluate temporal and spatial robot motion features in explicit interaction contexts.