Despite advancements in human-robot teamwork, limited progress was made in developing AI assistants capable of advising teams online during task time, due to the challenges of modeling both individual and collective beliefs of the team members. Dynamic epistemic logic has proved to be a viable tool for representing a machine Theory of Mind and for modeling communication in epistemic planning, with applications to human-robot teamwork. However, this approach has yet to be applied in an online teaming assistance context and fails to account for the real-life probabilities of potential team beliefs. We propose a novel blend of epistemic planning and POMDP techniques to create a risk-bounded AI team assistant, that intervenes only when the team’s expected likelihood of failure exceeds a predefined risk threshold or in the case of potential execution deadlocks. Our experiments and simulated demonstration on the Virtualhome testbed show that the assistant can effectively improve team performance.

Conventional approaches to most image understanding problems suffer from fragility when applied to natural environments, where the complexity in the environment becomes overwhelming. Complexity in intelligent systems can be managed by breaking the world into manageable contexts. GRAVA is a reflective architecture that supports self-adaptation and has been successfully applied to a number of visual interpretation domains. The GRAVA architecture supports robust performance by treating changes in the program's environment as context changes. Automatically tracking changes in the environment and making corresponding changes in the running program allows the program to operate robustly. We describe the architecture and explain how it achieves robustness. In particular, we present an algorithm based on minimal description length (MDL) that permits contexts to be automatically induced from corpus training data. The algorithm does not require prior assignment of the number of contexts.

A vision-based virtual mouse interface is described that utilizes a robotic head, visual tracking of the users head and hand positions and recognition of user hand signs to control an intelligent kiosk. The user interface supports, among other things, smooth control of the mouse pointer and buttons using hand signs and movements. The algorithms and architecture of real-time vision and robot controller are described.