In cooperative training, humans within a team coordinate on complex tasks, building mental models of their teammates and learning to adapt to teammates’ actions in real-time. To reduce the often prohibitive scheduling constraints associated with cooperative training, this article introduces a paradigm for cooperative asynchronous training of human teams in which trainees practice coordination with autonomous teammates rather than humans. We introduce a novel experimental design for evaluating autonomous teammates for use as training partners in cooperative training. We apply this design to a human-subjects experiment where humans are trained with either another human or an autonomous teammate and are evaluated with a new human subject in a new, partially observable, cooperative game developed for this study. Importantly, we employ an unsupervised sequential clustering methodology to partition teammate trajectories from demonstrations performed in the experiment to form a smaller number of training conditions. This results in a simpler experiment design, enabling us to conduct a complex cooperative training human-subjects study in a reasonable amount of time. Through a demonstration of the proposed experimental design, we provide takeaways and design recommendations for future research in the development of cooperative asynchronous training systems utilizing robot surrogates for human teammates.

To effectively coordinate with human teammates in environments where direct communication is limited or unavailable, autonomous agents must be able to infer macro-level strategies from demonstrated micro-level actions in real time. This article introduces UNsupervised Behavior Inference from Action Sequences (UNBIAS): a modular framework for the unsupervised learning of behaviors from sequences of demonstrated human actions. UNBIAS relies on an unsupervised sequential encoder to learn representations of behaviors from sequences of human actions. Behavior representations are then clustered to obtain an unsupervised behavior classification. Real-time strategy prediction is performed using UNBIAS by training a classification layer to learn strategies’ decision boundaries in the behavior representation space and predict cluster labels from encoded incomplete action sequences. In extensive numerical experiments on two real-world datasets of human decision sequences, behavior classifications learned through the UNBIAS framework are shown to better match demonstrator identity groupings than partitions obtained with related unsupervised clustering algorithms, and high-quality online strategy inference is demonstrated. Thus, UNBIAS effectively enables the unsupervised learning of human behaviors from observed actions, enabling autonomous agents to better adapt to human teammates’ strategies in human-machine teams.

Collaborative robots and machine learning-based virtual agents are increasingly entering the human workspace with the aim of increasing productivity and enhancing safety. Despite this, we show in a ubiquitous experimental domain, Overcooked-AI, that state-of-the-art techniques for human-machine teaming (HMT), which rely on imitation or reinforcement learning, are brittle and result in a machine agent that aims to decouple the machine and human’s actions to act independently rather than in a synergistic fashion. To remedy this deficiency, we develop HMT approaches that enable iterative, mixed-initiative team development allowing end-users to interactively reprogram interpretable AI teammates. Our 50-subject study provides several findings that we summarize into guidelines. While all approaches underperform a simple collaborative heuristic (a critical, negative result for learning-based methods), we find that white-box approaches supported by interactive modification can lead to significant team development, outperforming white-box approaches alone, and that black-box approaches are easier to train and result in better HMT performance highlighting a tradeoff between explainability and interactivity versus ease-of-training. Together, these findings present three important future research directions: 1) Improving the ability to generate collaborative agents with white-box models, 2) Better learning methods to facilitate collaboration rather than individualized coordination, and 3) Mixed-initiative interfaces that enable users, who may vary in ability, to improve collaboration.