The game of Codenames has recently emerged as a domain of interest for intelligent agent design, combining language and teammate coordination in a unique way. Previous approaches utilized a single internal language model, which performs well with similar teammates and poorly with others, as the agent cannot adapt to any specific teammate. In this paper we present the first adaptive meta-agent for Codenames. We adopt an ensemble approach, which determines which of a set of internal expert agents, each potentially with its own language model, is the best match for the current teammate. As part of this meta-agent, we propose a novel numerical metric to evaluate the performance of a Codenames team. We then propose an ensemble meta-agent which selects an internal expert on each turn in order to maximize this proposed metric. Experimental analysis shows successful adaptation to individual teammates, often performing nearly as well as the best internal expert with a teammate. Crucially, this success does not depend on any previous knowledge about the teammates, the ensemble agents, or their compatibility. This research represents an important step to making language-based agents for cooperative language settings like Codenames more adaptable to individual teammates.

In two-player cooperative games, agents can play together effectively when they have accurate assumptions about how their teammate will behave, but may perform poorly when these assumptions are inaccurate. In language games, failure may be due to disagreement in the understanding of either the semantics or pragmatics of an utterance. We model coarse uncertainty in semantics using a prior distribution of language models and uncertainty in pragmatics using the cognitive hierarchy, combining the two aspects into a single prior distribution over possible partner types. Fine-grained uncertainty in semantics is modeled using noise that is added to the embeddings of words in the language. To handle all forms of uncertainty we construct agents that learn the behavior of their partner using Bayesian inference and use this information to maximize the expected value of a heuristic function. We test this approach by constructing Bayesian agents for the game of Codenames, and show that they perform better in experiments where semantics is uncertain.

As Artificial Intelligence (AI) agents become more prevalent in a wider range of settings, it will become increasingly important for these agents to cooperate effectively with arbitrary teammates – including human agents. The game of Codenames has received interest in recent years from the AI and games communities as a fascinating testbed for exploring topics of cooperation with and adaptation to unknown teammates, all under natural language communication constraints. Evaluation of previous adaptive Codenames agents has been done between AI agents using different language models or strategies, with no exploration yet of how these adaptive approaches work with human teammates. In fact, a good understanding of how well basic non-adaptive Codenames agents perform with humans is also lacking. In this work we describe an experiment that was conducted to evaluate the effectiveness of both basic and adaptive Codenames AIs at cooperating with human teammates. Our results show that the adaptive agent does perform significantly better with humans, according to three of the four metrics that have been used in previous Codenames work. Interestingly, the human teammates expressed a preference for the basic agent over the adaptive agent, even while rating it better in several categories. This work lays the foundation for future experiments focusing on the cooperation of human and AI teammates, and also shows the importance of choosing correct metrics to align with human preferences in order to improve the effectiveness of AI agents at this important task.

Codenames is a cooperative game of constrained language communication. Success depends upon giving the best clues for your teammate, who may be unknown. Previous Codenames AI agents make the implicit assumption that their teammate is using the same language model. We show how team performance can often be improved if an agent also assumes that their teammate will only observe a noisy version of each clue. We propose and evaluate several methods for adapting the scale of this noise over time and show that by adapting the noise level we can significantly outperform agent teams that do not adapt. These results demonstrate a successful method for adapting language-model based agents so that they perform better with a variety of teammates.

In many real-world continuous action domains, human agents must decide which actions to attempt and then execute those actions to the best of their ability. However, humans cannot execute actions without error. Human performance in these domains can potentially be improved by the use of AI to aid in decision-making. One requirement for an AI to correctly reason about what actions a human agent should attempt is a correct model of that human’s execution error, or skill. Recent work has demonstrated successful techniques for estimating this execution error with various types of agents across different domains. However, this previous work made several assumptions that limit the application of these ideas to real-world settings. First, previous work assumed that the error distributions were symmetric normal, which meant that only a single parameter had to be estimated. In reality, agent error distributions might exhibit arbitrary shapes and should be modeled more flexibly. Second, it was assumed that the execution error of the agent remained constant across all observations. Especially for human agents, execution error changes over time, and this must be taken into account to obtain effective estimates. To overcome both of these shortcomings, we propose a novel particle-filter-based estimator for this problem. After describing the details of this approximate estimator, we experimentally explore various design decisions and compare performance with previous skill estimators in a variety of settings to showcase the improvements. The outcome is an estimator capable of generating more realistic, time-varying execution skill estimates of agents, which can then be used to assist agents in making better decisions and improve their overall performance.

Actions in most real-world continuous domains cannot be executed exactly. An agent’s performance in these domains is influenced by two critical factors: the ability to select effective actions (decision-making skill), and how precisely it can execute those selected actions (execution skill). This article addresses the problem of estimating the execution and decision-making skill of an agent, given observations. Several execution skill estimation methods are presented, each of which utilize different information from the observations and make assumptions about the agent’s decision-making ability. A final novel method forgoes these assumptions about decision-making and instead estimates the execution and decision-making skills simultaneously under a single Bayesian framework. Experimental results in several domains evaluate the estimation accuracy of the estimators, especially focusing on how robust they are as agents and their decision-making methods are varied. These results demonstrate that reasoning about both types of skill together significantly improves the robustness and accuracy of execution skill estimation. A case study is presented using the proposed methods to estimate the skill of Major League Baseball pitchers, demonstrating how these methods can be applied to real-world data sources.

The game of Codenames has attracted attention recently as it combines natural language and cooperation in a unique way. Codenames is a clue-giving game, where one teammate gives a clue to their partner, who then has to guess from a set of board words, given the clue. To date, the development of computational agents for Codenames has focused on the language aspects of the game. All clue-giving and guessing AI strategies explored thus far are static, basing their decisions only on the current state, without utilizing information obtained from previous clues and guesses. In this paper we present a preliminary exploration of the strategic aspects of agents for Codenames. We describe a deductive agent hierarchy, where agents utilize teammate models to make their decisions. Existing AI strategies for Codenames can be seen as occupying the bottom level of this hierarchy, and we describe general strategies for higher level agents. To explore these strategic ideas we also introduce a novel abstraction of Codenames called Codenums, which which can be seen as Codenames with numbers instead of words. We experimentally evaluate level 1 hierarchical agents in Codenums, demonstrating the potential for higher level strategic reasoning to influence agent design in Codenames.

The performance of agents in many domains with continuous action spaces depends not only on their ability to select good actions to execute, but also on their ability to execute planned actions precisely. This ability, which has been called an agent’s execution skill, is an important characteristic of an agent which can have a significant impact on their success. In this paper, we address the problem of estimating the execution skill of an agent given observations of that agent acting in a domain. Each observation includes the executed action and a description of the state in which the action was executed and the reward received, but notably excludes the action that the agent intended to execute. We previously introduced this problem and demonstrated that estimating an agent’s execution skill is possible under certain conditions. Our previous method focused entirely on the reward that the agent received from executed actions and assumed that the agent was able to select the optimal action for each state. This paper addresses the execution skill estimation problem from an entirely different perspective, focusing instead on the action that was executed. We present a Bayesian framework for reasoning about action observations and show that it is able to outperform previous methods under the same conditions. We also show that the flexibility of this framework allows it to be applied in settings where the previous limiting assumptions are not met. The success of the proposed method is demonstrated experimentally in a toy domain as well as the domain of computational billiards.

The Model AI Assignments session seeks to gather and disseminate the best assignment designs of the Artificial Intelligence (AI) Education community. Recognizing that assignments form the core of student learning experience, we here present abstracts of ten AI assignments from the 2019 session that are easily adoptable, playfully engaging, and flexible for a variety of instructor needs. Assignment specifications and supporting resources may be found at http: //modelai.gettysburg.edu.

This is a Work in Progress Research to Practice Category paper. Research has shown that novice programmers struggle with learning introductory concepts and find it difficult to monitor their own progress. Teachers often have hundreds of students and multiple sections of programming courses to teach, making it infeasible to provide the amount of independent feedback each student may need to flourish. With limited instructor feedback, students who can self-monitor and self-assess their programming metacognition have a higher chance of developing a process for solving programming challenges. In this paper, we expand on the literate programming paradigm by using natural language processing and machine learning methods to automatically analyze and classify student programming metacognition levels through their source code comments. Our intent is to ultimately integrate our classification models into an interactive developer environment to provide real-time feedback to students about their metacognition while learning to program.

Games with continuous state and action spaces present unique challenges from an artificial intelligence (AI) viewpoint. Billiards, or pool, is one such domain that has been the focus of several research efforts aimed at designing AI agents to play successfully. Due to the continuous nature of the actions, it is natural to believe that the more time an agent has to investigate actions, the better it will perform. This paper gives a thorough description of a successful agent with a novel distributed architecture, designed for being able to grant further time for shot simulation and analysis through the utilization of many CPUs. A brief analysis of the distributed component of the agent is presented, as well as how much the extra time thus obtained contributed to its success, especially when compared to its other novel components. The described agent, CueCard, won the Computer Olympiad computational pool tournament held in 2008.

Collusion is the practice of two or more parties deliberately cooperating to the detriment of others. While such behavior may be desirable in certain circumstances, in many it is considered dishonest and unfair. If agents otherwise hold strictly to the established rules, though, collusion can be challenging to police. In this paper, we introduce an automatic method for collusion detection in sequential games. We achieve this through a novel object, called a collusion table, that captures the effects of collusive behavior, i.e., advantage to the colluding parties, without assuming any particular pattern of behavior. We show the effectiveness of this method in the domain of poker, a popular game where collusion is prohibited.

When a robotic team is deployed to provide sensing and communication support for human activities, it is crucial that the robotic team be able to not only react to the current perceived location of the human of interest, but also predict his/her future locations. In this work we present a novel approach to predicting human motion in indoor environments. Our approach consists of an offline pre-processing phase, in which we analyze a map of the indoor environment and identify distinct areas of interest, such as rooms, and a run-time phase, in which we maintain hypotheses of destinations and probabilistically update these hypotheses based on observations of the human's motion. Our approach creates a probability distribution over future locations for the human user at each time step in the future. Such predictions can be used for the robotic team to effectively track and relay information from the human user to the base station.

We study repeated games in which players have imperfect execution skill and one player’s true skill is not common knowledge. In these settings the possibility arises of a player “hustling”, or pretending to have lower execution skill than they actually have. Focusing on repeated zero-sum games, we provide a hustle-proof strategy; this strategy maximizes a player’s payoff, regardless of the true skill level of the other player. 1