This paper provides the first formalization of self-interested planning in multiagent settings using expectation-maximization (EM). Our formalization in the context of infinite-horizon and finitely-nested interactive POMDPs (I-POMDP) is distinct from EM formulations for POMDPs and cooperative multiagent planning frameworks. We exploit the graphical model structure specific to I-POMDPs, and present a new approach based on block-coordinate descent for further speed up. Forward filtering-backward sampling -- a combination of exact filtering with sampling -- is explored to exploit problem structure.

Recursive reasoning of the form what do I think that you think that I think (and so on) arises often while acting in multiagent settings. Previously, multiple experiments studied the level of recursive reasoning generally displayed by humans while playing sequential general-sum and fixed-sum, two-player games. The results show that subjects experiencing a general-sum strategic game display first or second level of recursive thinking with the first level being more prominent. However, if the game is made simpler and more competitive with fixed-sum payoffs, subjects predominantly attributed first-level recursive thinking to opponents thereby acting using second level. In this article, we model the behavioral data obtained from the studies using the interactive partially observable Markov decision process, appropriately simplified and augmented with well-known models simulating human learning and decision. We experiment with data collected at different points in the study to learn the models parameters. Accuracy of the predictions by our models is evaluated by comparing them with the observed study data, and the significance of the fit is demonstrated by comparing the mean squared error of our model predictions with those of a random hypothesis. Accuracy of the predictions by the models suggest that these could be viable ways for computationally modeling strategic behavioral data in a general way. While we do not claim the cognitive plausibility of the models in the absence of more evidence, they represent promising steps toward understanding and computationally simulating strategic human behavior.