Accurate recognition of group behaviors is essential to the design of engaging networked multiplayer games. However, contemporary data-driven machine learning solutions are difficult to apply during the game development process, given that no authentic gameplay data is yet available for use as training data. In this paper, we investigate the use of synthetic training data, i.e., gameplay data that is generated by AI-controlled agent teams programmed to perform each of the behaviors to be recognized in groups of human players. The particular task we focus on is to recognize group movement formations in player-controlled avatars in a realistic virtual world. We choose five typical military team movement patterns for the formation recognition task and train machine learning models using procedurally generated unit trajectories as training data. The experiments were conducted using ResNet and EfficientNet, which are two popular convolutional neural network architectures for image classifications. The synthetic data is augmented by creating variations in image rotation, unit spacing, team size, and positional perturbations to bridge the gap between synthetic and human gameplay data. We demonstrate that high-accuracy behavior recognition can be achieved using deep neural networks by applying the aforementioned data augmentation methods to simulated gameplay data.

Intelligent user interfaces are finding new applications in interactive narratives, where players take on the role of a character in a fictional storyline. A recent example is the interactive audio narrative "Traveler", in which a combination of technologies for speech recognition and unsupervised text classification allow players to navigate a branching storyline via open-vocabulary spoken input. We hypothesize that the affordances of audio-based interaction in interactive narratives are different than text-based interaction, and that these differences change the player experience and their understanding of their fictional role. To test this hypothesis, we conducted a controlled experiment (n=39) to compare player interaction in "Traveler" with a text-only variant of the same storyline. We found significant differences in the types of input provided by players, suggesting that interaction modality impacts how players conceive of their relation to narrators of fictional storylines.

The ability to infer intentions, emotions, and other unobservable psychological states from people's behavior is a hallmark of human social cognition, and an essential capability for future Artificial Intelligence systems. The commonsense theories of psychology and sociology necessary for such inferences have been a focus of logic-based knowledge representation research, but have been difficult to employ in robust automated reasoning architectures. In this paper we model behavior interpretation as a process of logical abduction, where the reasoning task is to identify the most probable set of assumptions that logically entail the observable behavior of others, given commonsense theories of psychology and sociology. We evaluate our approach using Triangle-COPA, a benchmark suite of 100 challenge problems based on an early social psychology experiment by Fritz Heider and Marianne Simmel. Commonsense knowledge of actions, social relationships, intentions, and emotions are encoded as defeasible axioms in first-order logic. We identify sets of assumptions that logically entail observed behaviors by backchaining with these axioms to a given depth, and order these sets by their joint probability assuming conditional independence. Our approach solves almost all (91) of the 100 questions in Triangle-COPA, and demonstrates a promising approach to robust behavior interpretation that integrates both logical and probabilistic reasoning.

Health care providers continue to feel the pressure in providing adequate care for an increasing elderly population. If length of stay patterns for elderly patients in care can be captured through analytical modelling, then accurate predictions may be made on when they are expected to leave hospital. The Discrete Conditional Phase-type (DC-Ph) model is an effective technique through which length of stay in hospital can be modelled and consists of both a conditional and a process component. This research expands the DC-Ph model by introducing a survival tree as the conditional component, whereby covariates are used to partition patients into cohorts based on their distribution of length of stay in hospital. The Coxian phase-type distribution is then used to model the length of stay for patients belonging to each cohort. A demonstration of how patient length of stay may be predicted for new admissions using this methodology is then given. This tool has the benefit of providing an aid to the decision making processes undertaken by hospital managers and has the potential to result in the more effective allocation of hospital resources. Hospital admission data from the Lombardy region of Italy is used as a case-study.

People naturally anthropomorphize the movement of nonliving objects, as social psychologists Fritz Heider and Marianne Simmel demonstrated in their influential 1944 research study. When they asked participants to narrate an animated film of two triangles and a circle moving in and around a box, participants described the shapes' movement in terms of human actions. Using a framework for authoring and annotating animations in the style of Heider and Simmel, we established new crowdsourced datasets where the motion trajectories of animated shapes are labeled according to the actions they depict. We applied two machine learning approaches, a spatial-temporal bag-of-words model and a recurrent neural network, to the task of automatically recognizing actions in these datasets. Our best results outperformed a majority baseline and showed similarity to human performance, which encourages further use of these datasets for modeling perception from motion trajectories. Future progress on simulating human-like motion perception will require models that integrate motion information with top-down contextual knowledge.

Humans have a remarkable tendency to anthropomorphize moving objects, ascribing to them intentions and emotions as if they were human. Early social psychology research demonstrated that animated film clips depicting the movements of simple geometric shapes could elicit rich interpretations of intentional behavior from viewers. In attempting to model this reasoning process in software, we first address the problem of automatically recognizing humanlike actions in the trajectories of moving shapes. There are two main difficulties. First, there is no defined vocabulary of actions that are recognizable to people from motion trajectories. Second, in order for an automated system to learn actions from motion trajectories using machine-learning techniques, a vast amount of these action- trajectory pairs is needed as training data. This paper describes an approach to data collection that resolves both of these problems. In a web-based game, called Triangle Charades, players create motion trajectories for actions by animating a triangle to depict those actions. Other players view these animations and guess the action they depict. An action is considered recognizable if players can correctly guess it from animations. To move towards defining a controlled vocabulary and collecting a large dataset, we conducted a pilot study in which 87 users played Triangle Charades. Based on this data, we computed a simple metric for action recognizability. Scores on this metric formed a gradual linear pattern, suggesting there is no clear cutoff for determining if an action is recognizable from motion data. These initial results demonstrate the advantages of using a game to collect data for this action recognition task.

An effective means of retrieving relevant photographs from the web is to search for terms that would likely appear in the surrounding text in multimedia documents. In this paper, we investigate the complementary search strategy, where relevant multimedia documents are retrieved using the photographs they contain. We concentrate our efforts on the retrieval of large numbers of personal stories posted to Internet weblogs that are relevant to a particular search topic. Photographs are often included in posts of this sort, typically taken by the author during the course of the narrated events of the story. We describe a new story search tool, PhotoFall, which allows users to quickly find stories related to their topic of interest by judging the relevance of the photographs extracted from top search results. We evaluate the accuracy of relevance judgments made using this interface, and discuss the implications of the results for improving topic-based searches of multimedia content.

We investigate new interfaces that allow users to specify topics of interest in streams of weblog stories by providing relevance feedback to a search algorithm. Noting that weblog stories often contain photographs taken by the blogger during the course of the narrated events, we investigate whether these photographs can serve as a proxy for the whole post when users are making judgments as to the post's relevance. We developed a new story annotation interface for collecting relevance feedback with three variations: users are presented either with the full post as it appears in a weblog, an embedded photograph, or only the title of the post. We describe a user evaluation that compares annotation time, quality, and subjective user experience across each of these three conditions. The results show that relevance judgments based on embedded photographs or titles are far less accurate than when reading the whole weblog post, but the time required to acquire an accurate model of the user's topic interest is greatly reduced.

We describe Say Anything, a new interactive storytelling system that collaboratively writes textual narratives with human users. Unlike previous attempts, this interactive storytelling system places no restrictions on the content or direction of the user’s contribution to the emerging storyline. In response to these contributions, the computer continues the storyline with narration that is both coherent and entertaining. This capacity for open-domain interactive storytelling is enabled by an extremely large repository of nonfiction personal stories, which is used as a knowledge base in a case-based reasoning architecture. In this article, we describe the three main components of our case-based reasoning approach: a million-item corpus of personal stories mined from internet weblogs, a case retrieval strategy that is optimized for narrative coherence, and an adaptation strategy that ensures that repurposed sentences from the case base are appropriate for the user’s emerging fiction. We describe a series of evaluations of the system’s ability to produce coherent and entertaining stories, and we compare these narratives with single-author stories posted to internet weblogs.

The personal stories that people write in their Internet weblogs include a substantial amount of information about the causal relationships between everyday events. In this paper we describe our efforts to use millions of these stories for automated commonsense causal reasoning. Casting the commonsense causal reasoning problem as a Choice of Plausible Alternatives, we describe four experiments that compare various statistical and information retrieval approaches to exploit causal information in story corpora. The top performing system in these experiments uses a simple co-occurrence statistic between words in the causal antecedent and consequent, calculated as the Pointwise Mutual Information between words in a corpus of millions of personal stories.

This paper is a study of causation as it occurs in different domains and genres of discourse. There have been various initiatives to extract causality from discourse using causal markers. However, to our knowledge, none of these approaches have displayed similar results when applied to other styles of discourse. In this study we evaluate the nature of causal markers - specifically causatives, between corpora in different domains and genres of discourse and measure the overlap of causal markers using two metrics - Term Similarity and Causal Precision. We find that causal markers, specially causatives (causal verbs) are extremely domain dependent, and moderately genre dependent.

In the context of developing formal theories of commonsense psychology, or how peole think they think, we have developed a formal theory of goals. In it we explicate and axiomatize, among others, the goal-related notions of trying, success, failure, functionality, intactness, and importance.

We present an overview of the workshop on Common Sense Knowledge and Goal-Oriented Interfaces held at the 2008 Intelligent User Interfaces conference. Six papers were accepted from diverse research groups, each offering innovative new research on interfaces that incorporate common sense knowledge and that are oriented around the goals of their users.

Among the most interesting ways that people share knowledge is through the telling of stories, i.e. first-person narratives about real-life experiences. Millions of these stories appear in Internet weblogs, offering a potentially valuable resource for future knowledge management and training applications. In this paper we describe efforts to automatically capture stories from Internet weblogs by extracting them using statistical text classification techniques. We evaluate the precision and recall performance of competing approaches. We describe the large-scale application of story extraction technology to Internet weblogs, producing a corpus of stories with over a billion words.

Question-answering dialogue systems have found many applications in interactive learning environments. This paper is concerned with one such application for Army leadership training, where trainees input free-text questions that elicit pre-recorded video responses. Since these responses are already crafted before the question is asked, a certain degree of incoherence exists between the question that is asked and the answer that is given. This paper explores the use of short linking dialogues that stand in between the question and its video response to alleviate the problem of incoherence. We describe a set of experiments with human generated linking dialogues that demonstrate their added value. We then describe our implementation of an automated method for utilizing linking dialogues and show that these have better coherence properties than the original system without linking dialogues.

In this paper, we describe Fourth Frame Forums, an application that combines traditional four-frame comic strips with online web-based discussion forums. In this application, users are presented with a four-frame comic strip where the last dialogue balloon of the fourth frame is left blank. By typing a statement into this dialogue balloon, the user creates a new discussion thread in the forum, where the user's dialogue choice can be critiqued and discussed by other users of the forum. We argue that Fourth Frame Forums provide an elegant and cost-effective solution for online education and training environments for communities of learners. We provide examples from the domain of US Army leadership development, and compare Fourth Frame Forums to alternative methods of story-directed simulation and training.

While storytelling has long been recognized as an important part of effective knowledge management in organizations, knowledge management technologies have generally not distinguished between stories and other types of discourse. In this paper we describe a new type of technological support for storytelling that involves automatically capturing the stories that people tell to each other in conversations. We describe our first attempt at constructing an automated story extraction system using statistical text classification and a simple voting scheme. We evaluate the performance of this system and demonstrate that useful levels of precision and recall can be obtained when analyzing transcripts of interviews, but that performance on speech recognition data is not above what can be expected by chance. This paper establishes the level of performance that can be obtained using a straightforward approach to story extraction, and outlines ways in which future systems can improve on these results and enable a wide range of knowledge socialization applications.

Abstract To support browsing‐based subject access to image collections, it is necessary to provide users with networks of subject terms that are organized in an intuitive, richly interconnected manner. A principled approach to this task is to organize the subject terms by their relationship to activity contexts that are commonly understood among users. This article describes a methodology for creating networks of subject terms by manually representing a large number of common‐sense activities that are broadly related to image subject terms. The application of this methodology to the Library of Congress Thesaurus for Graphic Materials produced 768 representations that supported users of a prototype browsing‐based retrieval system in searching large, indexed photograph collections.

The ability to deliver appropriate information to learners at the most appropriate time is an essential component of good instruction. In the best learning environments, this information is received in the context of the performance of the skills that are being acquired. This paper explores a technological approach to situated information retrieval by linking materials to segments of a video recording a skill performance. An interface is described where users navigate through a video performance and are presented with information relevant to the current video location. An approach to algorithmically generating interfaces of this type is then presented. The system takes as input annotations that describe a video recording of a performance, translates these annotations into subject terms used to catalog information resources, and then retrieves materials from online database servers using the Z39.50 information retrieval protocol. As an example application, the system was used to generate online teacher professional development materials by linking annotated video of classroom teaching with resources cataloged in the ERIC database.

Article Free Access Share on Deja Vu: a knowledge-rich interface for retrieval in digital libraries Authors: Andrew S. Gordon The Institute for the Learning Sciences, Northwestern University, 1890 Maple Avenue, Evanston, IL The Institute for the Learning Sciences, Northwestern University, 1890 Maple Avenue, Evanston, ILView Profile , Eric A. Domeshek Alphatech, Inc., 50 Mall Road, Burlington, MA Alphatech, Inc., 50 Mall Road, Burlington, MAView Profile Authors Info & Claims IUI '98: Proceedings of the 3rd international conference on Intelligent user interfacesJanuary 1998 Pages 127–134https://doi.org/10.1145/268389.268412Published:01 January 1998Publication History 6citation465DownloadsMetricsTotal Citations6Total Downloads465Last 12 Months15Last 6 weeks2 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF